using stop sequences with gpt-4

How to Use Stop Sequences With GPT

by

Stop Sequences are an essential part of working with the GPT API, allowing you to control when the model stops generating text. This technical guide will help users with little coding experience understand and implement Stop Sequences in various scenarios.

Using Stop Sequences in the OpenAI GPT Playground

In the GPT Playground, Stop Sequences control the model’s output and terminate it at a desired point, such as the conclusion of a sentence or list. The return key is commonly used as an effective Stop Sequence for single-line completions.

As a configurable option within the GPT Playground, Stop Sequences signal when to cease generating tokens and permit users to input up to four stop sequences. If no specific sequence is provided, the default “endoftext” token is used, signifying a potential stopping point within the text.

In the GPT Playground, Stop Sequences are configurable options that dictate when the model should stop generating tokens, with the default being the “endoftext” token.

What are Stop Sequences?

Stop Sequences are used to make the model stop generating text at a desired point, such as the end of a sentence or a list. The Stop Sequence is an optional setting to pass to the API. You can provide up to four different stop sequences. If none are passed, it defaults to the token “.”, which represents a probable stopping point in the text.

using stop sequences gpt 4

Using Stop Sequences in Chat Conversations

In chat examples, three Stop Sequences are often used: A new line, the value “Human:”, and the value “AI:”. These stop sequences ensure that:

  1. The completion can’t go to a new line.
  2. The completion can’t change the speaker.
  3. The completion won’t allow the speaker to speak twice in a row.

This approach generates only a single line of text corresponding to the current speaker.

Using Stop Sequences in Q&A Scenarios

In Q&A examples, the pattern of separating each Q&A pair with a new line makes the return key (new line) a suitable stop sequence. By using the return key as the stop sequence, the completion will stop after a single line, which follows “A:”.

stop sequences gpt playground

Using Stop Sequences for Lists

Stop Sequences can also be used to generate lists with a specific number of items. For instance, by using “11.” as a stop sequence, you can generate a list with 10 items, as the completion will stop when “11.” is reached.

Implementing Stop Sequences Using OpenAI API for GPT

When using the GPT API, you can pass the stop sequences as an array within the API call. Here’s a simple example in Python:

pythonCopy codeimport openai

openai.api_key = "your-api-key"

response = openai.Completion.create(
    engine="davinci-codex",
    prompt="Q: What is the capital of France?",
    max_tokens=50,
    n=1,
    stop=["\n"],
)

print(response.choices[0].text.strip())

In this example, the stop sequence is passed as an array containing a new line character ("\n"), ensuring that the generated answer stops after a single line.

Examples of how to use stop sequences with GPT

stop sequences in gpt playground

Use Stop Sequences For Q&A Prompts in the GPT Playground

Stop Sequences in Q&A Prompts

In Q&A examples, the pattern of separating each Q&A pair with a new line makes the return key (new line) a suitable stop sequence. By using the return key as the stop sequence, the completion will stop after a single line, which follows “A:”.

Use Stop Sequences For Creating Lists

gpt stop sequences for lists

If you want GPT to generate a list of items, you can set a specific item or a pattern as the stop sequence. Remember, the stop sequences should be chosen carefully and thoughtfully, as they can significantly influence the generated output.

Use GPT Stop Sequences For Chat Prompts

using gpt stop sequences for chat prompts

In chat examples, three Stop Sequences are often used: A new line, the value “Human:”, and the value “AI:”. These stop sequences ensure that: The completion can’t go to a new line, completion can’t change the speaker, completion won’t allow the speaker to speak twice in a row. This approach generates only a single line of text corresponding to the current speaker.

  1. Story ending

If you want GPT to generate a short story that ends with “The End”, you can set the stop sequence as “The End”. GPT will generate a story and stop when it reaches “The End”.

pythonCopy codeprompt = "Once upon a time in a small village, there lived a young girl named Lucy."
stop_sequence = "The End"

story = gpt.generate(prompt, stop_sequence=stop_sequence)
print(story)
  1. Email signature

When generating an email, you may want the model to stop after adding an email signature. You can set the stop sequence as “Best regards,” or “Sincerely,”.

pythonCopy codeprompt = "Dear John, I wanted to discuss our upcoming project."
stop_sequence = "Best regards,"

email = gpt.generate(prompt, stop_sequence=stop_sequence)
print(email)
  1. Specific question

If you’re looking for an answer to a particular question, you can use the question itself as a stop sequence.

pythonCopy codeprompt = "What are the benefits of exercising regularly?"
stop_sequence = "?"

answer = gpt.generate(prompt, stop_sequence=stop_sequence)
print(answer)
  1. Time-based events

When generating a text that describes events in chronological order, you can set the stop sequence to a specific date or time.

pythonCopy codeprompt = "Provide a brief history of the internet from its inception."
stop_sequence = "by the year 2000."

history = gpt.generate(prompt, stop_sequence=stop_sequence)
print(history)
  1. List generation

If you want GPT to generate a list of items, you can set a specific item or a pattern as the stop sequence.

pythonCopy codeprompt = "List the first five planets of our solar system:"
stop_sequence = "5. Jupiter"

planets = gpt.generate(prompt, stop_sequence=stop_sequence)
print(planets)

Remember, the stop sequences should be chosen carefully and thoughtfully, as they can significantly influence the generated output.

Understanding and implementing Stop Sequences is crucial when working with GPT, as they allow you to control the generated text output. With this guide, even users with little coding experience should be able to use Stop Sequences effectively in different scenarios.