GPT-Plus Pack Document

Explore

Pack Document: GPT-Plus

Documentation

The

GPT-Plus⁠

Pack currently exposes five primary formulas: conversation_raw, chat_raw, gpt, gpt_raw and dalle. Some usage examples are below, this documentation is a work in progress until I hit v1.0.

GPT-Plus.conversation_raw

conversation_raw never hits a remote endpoint, it just returns a structured object that you can use to inform subsequent invocations to GPT-Plus.chat_raw if you’d like to maintain a consistent thread.

GPT-Plus.gpt

gpt is a mostly preconfigured invocation of GPT-3 with sensible defaults and the ability to override a few parameters.

The gpt formula has the following signature: gpt(prompt, user, temperature, size). A typical invocation will use some text prompt with User().Email as the user value, a value between 0 and 1 for temperature, and a size value ranging from 16 to up to 4000 for the current configuration.

Arguments

prompt is the input to the GPT engine. You can say something like “Tell me a fairytale about gremlins” or something like “Summarize the following row of data with respect to...” and get a reasonably good output. Your prompt can be fairly long but if it’s too complex you may get an empty string back if the system got confused.

user is a unique identifier to track the actual person submitting the prompt. This is for your protection as a document owner — if they are abusing the API and no user tracking is set then you may be held accountable.

temperature is a value that controls how “random” the output is. If temperature is 0 then you’ll get the same response to the same prompt every time. If temperature is 1 you’ll get very different responses each time. Try to find a value that works for your use case; this system uses .7 as a default value.

size is the number of tokens in the response — basically, the higher this number the longer the response it can generate, but the more you’ll be paying for it. Tune this by experimenting — the algorithm doesn’t really hit its stride until about 512, but you can go much higher. Just be careful, this is directly related to cost. A “token” is currently about 3-4 characters, so the pricing can be tricky to calculate.

Example

Given the following invocation:

GPT-Plus.gpt(<my user account>, "Why do you think Batman is so sad all the time and doesn't just go to therapy? Give me three possible reasons, and how I could help him.", "dick@wayne.org", 1, 1024)

I received the following response:

1. Batmans parents were killed in front of him when he was a child, and he has never been able to fully grapple with that trauma. I could help Batman by providing him with therapy and counseling to help him work through his grief and pain.

2. As Batman, Bruce Wayne has seen firsthand the dark side of humanity, and this can take a toll on someone over time. I could help Batman by providing him with support and encouragement to continue fighting for justice despite the odds.

3. Bruce Wayne is constantly putting himself in danger as Batman, and this can lead to feelings of isolation and loneliness. I could help Batman by being someone he can confide in, whether it's about his crime-fighting exploits or anything else going on in his life.

GPT-Plus.gpt_raw

Unlike gpt, gpt_raw exposes most of the underlying parameters from the GPT-3 API. This is powerful and allows you to tune the performance of your query in some powerful and interesting ways.

Note that not all parameters are mapped as arguments — GPT-3’s API supports a few things that this Pack is not currently capable of supporting, like live-streaming responses back.

Arguments (* denotes required)

prompt* is the input against which to generate completions, encoded as a string, array of strings, array of tokens, or array of token arrays. Note that <|endoftext|> is the document separator that the model sees during training, so if a prompt is not specified the model will generate as if from the beginning of a new document. See the

docs⁠

for more information.

user* is a unique identifying string capturing the ID of your user who is invoking GPT, used for security purposes. Use something like the currently logged in user's email address.

model allows you to specify an alternative model to run your prompt against. Different models have different use cases and settings, so only tweak this if you know what you’re doing.

suffix is text to append to the end of your query. See

docs⁠

for more information.",

max_tokens relates to the size of the generated response. Higher values give longer, better results -- but use up more tokens. The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model's context length. Most models have a context length of 2048 tokens (except for the newest models, which support 4096). See

docs⁠

for more info.

temperature is the sampling temperature. Higher values means the model will take more risks. Try 0.9 for more creative applications, and 0 (argmax sampling) for ones with a well-defined answer. We generally recommend altering this or top_p but not both. See

docs⁠

to learn more.",

top_p is an alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both. See the

docs⁠

to learn more.

echo is a boolean flag specifying whether or not to echo back the prompt in addition to the completion. See the

docs⁠

stop can contain up to four sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence. See

docs⁠

to learn more.

presence_penalty is a number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. See

docs⁠

frequency_penalty is a number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. See

docs⁠

best_of is the number of completions to generate before selecting the ‘best’ one (the one with the highest log probability per token). Results cannot be streamed (which is cool cuz we don’t do that here anyway — Pack Author). Note: Because this parameter generates many completions, it can quickly consume your token quota. Use carefully and ensure that you have reasonable settings for max_tokens and stop . See

docs⁠

logit_bias modifies the likelihood of specified tokens appearing in the completion. Accepts an array of strings in the format "<token>:<value>" that map tokens (specified by their token ID in the GPT tokenizer) to an associated bias value from -100 to 100. You can use this tokenizer tool (which works for both GPT-2 and GPT-3) to convert text to token IDs. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token. As an example, you can pass ["50256:-100"] to prevent the <|endoftext|> token from being generated. See the

docs⁠

for more details, but note that our input format in Coda differs from the object notation the raw API normally takes!

Examples

Using the default parameters it may be tempting to treat this like a passthrough to gpt, but I used the OpenAI defaults for each parameter. That means that when you run a formula like this:

gpt_raw(<My user account>, "Why do you think Superman is so sad all the time and doesn't just go to therapy? Give me three possible reasons, and how I could help him.", "dick@wayne.org")

You get a response like this:

1. Superman may be sad because he feels like he needs to shoulder

This is because OpenAI’s default max_tokens value (which I called size in gpt but expose under its true name in gpt_raw) defaults to 16 here not the more sensible 256 or 512.

A more robust invocation of gpt_raw may look something like this:

gpt_raw([my user], "Why do you think Superman is so sad all the time and doesn't just go to therapy? Give me three possible reasons, and how I could help him.", "dick@wayne.org", max_tokens: 1024,temperature: 1, presence_penalty: .8, frequency_penalty: .8, best_of:3)

The above query generated the following response for me:

1. Civilization Rejection: Superman may feel like he doesn't fit in with the rest of humanity because he is so powerful and they are not. He may feel like he can never truly connect with anyone because of this difference in ability, leading to loneliness and sadness. I could help Superman by talking to him about his feelings of difference and helping him find ways to connect with others despite these differences.

2. Fear of Loss: Given how powerful Superman is, he may often worry about losing those he loves due to his own abilities. He may be afraid that his loved ones will be hurt or killed because of his powers, leading to a great deal of sadness and anxiety. I could help Superman by talking to him about his fears and helping him find ways to cope with them.

3. Guilt: Given all the good that Superman does, he may often feel guilty about the bad that happens in the world. He may feel like he could have done more to prevent tragedies from occurring, leading to a great deal of sadness and regret. I could help Superman by talking to him about his guilt and helping him find ways to forgive himself for things that are beyond his control.

GPT-Plus.dalle

OpenAI’s other offering is DALL-E, a powerful tool for generating images from text. It has certain limitations right now — you can’t control the shape, it has to be a square for instance. But the capabilities are astonishing, and it’s always getting better.

Arguments

DALL-E’s API is a lot simpler than GPT-3’s for now — it just wants three arguments, prompt, user and size.

prompt is your image prompt, something like “A happy puppy playing on the moon” or “a bustling office scene”. Have fun with this!

user is a unique identifying string capturing the ID of your user who is invoking GPT, used for security purposes. Use something like the currently logged in user's email address.

size must be either the string “small”, the string “medium” or the string “large”. These map under the hood to ”256x256”, ”512x512” and ”1024x1024", which are the only three currently supported resolutions.

Examples

This API returns a URL to the generated image.

dalle(<my user>, "A happy puppy playing fetch on the moon with a chill alien", "cosmo@ussr.org", "large")

Generated this delightful image:

⁠

This link can't be embedded.

⁠

GPT-Plus.conversation_raw