How is the API request formatted for the user message along with the stored messages?

[marius-manola]

When making a request to OpenAI API, how does the request look like along with stored messages from the history?
Do those messages get attached to the user content?

role: assistant
content: "as it was"

role: user
content: "previous messages are added + the actual user input"

Do they get appended to the system prompt? Or are they sent as something separate?

[lgrammel]

How are you calling the OpenAI API? Can you provide a code snippet?

[marius-manola]

import {Configuration, OpenAIApi} from "openai-edge";
import {OpenAIStream, StreamingTextResponse} from "ai";



export const runtime = "edge";

const config = new Configuration({
    apiKey: process.env.OPENAI_API_KEY
})

const openai = new OpenAIApi(config)


export async function POST(request: Request){
    
    const{ messages } = await request.json();
    

    //create a chat completion
    const response = await openai.createChatCompletion({
        model: 'gpt-4',
        messages: [
            {role:"system", content: "You are a helpful assistant"},
            ...messages
        ],
        stream: true,
        
    })

    //create a stream of data from openai
    const stream = await OpenAIStream(response);

    //send the stream as a response to our client
    return new StreamingTextResponse(stream);
}

[marius-manola]

const {input, setInput, handleInputChange, handleSubmit, messages} = useChat(); 

[lgrammel]

You need to provide all messages here, nothing is stored (you need to keep the history yourself when you use openai chat, it is stateless).