How do you add stream.value to AIState in the server component?

[james-pratama]

Hi! The only way to immediately stream responses to the frontend seems to be to wrap the model call around the immediately invoked (async () =>, which will then return the stream.value and be ingested by readStreamableValue. However, readStreamableValue only works in the client.

Since AIState cannot be manipulated client side, how can the model's streamed responses also be accumulated and added to the AIState on the server?

Below are examples from the docs.

'use server';

import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { createStreamableValue } from 'ai/rsc';

export async function generate(input: string) {
  'use server';

  const stream = createStreamableValue('');

  (async () => {
    const { textStream } = await streamText({
      model: openai('gpt-3.5-turbo'),
      prompt: input,
    });

    for await (const delta of textStream) {
      stream.update(delta);
    }

    stream.done();
  })();

  return { output: stream.value };
}

'use client';

import { useState } from 'react';
import { generate } from './actions';
import { readStreamableValue } from 'ai/rsc';

export default function Home() {
  const [generation, setGeneration] = useState<string>('');

  return (
    <div>
      <button
        onClick={async () => {
          const { output } = await generate('Why is the sky blue?');

          for await (const delta of readStreamableValue(output)) {
            setGeneration(currentGeneration => `${currentGeneration}${delta}`);
          }
        }}
      >
        Ask
      </button>

      <div>{generation}</div>
    </div>
  );
}

[doureyd]

Hi, you can do it like this :

export async function generate(input: string) {
  'use server';

  // 1. Get a mutable reference to the AI state
  const aiState = getMutableAIState<typeof AI>()

  const stream = createStreamableValue('');

  (async () => {
    const { textStream } = await streamText({
      model: openai('gpt-3.5-turbo'),
      prompt: input,
    });

    // 2. Create a message variable
    let message = "";

    for await (const delta of textStream) {
      stream.update(delta);
      // 3. Update the message with the new delta
      message += delta;
    }

    // 4. Add the message to the AI state
    aiState.done([
      ...aiState.get(),
      {
        id: nanoid(),
        role: 'user',
        content: stream.value as string
      }
    ])

    stream.done();
  })();

  return { output: stream.value };
}

[james-pratama]

Thanks so much for this! I did realize this was the way to go too.