[BUGFIX] - useAssistant - Check for empty array

[tchanxx]

When I was using the useAssistant hook, I noticed this error: Unhandled Runtime Error TypeError: Cannot set properties of undefined (setting 'id')

This error was kind of cryptic since it doesn't say which element in useAssistant is causing this error. However, I figured out that it's due to lastMessage being undefined in the following code:

const lastMessage = messages[messages.length - 1];
  return [
    ...messages.slice(0, messages.length - 1),
    {
      id: lastMessage.id,
      role: lastMessage.role,
      content: lastMessage.content + value,
    },
  ];

When messages has a length of 0, then lastMessage becomes undefined and then we get a runtime error at lastMessage.id.

[lgrammel]

There should always be a message at the point when text fragments start streaming in, so the root cause could be different. Can you share the code that caused this bug? One potential cause could be using setMessages while there is ongoing streaming.

[lgrammel]

Marking as on hold for now. The fix most likely needs to be more involved.

[tchanxx]

One potential cause could be using setMessages while there is ongoing streaming.

@lgrammel Ahh that's a good point. It's hard for me to share the entire code, but basically I have a setup similar to below:

useEffect(() => {
  if (selectedThread) {
    fetchMessages()
  } else {
    setChatMessages([])
  }
}, [selectedThread, setChatMessages])


const handleSendMessage = async () => {
  let currentThread = selectedThread ? { ...selectedThread } : null

  if (!currentThread) {
    currentThread = await handleCreateThread(
      user.id,
      userInput,
      chatSettings!,
      selectedWorkspace!,
      setSelectedThread,
      setThreads
    )
  }

  await submitMessage()
}

I think my issue is that when I create a new thread in handleSendMessage(), I create a new thread and then call setSelectedThread() which triggers the useEffect() hook to clear my messages via setChatMessages([]) but those are all asynchronous calls so setChatMessages([]) may happen during await submitMessage().

I can reconfigure my app so that it relies on the thread creation in the useAssistant() hook instead of creating it separately which should resolve this issue.

---

I am curious though, in this useAssistant() code, how does it ensure that there's always a message in the array before hitting those lastMessage code? This part is async so it can't be guaranteed to complete first so that means assistant_message or data_message would always have to be sent before text or assistant_control_data?

I am looking at the streaming code and it seems that assistant_control_data is always enqueued first so that may cause the id issue I'm seeing as well?

[lgrammel]

I am curious though, in this useAssistant() code, how does it ensure that there's always a message in the array before hitting those lastMessage code? This part is async so it can't be guaranteed to complete first so that means assistant_message or data_message would always have to be sent before text or assistant_control_data?

I am looking at the streaming code and it seems that assistant_control_data is always enqueued first so that may cause the id issue I'm seeing as well?

The OpenAI stream sends a thread.message.created event first, before the text delta stream starts. This trigger the sending of an assistant_message:

ai/packages/core/streams/assistant-response.ts

Line 50 in fac9a6e

case 'thread.message.created': {

[tchanxx]

@lgrammel Sorry, I'm still confused. Isn't assistant_control_data enqueued prior to forwardStream being called here?

ai/packages/core/streams/assistant-response.ts

Lines 88 to 105 in fac9a6e

	// send the threadId and messageId as the first message:
	controller.enqueue(
	textEncoder.encode(
	formatStreamPart('assistant_control_data', {
	threadId,
	messageId,
	}),
	),
	);
	try {
	await process({
	threadId,
	messageId,
	sendMessage,
	sendDataMessage,
	forwardStream,
	});

[lgrammel]

@lgrammel Sorry, I'm still confused. Isn't assistant_control_data enqueued prior to forwardStream being called here?

ai/packages/core/streams/assistant-response.ts

Lines 88 to 105 in fac9a6e

	// send the threadId and messageId as the first message:
	controller.enqueue(
	textEncoder.encode(
	formatStreamPart('assistant_control_data', {
	threadId,
	messageId,
	}),
	),
	);
	try {
	await process({
	threadId,
	messageId,
	sendMessage,
	sendDataMessage,
	forwardStream,
	});

Yes, first there is a control data message, then the assistant_message, and then the streaming. Am I missing something?

[tchanxx]

If the data control message comes first, then couldn't this fail since messages may be empty then lastMessage is undefined?

https://github.com/vercel/ai/blob/main/packages/core/react/use-assistant.ts#L212-L223

[lgrammel]

If the data control message comes first, then couldn't this fail since messages may be empty then lastMessage is undefined?

https://github.com/vercel/ai/blob/main/packages/core/react/use-assistant.ts#L212-L223

Yes, good point. This would need a check if there are any messages.

[lgrammel]

@tchanxx Can you update the PR to only check in the control data part, and move the if check outside of setMessages?

[lgrammel]

This can be checked before setMessages, e.g. like this:

if (messages.length > 0) {
  setMessages ...
}