parameter: messages with role 'tool' must be a response to a preceeding message with 'tool_calls'

[Lexian-droid]

Hello, I am having an issue with my code and I can't figure out the problem, I get the function working and everything, you can talk to the AI just fine, this is using chat completions api, by the way.

The trouble is, when I have it run the function, it outputs 1 assistant message that's blank, has a finish reason of tool_calls, and then another message is outputted from a tool with the finish reason stop. The issue is, when I message it again, I get an error " parameter: messages with role 'tool' must be a response to a preceeding message with 'tool_calls' " and I don't know what it means.

I tried reading the code in the documentation, but I'm new to C# programming so I haven't got any idea of what it means. My code might not be as clear either, but I tried asking chatgpt, tried looking at the sample code, and finally tried reading the documentation code, and I still haven't yet figured it out.

Can anyone help me? this is my script:

public class ChatManager : MonoBehaviour
{
    public ScoreTracker scoreTracker;
    public List<Message> messages;
    public List<Tool> tools;
    private OpenAIClient api;
    public ChatHistoryTracker chatHistoryTracker;

    /// <summary>
    /// Sends a message and receives a response asynchronously.
    /// </summary>
    /// <param name="content">The content of the message to send.</param>
    /// <returns>A task representing the asynchronous operation.</returns>
    public new async Task SendMessage(string content)
    {
        AddMessage(
            new Message(
                Role.User,
                content
            )
        );

        // Define the tools that the assistant is able to use
        tools = new List<Tool>{
            new Function(
                nameof(scoreTracker.AddPoints),
                "Grant the user a point",
                new JObject
                {
                    ["type"] = "object",
                    ["properties"] = new JObject {
                        ["score"] = new JObject {
                            ["type"] = "integer",
                            ["description"] = "The number of points to add"
                        }
                    },
                    ["required"] = new JArray { "score" }
                })
        };

        var chatRequest = new ChatRequest(messages, tools: tools, toolChoice: "auto", Model.GPT4);
        var response = await api.ChatEndpoint.GetCompletionAsync(chatRequest);
        var choice = response.FirstChoice;
        var reason = choice.FinishReason;

        AddMessage(choice.Message, reason);
    }

    void AddMessage(Message message, string reason = "stop")
    {
        messages.Add(new Message(message.Role, message, reason));
        Debug.Log($"{message.Role}: {message} | {reason}");

        if (!string.IsNullOrEmpty(message))
        {
            chatHistoryTracker.Append(message.Role.ToString(), message);
        }

        if (reason != "stop")
        {
            var toolChoice = message.ToolCalls[0];
            var usedTool = toolChoice;

            if (toolChoice.Function.Name == "AddPoints")
            {
                // Deserialize the function arguments
                var functionArgs = JsonConvert.DeserializeObject<JObject>(usedTool.Function.Arguments.ToString());
                var functionResult = scoreTracker.AddPoints(functionArgs["score"].Value<int>());
                AddMessage(
                    new Message(
                        Role.Tool,
                        functionResult,
                        Role.Tool.ToString()
                    )
                );
            }
        }
    }
}

[StephenHodgson]

Hi @Lexian-droid thanks for using my package.

I see a few problems out the gate.

First, remove the new keyword from your method signature and rename your other method so that their names do not collide. Something like AddMessageInternal should suffice.

I think part of the problem is that your public method is stomping your private one and the code is not executing exactly like you think it is.

[Lexian-droid]

So, I rounded up re-writing it, and I solved the issue where the "tool_calls" isn't in the assistant message json, unfortunately though, a new error popped up, and I have no idea once more how to resolve it, I don't know how i am so stuck on this to be honest. If you maybe know what's causing it maybe? I don't know if I can continue this project if I can't get this error squared away.

{
  "error": {
    "message": "Missing parameter 'tool_call_id': messages with role 'tool' must have a 'tool_call_id'.",
    "type": "invalid_request_error",
    "param": "messages.[8].tool_call_id",
    "code": null
  }
}

and here is my script:

public class ChatManager : MonoBehaviour
{
    public ScoreTracker scoreTracker;
    public List<Message> messages;
    public List<Tool> tools;
    private OpenAIClient api;
    public ChatHistoryTracker chatHistoryTracker;

    // Start is called before the first frame update
    void Start()
    {
        api = new OpenAIClient()
        {
            EnableDebug = true
        };

        // Define the tools that the assistant is able to use
        tools = new List<Tool>{
            new Function(
                nameof(scoreTracker.AddPoints),
                "Grant the user a point",
                new JObject
                {
                    ["type"] = "object",
                    ["properties"] = new JObject {
                        ["score"] = new JObject {
                            ["type"] = "integer",
                            ["description"] = "The number of points to add"
                        }
                    },
                    ["required"] = new JArray { "score" }
                }
            )
        };

        messages.Add(
            new Message(Role.Assistant, "Hello.")
        );
    }

    /// <summary>
    /// Sends a message and receives a response asynchronously.
    /// </summary>
    /// <param name="content">The content of the message to send.</param>
    /// <returns>A task representing the asynchronous operation.</returns>
    public async Task SendMessage(string content)
    {
        messages.Add(
            new Message(Role.User, content)
        );

        MakeContact();
    }

    async void MakeContact()
    {
        var chatRequest = new ChatRequest(messages, tools: tools, toolChoice: "auto", Model.GPT3_5_Turbo_16K);
        var response = await api.ChatEndpoint.GetCompletionAsync(chatRequest);
        var choice = response.FirstChoice;

        messages.Add(choice.Message);
        Debug.Log(choice.FinishReason);

        if (choice.FinishReason == "tool_calls")
        {
            var tools = choice.Message.ToolCalls;
            var toolChoice = tools[0];
            var usedTool = toolChoice;

            Debug.Log($"{JsonConvert.SerializeObject(choice.Message)}");

            if (toolChoice.Function.Name == nameof(scoreTracker.AddPoints))
            {
                var functionArgs = JsonConvert.DeserializeObject<JObject>(usedTool.Function.Arguments.ToString());
                var functionResult = scoreTracker.AddPoints(functionArgs["score"].Value<int>());
                messages.Add(new Message(Role.Tool, functionResult, nameof(scoreTracker.AddPoints)));
            }
        }
    }
}

any help is always appreciated, thanks

[alexsafayan]

I have no experience in this package, but I think you may be able to get the tool id using this line of code: var toolID = tools.Id

Put it here with these:

var tools = choice.Message.ToolCalls;
var toolChoice = tools[0];
var usedTool = toolChoice;

I found this information in ToolCall.cs:

/// <summary>
/// The ID of the tool call.
/// This ID must be referenced when you submit the tool outputs in using the Submit tool outputs to run endpoint.
/// </summary>
[Preserve]
[JsonProperty("id")]
public string Id { get; }

[Lexian-droid]

Thanks, I appreciate the help,

            var tools = choice.Message.ToolCalls;
            var toolChoice = tools[0];
            var toolID = tools[0].Id;
            var usedTool = toolChoice;

I did what you said, except I had to make a a modification, tools is an array so i had to select the 0 option in the array.
Unfortunately, this didn't solve the issue, the trouble is more this:

the variable, "Tool Call Id" is blank, which I believe is the issue causing it

[Lexian-droid]

Found the issue, you have to add the message, but use "usedTool" as the role, like this:

messages.Add(new Message(usedTool, functionResult));

I didn't notice until just now that in the sample code (in the readme.md) there is the tool message that is using usedTool as it's role, and I thought that was invalid, so I changed it to Role.Tool, I did it because the type was OpenAI.Tool, not OpenAI.Role, which until now I didn't know that type of syntax was possible. That was definitely a big mistake on my part, thanks for the help, and I apologize for any time that I may have wasted.

[StephenHodgson]

No time wasted, people in the future will find it helpful I'm sure 😊. Sorry I was asleep, which is why I didn't respond sooner.

I'll try to update the docs to make this clearer.

[Lexian-droid]

No worries, thanks again for the help 😊
i hope ya'll have a great day or night!