Multiple tool calls

[covitof]

Hello there,

while playing with instructor and openAI, I have stumbled upon the case of multiple tool calls (relevant to my application). Instructor does not support it and the error message suggests to "use List[Model] instead". Can anybody explain what this means?
Some example code to make the discussion easier:

from instructor import OpenAISchema, patch, Mode
from openai import AsyncOpenAI

class test(OpenAISchema):
    """Test schema."""
    a: float
    b: float

client = patch(AsyncOpenAI(), Mode.TOOLS)

response = await client.chat.completions.create(
    model="gpt-3.5-turbo-1106",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "call the function test with arguments 2 and 3 and then with 3 and 4"},
    ],
    tools=[{"type": "function", "function": test.openai_schema},],
    temperature=0.2,
)

test.from_response(response, mode=Mode.TOOLS)

response.choices[0].message.tool_calls contains two tool_calls. This triggers the error "Instructor does not support multiple tool calls, use List[Model] instead.".

I'd loop over the tool calls, but instructor manually selects message.tool_calls[0]. Is the only solution creating artificial responses with the single calls?

[jxnl]

https://jxnl.github.io/instructor/concepts/union/

and https://jxnl.github.io/instructor/concepts/lists/

[covitof]

I did not know about the documentation 😅 thank you. This does not directly solve my problem, since it would be cases where I have different things happening in the same command, e.g. extract two different models (and not always the same pair) from one chunk of text. But I see the point now. Plus, I guess I can deal with this extra complication via the task planner.

Thank you!

[jxnl]

thats where the union would come in!

[zby]

Now I think that union solution is not optimal - it complicates the schema and llm compliance with the schema is still limited. It would be better to put a list of tool definitions into the tools array. But after looking into the source I could not find any easy way to do that with Instructor.

[jxnl]

does https://www.askmarvin.ai/welcome/what_is_marvin/ work?

[covitof]

would there be anything wrong/cumbersome with adding an additional optional tool_index argument in from_response? it could simply then be used on this line to select which tool to call. Or maybe I don't understand why it's important to keep instructor only with the single tool mode.