Anthropic Claude 3 function calling: List[Literal] and List[Literal] | None fail

[timothyasp]

• This is an actually a bug report.
• I am not getting good LLM Results
• I have tried asking for help in the community on discord or discussions and have not received a response.
• I have tried searching the documentation and have not found an answer.

What Model are you using?

• gpt-3.5-turbo
• gpt-4-turbo
• gpt-4
• Other (claude3 models)

Describe the bug

Here's some sample code that fails:

languages = Literal["python", "javascript", "typescript", "java", "haskell", "php"]
class ListLiteral(BaseModel):
    languages: List[languages]

resp = create(
    model="claude-3-haiku-20240307",
    max_tokens=256,
    max_retries=0,
    messages=[
        {
            "role": "user",
            "content": "What are your top 3 favorite programming languages?",
        }
    ],
    response_model=ListLiteral,
)  # type: ignore

assert isinstance(resp, ListLiteral)
assert len(resp.languages) == 3

with error:

self = <xml.dom.expatbuilder.ExpatBuilderNS object at 0x1115f9cf0>
string = '<tool_description><tool_name>ListLiteral</tool_name><description>This is the function that must be used to construct ...AMETER_VALUE</$PARAMETER_NAME> tags for each item in the list. XML tags should only contain the name of the parameter.'

    def parseString(self, string):
        """Parse a document from a string, returning the document node."""
        parser = self.getParser()
        try:
>           parser.Parse(string, True)
E           xml.parsers.expat.ExpatError: junk after document element: line 2, column 0

/usr/local/Cellar/python@3.10/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/xml/dom/expatbuilder.py:223: ExpatError

And a different error when allowing anyOf with List[Literal] | None
Code:

languages = Literal["python", "javascript", "typescript", "java", "haskell", "php"]

def test_anthropic_list_literal_or_none():
    class ListLiteral(BaseModel):
        languages: List[languages] | None

    resp = create(
        model="claude-3-haiku-20240307",
        max_tokens=256,
        max_retries=0,
        messages=[
            {
                "role": "user",
                "content": "What are your top 3 favorite programming languages?",
            }
        ],
        response_model=ListLiteral,
    )  # type: ignore

    assert isinstance(resp, ListLiteral)
    assert len(resp.languages) == 3

With error:

root = <Element 'parameters' at 0x10f747d80>
model_dict = {'properties': {'languages': {'anyOf': [{'items': {'enum': [...], 'type': 'string'}, 'type': 'array'}, {'type': 'null'}], 'title': 'Languages'}}, 'required': ['languages'], 'title': 'ListLiteral', 'type': 'object'}
references = {}

    def _add_params(
        root: ET.Element, model_dict: Dict[str, Any], references: Dict[str, Any]
    ) -> bool:  # Return value indiciates if we ever came across a param with type List
        # TODO: handling of nested params with the same name
        properties = model_dict.get("properties", {})
        list_found = False
        nested_list_found = False
    
        for field_name, details in properties.items():
            parameter = ET.SubElement(root, "parameter")
            name = ET.SubElement(parameter, "name")
            name.text = field_name
            type_element = ET.SubElement(parameter, "type")
    
            # Get type
            if "anyOf" in details:  # Case where there can be multiple types
                # supports:
                # case 1: List type (example json: {'anyOf': [{'items': {'$ref': '#/$defs/PartialUser'}, 'type': 'array'}, {'type': 'null'}], 'default': None, 'title': 'Users'})
                # case 2: nested model (example json: {'anyOf': [{'$ref': '#/$defs/PartialDate'}, {'type': 'null'}], 'default': {}})
                field_type = " or ".join(
                    [
                        d["type"]
                        if "type" in d
                        else (d["$ref"] if "$ref" in d else "unknown")
                        for d in details["anyOf"]
                    ]
                )
            else:
                field_type = details.get(
                    "type", "unknown"
                )  # Might be better to fail here if there is no type since pydantic models require types
    
            if "array" in field_type and "items" not in details:
>               raise ValueError("Invalid array item.")
E               ValueError: Invalid array item.

../../instructor/anthropic_utils.py:80: ValueError

Expected behavior
This works well with OpenAI with instructor, so I'd expect the same results. It looks like it's an edge case in the json_to_xml function. I'll work on a fix, but opening this just in case someone already has a fix or idea how best to fix it.

[jxnl]

ty!

[BenDLH]

Hitting this issue too 👍

[virattt]

Same here, getting this issue too - unable to use claude (but openai works fine!)

[Cruppelt]

Very similar thing needs to happen as #524. I've added this to my test cases but the current structure needs to be revamped.

Added support for Literal #534 here. Union (|) not supported yet but getting there.