feat: Return the messages as well in response for potentially fewer retries.

[HamzaFarhan]

PR for issue: 522

---

	🚀 This PR description was created by Ellipsis for commit fd44008.

Summary:

This PR modifies the retry functions and update_total_usage to include messages in the response, potentially reducing the number of retries.

Key points:

• Modified retry_sync and retry_async in /instructor/retry.py to include messages in CompletionUsage.
• Updated update_total_usage in /instructor/utils.py to add messages to total_usage.
• Aimed at reducing retries by including messages in the response.

---

Generated with ❤️ by ellipsis.dev

[ellipsis-dev]

❌ Changes requested.

• Reviewed the entire pull request up to fd44008
• Looked at 73 lines of code in 2 files
• Took 53 seconds to review

More info

• Skipped 0 files when reviewing.
• Skipped posting 1 additional comments because they didn't meet confidence threshold of 85%.

1. instructor/utils.py:66:

• Assessed confidence : 100%
• Grade: 70%
• Comment:
  The 'messages' attribute is not part of the CompletionUsage class. This will cause an AttributeError when trying to assign the 'messages' attribute to the 'total_usage' object. Please remove the 'messages' attribute from the 'total_usage' object.

    if isinstance(response, ChatCompletion) and response.usage is not None:
        total_usage.completion_tokens += response.usage.completion_tokens or 0
        total_usage.prompt_tokens += response.usage.prompt_tokens or 0
        total_usage.total_tokens += response.usage.total_tokens or 0

• Reasoning:
  The 'messages' attribute is not part of the CompletionUsage class. This will cause an AttributeError when trying to assign the 'messages' attribute to the 'total_usage' object. The 'messages' attribute should be removed from the 'total_usage' object.

Workflow ID: wflow_0DDVOPedBmZyL2Hf

---

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. We'll respond in a few minutes. Learn more here.

[HamzaFarhan]

Here's an example.

This does not work when messages = [] and max_retries = 1

It does work when:

• messages = [] but max_retries = 2
• messages is this list below and max_retries = 1

def chat_message(role: str, content: str) -> dict:
    return {"role": role, "content": content}


def user_message(content: str) -> dict:
    return chat_message(role="user", content=content)


class People(BaseModel):
    people: list[str] = Field(..., examples=["Hamza", "Jason"], min_length=3)
    
messages = [
    {"role": "user", "content": "2 french names"},
    {
        "role": "assistant",
        "content": "",
        "tool_calls": [
            {
                "id": "call_UtoQ9RpuVKmXEmu9DahWA7Mw",
                "function": {
                    "arguments": '{"people":["Jean","Marie"]}',
                    "name": "People",
                },
                "type": "function",
            }
        ],
    },
    {
        "role": "tool",
        "tool_call_id": "call_UtoQ9RpuVKmXEmu9DahWA7Mw",
        "name": "People",
        "content": "Validation Error found:\n1 validation error for People\npeople\n  List should have at least 3 items after validation, not 2 [type=too_short, input_value=['Jean', 'Marie'], input_type=list]\n    For further information visit https://errors.pydantic.dev/2.6/v/too_short\nRecall the function correctly, fix the errors",
    },
]
max_retries = 1

prompt = "2 french names"
messages = [user_message(prompt)] + messages
res = ask_oai(
    messages=messages, model=MODEL, max_retries=max_retries, response_model=People
)
# People(people=['Jean', 'Marie', 'Pierre'])

[jxnl]

We can definitely move it somewhere else, but we shouldn't be moving it into the usage.

[HamzaFarhan]

Cool. What would you suggest? I played around with this today and it feels like a simplified version of DSPy since both approaches give us a better input for future calls. So it's definitely useful.