custom message and query in llm

[PradiptaAtByteIQMind-Graph2023]

Hi! Team

I got stucked here below:

responses = {
"user_query": prompt,
"sql_response": await self.postgres.run_query(prompt),
"mongo_response": await self.mongo.run_query(prompt),
"docusign_response": await self.docusign.run_query(prompt)
}

    i am getting the responses from every data sources after quering the user prompt 
    now i want to pass the response to the llm to give the summarize prompt 

in the current setup i am passing like below:

response = f"""
{responses} \n
----------------------------------------------
{instruction_prompt} \n
==============================================
Required Return Format \n
{required_format}
"""
parse_response = self.ai_services.llm.chat(response)

getting the below error : AttributeError: 'str' object has no attribute 'role'

can someone let me know what mis-implementation i am doing?

Thanks and Regards
Pradipta

[dosubot[bot]]

Hey there, @PradiptaAtByteIQMind-Graph2023! Fancy seeing you here again. Hope you're ready to dive back into the world of code with us 🚀

Based on the error message you're receiving, it seems like you're trying to access a role attribute on a string object, which is causing the AttributeError: 'str' object has no attribute 'role'. This error typically occurs when you're treating a string as a ChatMessage or another object that has a role attribute.

In your code, you're passing a formatted string to self.ai_services.llm.chat(response). If self.ai_services.llm.chat() expects a ChatMessage object, you should create a ChatMessage instance and pass it instead of a string. Here's how you can do it:

from llama_index.core.base.llms.types import ChatMessage, MessageRole

# Create a ChatMessage instance
message_content = f"""
    {responses} \n
    ----------------------------------------------
    {instruction_prompt} \n
    ==============================================
    Required Return Format \n
    {required_format}            
"""
message = ChatMessage.from_str(content=message_content, role=MessageRole.SYSTEM)

# Pass the ChatMessage instance to the chat method
parse_response = self.ai_services.llm.chat(message)

In this example, ChatMessage.from_str() is a class method that creates a ChatMessage instance from a string. The role parameter is set to MessageRole.SYSTEM, but you can adjust it according to your needs.

Please review your code and ensure that you're passing the correct types of objects to your methods. If self.ai_services.llm.chat() or any other part of your code expects a ChatMessage or similar object, you should create and pass an instance of that object instead of a string.

For more information about the ChatMessage class and its usage, you can refer to the LlamaIndex codebase.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

• llama-index-core/llama_index/core/base/llms/types.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[PradiptaAtByteIQMind-Graph2023]

@dosu-bot

{
"detail": "Internal Server Error from 'tuple' object has no attribute 'role'"
}

am getting this error

[dosubot]

Based on the error message you're encountering, it seems like a tuple object is being passed to a function or method that expects an object with a role attribute, such as a ChatMessage instance. This could be happening if a function that's supposed to return a ChatMessage instance is accidentally returning a tuple, or if a tuple is being passed in place of a ChatMessage.

Here are some steps you can take to resolve this issue:

1. Check the Return Values of Functions: Make sure that all functions that are supposed to return a ChatMessage instance are not accidentally returning a tuple. For instance, both call_function and acall_function return a tuple of (ChatMessage, ToolOutput). If you're using the result of these functions directly as input to another function expecting a ChatMessage, you need to extract the ChatMessage from the tuple first.

2. Review How ChatMessage Instances are Passed: When passing a ChatMessage instance to self.ai_services.llm.chat(response), ensure that you're not inadvertently passing a tuple or any other type. If you're manipulating or transforming the ChatMessage instance before passing it, double-check that process to ensure it doesn't result in a type change.

3. Debugging Tip: You can temporarily add logging or print statements to inspect the type and attributes of the object being passed into the function or method where the error occurs. This can help you pinpoint where the mismatch between expected and actual input types happens.

For your reference, here is the structure of the ChatMessage class in the LlamaIndex codebase:

class ChatMessage(BaseModel):
    """Chat message."""

    role: MessageRole = MessageRole.USER
    content: Optional[Any] = ""
    additional_kwargs: dict = Field(default_factory=dict)

    def __str__(self) -> str:
        return f"{self.role.value}: {self.content}"

    @classmethod
    def from_str(
        cls,
        content: str,
        role: Union[MessageRole, str] = MessageRole.USER,
        **kwargs: Any,
    ) -> "ChatMessage":
        if isinstance(role, str):
            role = MessageRole(role)
        return cls(role=role, content=content, **kwargs)

You can find this in the llama-index-core/llama_index/core/base/llms/types.py file.

Without specific details on how self.ai_services.llm.chat(response) is implemented and how it's called with the ChatMessage instance, these are general guidelines to troubleshoot and resolve the issue based on the provided context. If you continue to encounter issues, please provide more details about your implementation and I'll be happy to help further.

Sources

• llama-index-integrations/agent/llama-index-agent-openai/llama_index/agent/openai/openai_assistant_agent.py
• llama-index-core/llama_index/core/base/llms/types.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.