Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] ChatAgent Fails to Process Function Calls in Messages Containing Single Quotes #541

Open
2 of 3 tasks
yiyiyi0817 opened this issue May 6, 2024 · 0 comments · May be fixed by #529
Open
2 of 3 tasks

[BUG] ChatAgent Fails to Process Function Calls in Messages Containing Single Quotes #541

yiyiyi0817 opened this issue May 6, 2024 · 0 comments · May be fixed by #529
Assignees
Labels
bug Something isn't working

Comments

@yiyiyi0817
Copy link
Member

yiyiyi0817 commented May 6, 2024

Required prerequisites

What version of camel are you using?

0.1.3

System information

3.10.9 | packaged by Anaconda, Inc. | (main, Mar 1 2023, 18:18:15) [MSC v.1916 64 bit (AMD64)] win32

Problem description

In line 537 of chat_agent.py, within the step_function_call function of the class ChatAgent, converting the message returned by the LLM into JSON format:

func_name = choice.message.function_call.name
func = self.func_dict[func_name]

args_str: str = choice.message.function_call.arguments
args = json.loads(args_str.replace("'", "\""))

I think this might be due to the fact that sometimes the string returned by the LLM is not standardized; for example, in such cases, we need to replace single quotes with double quotes in key-value pairs to load JSON.
{'jobTitle_in_query': 'basketball coach', 'city_in_query': 'London'}.

For this situation, a simple args_str.replace("'", "\"") can effectively achieve the desired outcome.

However, when I introduce more functions, often the value of the string itself contains single quotes, such as

{
  "requestBody": {
    "phrase_to_translate": "A stitch in time saves nine",
    "learning_language": "zh",
    "native_language": "en",
    "additional_context": "",
    "full_query": "Translate the phrase 'A stitch in time saves nine' to Chinese"
  }
}

In this case, using args_str.replace("'", "\"") would transform the string to look something like this:

{
  "requestBody": {
    "phrase_to_translate": "A stitch in time saves nine",
    "learning_language": "zh",
    "native_language": "en",
    "additional_context": "",
    "full_query": "Translate the phrase "A stitch in time saves nine" to Chinese"
  }
}

"Due to the multiple double quotes in the last key, if we still run args = json.loads(args_str.replace("'", "\"")), this would lead to an error: json.decoder.JSONDecodeError: Expecting ',' delimiter: line 8 column 42 (char 211).

The occurrence of single quotes in the LLM's return values is not very rare. I often encounter this error causing interruptions in the dialogue between the AI user and assistant when running functions related to language and translation. Perhaps we need to add code to handle different scenarios involving single quotes differently.

Reproducible example code

The Python snippets:

import json

args_str = '''
{
  "requestBody": {
    "phrase_to_translate": "A stitch in time saves nine",
    "learning_language": "zh",
    "native_language": "en",
    "additional_context": "",
    "full_query": "Translate the phrase 'A stitch in time saves nine' to Chinese"
  }
}
'''

print(args_str)
print('After replace:')
print(args_str.replace("'", "\""))
args = json.loads(args_str)
print('After replace:')
args = json.loads(args_str.replace("'", "\""))

Traceback

No response

Expected behavior

No response

Additional context

No response

@yiyiyi0817 yiyiyi0817 added the bug Something isn't working label May 6, 2024
@Wendong-Fan Wendong-Fan linked a pull request May 6, 2024 that will close this issue
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Backlog
Development

Successfully merging a pull request may close this issue.

2 participants