feat(agent/core): Add `GroqProvider` #7130

Pwuts · 2024-05-07T15:37:30Z

Background

This has issues with the profile generator, so maybe merge #7121 first

Changes 🏗️

Add GroqProvider
- Add model_providers.groq
- Add to model_providers.multi
- Update .env.template
- Update docs

PR Quality Scorecard ✨

Have you used the PR description template? +2 pts
Is your pull request atomic, focusing on a single change? +5 pts
Have you linked the GitHub issue(s) that this PR addresses? +5 pts
Have you documented your changes clearly and comprehensively? +5 pts
Have you changed or added a feature? -4 pts
- Have you added/updated corresponding documentation? +4 pts
- Have you added/updated corresponding integration tests? +5 pts
Have you changed the behavior of AutoGPT? -5 pts
- Have you also run agbenchmark to verify that these changes do not regress performance? +10 pts

PR Type

Enhancement

Description

Introduced GroqProvider to manage interactions with the Groq API, including methods for chat completions and API retries.
Updated multi.py to integrate GroqProvider alongside other model providers.
Extended ModelProviderName enum to include GROQ.
Added the groq package to project dependencies to ensure API interaction capabilities.

Changes walkthrough 📝

Relevant files

Enhancement

groq.py `Implement GroqProvider for Groq API Interaction` autogpts/autogpt/autogpt/core/resource/model_providers/groq.py Introduced a new `GroqProvider` class for handling interactions with the Groq API. Defined model configurations, credentials, and settings for Groq. Implemented methods for creating chat completions and handling API retries.	+417/-0
multi.py `Integrate GroqProvider into MultiProvider` autogpts/autogpt/autogpt/core/resource/model_providers/multi.py Added `GroqProvider` to the list of model providers. Updated the `ModelName` type to include `GroqModelName`. Included `GROQ_CHAT_MODELS` in the combined chat models dictionary.	+6/-4
schema.py `Update ModelProviderName Enum with GROQ` autogpts/autogpt/autogpt/core/resource/model_providers/schema.py Added `GROQ` to the `ModelProviderName` enum.	+1/-0

Dependencies

pyproject.toml `Add Groq Dependency in pyproject.toml` autogpts/autogpt/pyproject.toml Added `groq` package as a dependency.	+1/-0

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

netlify · 2024-05-07T15:37:55Z

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	`d5a70c5`
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/663a5c9cf59a2a00088a99c2

codiumai-pr-agent-pro · 2024-05-07T15:38:14Z

PR Description updated to latest commit (976396c)

Copy walkthrough table to "Files Changed" Tab

codiumai-pr-agent-pro · 2024-05-07T15:38:33Z

PR Review 🔍

⏱️ Estimated effort to review [1-5]	4, because the PR introduces a significant amount of new code (418 lines) related to the integration of a new model provider (`GroqProvider`). The complexity of the code, which includes handling API connections, error parsing, retries, and token management, requires a thorough review to ensure functionality and maintainability.
🧪 Relevant tests	No
⚡ Possible issues	Possible Bug: The method `count_tokens` and `count_message_tokens` in `GroqProvider` return 0, which seems incorrect and could lead to issues with token counting.
	Error Handling: The error handling in `create_chat_completion` might not properly manage all exceptions, especially with nested try-except blocks.
	Performance Concern: The recursive appending of messages in `create_chat_completion` during parsing failures might lead to excessive memory use or infinite loops if not controlled properly.
🔒 Security concerns	No

codiumai-pr-agent-pro · 2024-05-07T15:40:15Z

PR Code Suggestions ✨

Category	Suggestions
Enhancement	Replace hardcoded token costs with dynamic fetching to enhance maintainability. Replace the hardcoded token costs with a more dynamic approach, such as fetching them from an external configuration or API. This will make the system more flexible and easier to maintain as token costs change. autogpts/autogpt/autogpt/core/resource/model_providers/groq.py [56-57] -prompt_token_cost=0.05 / 1e6, -completion_token_cost=0.10 / 1e6, +prompt_token_cost=self.get_token_cost('prompt'), +completion_token_cost=self.get_token_cost('completion'),
Enhancement	Add logging for token limits to assist in debugging and monitoring. Add logging for the token costs in the `get_token_limit` method to aid in debugging and monitoring the token usage. autogpts/autogpt/autogpt/core/resource/model_providers/groq.py [154-156] def get_token_limit(self, model_name: str) -> int: - return GROQ_CHAT_MODELS[model_name].max_tokens + max_tokens = GROQ_CHAT_MODELS[model_name].max_tokens + self._logger.debug(f"Token limit for model {model_name}: {max_tokens}") + return max_tokens
Robustness	Add error handling for API key retrieval to prevent runtime errors. Implement error handling for the API key retrieval in `GroqCredentials` to ensure robustness in scenarios where the environment variables are not set or are invalid. autogpts/autogpt/autogpt/core/resource/model_providers/groq.py [96] -api_key: SecretStr = UserConfigurable(from_env="ANTHROPIC_API_KEY") +api_key: SecretStr = UserConfigurable(from_env="ANTHROPIC_API_KEY", default=SecretStr('default_api_key'))
Robustness	Improve error handling in `_parse_assistant_tool_calls` for better fault tolerance. Use a more robust error handling strategy in the `_parse_assistant_tool_calls` method to ensure that all errors are logged and appropriately handled, rather than just continuing on failure. autogpts/autogpt/autogpt/core/resource/model_providers/groq.py [370-383] if assistant_message.tool_calls: for _tc in assistant_message.tool_calls: try: parsed_arguments = json_loads(_tc.function.arguments) except Exception as e: err_message = ( f"Decoding arguments for {_tc.function.name} failed: " + str(e.args[0]) ) - parse_errors.append( - type(e)(err_message, *e.args[1:]).with_traceback( - e.__traceback__ - ) - ) + self._logger.error(err_message) + sentry_sdk.capture_exception(e) continue
Maintainability	Refactor `create_chat_completion` to improve code readability and maintainability. Refactor the `create_chat_completion` method to separate the concerns of API calling and response parsing into distinct methods. This will improve the readability and maintainability of the code. autogpts/autogpt/autogpt/core/resource/model_providers/groq.py [175-184] async def create_chat_completion( self, model_prompt: list[ChatMessage], model_name: GroqModelName, completion_parser: Callable[[AssistantChatMessage], _T] = lambda _: None, functions: Optional[list[CompletionModelFunction]] = None, max_output_tokens: Optional[int] = None, prefill_response: str = "", kwargs, ) -> ChatModelResponse[_T]: + groq_messages, completion_kwargs = self._prepare_completion_kwargs( + model_prompt, model_name, functions, max_output_tokens, kwargs + ) + response = await self._call_api_for_completion(completion_kwargs) + return self._parse_api_response(response, completion_parser)

codiumai-pr-agent-pro · 2024-05-07T15:40:22Z

Changelog updates: 🔄

2024-05-07

Added

Introduced GroqProvider to manage interactions with Groq's API, expanding the available model providers.

to commit the new content to the CHANGELOG.md file, please type:
'/update_changelog --pr_update_changelog.push_changelog_changes=true'

codiumai-pr-agent-pro · 2024-05-07T15:40:24Z

PR Analysis 🔬

This screen contains a list of code components that were changed in this PR.
You can initiate specific actions for each component, by checking the relevant boxes.
After you check a box, the action will be performed automatically by PR-Agent.
Results will appear as a comment on the PR, typically after 30-60 seconds.

file

Changed components

groq.py

Test Docs Improve Similar	GroqModelName ^(class)	+5/-0
Test Docs Improve Similar	GroqConfiguration ^(class)	+2/-0
Test Docs Improve Similar	GroqCredentials ^(class)	+17/-0
Test Docs Improve Similar	GroqSettings ^(class)	+4/-0
Test Docs Improve Similar	GroqProvider ^(class)	+300/-0

multi.py

Test Docs Improve Similar	_get_model_provider_class ^{(method of MultiProvider)}	+2/-2
Test Docs Improve Similar	_get_provider_class ^{(method of MultiProvider)}	+3/-2

schema.py

Test Docs Improve Similar	ModelProviderName ^(class)	+1/-0

💡 Usage guide:

Using static code analysis capabilities, the analyze tool scans the PR code changes and find the code components (methods, functions, classes) that changed in the PR.

The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR:

/analyze

Language that are currently supported: Python, Java, C++, JavaScript, TypeScript, C#.
See more information about the tool in the docs.

codecov · 2024-05-07T15:40:44Z

Codecov Report

Attention: Patch coverage is 37.41497% with 92 lines in your changes are missing coverage. Please review.

Project coverage is 44.45%. Comparing base (08c32a7) to head (d5a70c5).
Report is 10 commits behind head on master.

Files	Patch %	Lines
...ogpt/autogpt/core/resource/model_providers/groq.py	35.66%	90 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7130      +/-   ##
==========================================
- Coverage   44.63%   44.45%   -0.19%     
==========================================
  Files         133      134       +1     
  Lines        6309     6454     +145     
  Branches      823      848      +25     
==========================================
+ Hits         2816     2869      +53     
- Misses       3382     3472      +90     
- Partials      111      113       +2

Flag	Coverage Δ
Linux	`44.37% <37.41%> (-0.18%)`	⬇️
Windows	`42.57% <37.41%> (-0.14%)`	⬇️
autogpt-agent	`44.42% <37.41%> (-0.19%)`	⬇️
macOS	`43.77% <37.41%> (-0.17%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ntindle · 2024-05-09T06:54:23Z

autogpts/autogpt/autogpt/core/resource/model_providers/groq.py

+        ChatModelInfo(
+            name=GroqModelName.LLAMA3_8B,
+            provider_name=ModelProviderName.GROQ,
+            prompt_token_cost=0.05 / 1e6,


consider the following format 1_000_000 rather than 1e6

ntindle · 2024-05-09T06:54:50Z

autogpts/autogpt/.env.template

+## GROQ_API_KEY - Groq API Key (Example: gsk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx)
+# GROQ_API_KEY=
+


GROQ_API_BASE_URL missing

ntindle · 2024-05-09T06:55:44Z

docs/content/AutoGPT/configuration/options.md

@@ -23,6 +23,7 @@ Configuration is controlled through the `Config` object. You can set configurati
 - `GITHUB_USERNAME`: GitHub Username. Optional.
 - `GOOGLE_API_KEY`: Google API key. Optional.
 - `GOOGLE_CUSTOM_SEARCH_ENGINE_ID`: [Google custom search engine ID](https://programmablesearchengine.google.com/controlpanel/all). Optional.
+- `GROQ_API_KEY`: Set this if you want to use Groq models with AutoGPT


GROQ_API_BASE

ntindle · 2024-05-09T06:56:12Z

autogpts/autogpt/autogpt/core/resource/model_providers/multi.py

@@ -143,16 +144,17 @@ def _get_provider(self, provider_name: ModelProviderName) -> ChatModelProvider:
    @classmethod
    def _get_model_provider_class(
        cls, model_name: ModelName
-    ) -> type[AnthropicProvider | OpenAIProvider]:
+    ) -> type[AnthropicProvider | GroqProvider | OpenAIProvider]:


Type alias this like ModelName is above

ntindle · 2024-05-09T06:57:46Z

autogpts/autogpt/autogpt/core/resource/model_providers/groq.py

+
+
+class GroqConfiguration(ModelProviderConfiguration):
+    fix_failed_parse_tries: int = UserConfigurable(3)


This needs a from_env right?

ntindle · 2024-05-09T06:59:09Z

autogpts/autogpt/autogpt/core/resource/model_providers/groq.py

+
+    @classmethod
+    def get_tokenizer(cls, model_name: GroqModelName) -> ModelTokenizer:
+        # HACK: No official tokenizer is available for Claude 3


Remove comment here

ntindle · 2024-05-09T06:59:29Z

autogpts/autogpt/autogpt/core/resource/model_providers/groq.py

+
+    @classmethod
+    def count_tokens(cls, text: str, model_name: GroqModelName) -> int:
+        return 0  # HACK: No official tokenizer is available for Claude 3


Remove comment here, and give an attempt at counting tokens lol

ntindle · 2024-05-09T06:59:57Z

autogpts/autogpt/autogpt/core/resource/model_providers/groq.py

+        messages: ChatMessage | list[ChatMessage],
+        model_name: GroqModelName,
+    ) -> int:
+        return 0  # HACK: No official tokenizer is available for Claude 3


Remove comment here, and give an attempt at counting tokens

for now, I'd take tiktoken and hope for the best

Swiftyos · 2024-05-10T13:54:31Z

Why do we need a new provider, when Groq is compatible with OpenAI and just needs the base_url changing?

ntindle · 2024-05-10T14:56:22Z

Why do we need a new provider, when Groq is compatible with OpenAI and just needs the base_url changing?

We would want a provider to support additional models with different features, costs, and requirements than OpenAI. Currently, we support anthropic and OpenAI, and both require different things like the above. In the future, this is where things like a rate limit to prevent overages would live as well.

github-actions · 2024-05-14T14:13:27Z

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

Pwuts requested a review from a team as a code owner May 7, 2024 15:37

github-actions bot added AutoGPT Agent size/l labels May 7, 2024

codiumai-pr-agent-pro bot added the enhancement New feature or request label May 7, 2024

codiumai-pr-agent-pro bot added the Review effort [1-5]: 4 label May 7, 2024

Pwuts force-pushed the groq branch from 976396c to bcebc7a Compare May 7, 2024 15:40

github-actions bot added the documentation Improvements or additions to documentation label May 7, 2024

feat(agent/core): Add GroqProvider

02c8194

Pwuts force-pushed the groq branch from bcebc7a to 02c8194 Compare May 7, 2024 15:48

Merge branch 'master' into groq

d5a70c5

ntindle reviewed May 9, 2024

View reviewed changes

ntindle requested changes May 9, 2024

View reviewed changes

github-actions bot added the conflicts Automatically applied to PRs with merge conflicts label May 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(agent/core): Add `GroqProvider` #7130

feat(agent/core): Add `GroqProvider` #7130

Pwuts commented May 7, 2024 •

edited

netlify bot commented May 7, 2024 •

edited

codiumai-pr-agent-pro bot commented May 7, 2024

codiumai-pr-agent-pro bot commented May 7, 2024

codiumai-pr-agent-pro bot commented May 7, 2024

codiumai-pr-agent-pro bot commented May 7, 2024

codiumai-pr-agent-pro bot commented May 7, 2024

codecov bot commented May 7, 2024 •

edited

ntindle May 9, 2024

ntindle May 9, 2024

ntindle May 9, 2024

ntindle May 9, 2024

ntindle May 9, 2024

ntindle May 9, 2024

ntindle May 9, 2024

ntindle May 9, 2024

ntindle May 9, 2024

Swiftyos commented May 10, 2024

ntindle commented May 10, 2024

github-actions bot commented May 14, 2024

		## GROQ_API_KEY - Groq API Key (Example: gsk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx)
		# GROQ_API_KEY=



		class GroqConfiguration(ModelProviderConfiguration):
		fix_failed_parse_tries: int = UserConfigurable(3)

feat(agent/core): Add GroqProvider #7130

Are you sure you want to change the base?

feat(agent/core): Add GroqProvider #7130

Conversation

Pwuts commented May 7, 2024 • edited

Background

Changes 🏗️

PR Quality Scorecard ✨

PR Type

Description

Changes walkthrough 📝

netlify bot commented May 7, 2024 • edited

✅ Deploy Preview for auto-gpt-docs canceled.

codiumai-pr-agent-pro bot commented May 7, 2024

codiumai-pr-agent-pro bot commented May 7, 2024

PR Review 🔍

codiumai-pr-agent-pro bot commented May 7, 2024

PR Code Suggestions ✨

codiumai-pr-agent-pro bot commented May 7, 2024

2024-05-07

Added

codiumai-pr-agent-pro bot commented May 7, 2024

PR Analysis 🔬

codecov bot commented May 7, 2024 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Swiftyos commented May 10, 2024

ntindle commented May 10, 2024

github-actions bot commented May 14, 2024

feat(agent/core): Add `GroqProvider` #7130

feat(agent/core): Add `GroqProvider` #7130

Pwuts commented May 7, 2024 •

edited

netlify bot commented May 7, 2024 •

edited

codecov bot commented May 7, 2024 •

edited