New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Freature #1005 - Add streaming API for Bedrock Anthropics #1006
base: main
Are you sure you want to change the base?
Conversation
@langchain4j Previous comments were addressed. Would be amazing if you could run CI I checked locally everything compiles and tests pass. |
PR doesn't address token count as Bedrock doesn't expose such information as part of response. This will require token estimations which will be addressed in the next PR. |
@langchain4j would be amazing if you could give a look on it. |
@michalkozminski I will review it asap, in the order of priorities. Thanks for your patience! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@michalkozminski thanks a lot! Just a few minor comments
@SuperBuilder | ||
public class BedrockAnthropicStreamingChatModel extends AbstractBedrockStreamingChatModel { | ||
@Builder.Default | ||
private final String model = BedrockAnthropicStreamingChatModel.Types.AnthropicClaudeV2.getValue(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor: should it use one of v3 models (e.g. haiku) by default?
@@ -179,7 +179,7 @@ See example [here](https://github.com/langchain4j/langchain4j-examples/blob/main | |||
| [OpenAI](https://docs.langchain4j.dev/integrations/language-models/open-ai) | ✅ | ✅ | ✅ | ✅ | ✅ | | ✅ | | |||
| [Azure OpenAI](https://docs.langchain4j.dev/integrations/language-models/azure-open-ai) | | ✅ | ✅ | ✅ | ✅ | | ✅ | | |||
| [Hugging Face](https://docs.langchain4j.dev/integrations/language-models/hugging-face) | | ✅ | | ✅ | | | | | | |||
| [Amazon Bedrock](https://docs.langchain4j.dev/integrations/language-models/amazon-bedrock) | | ✅ | | ✅ | ✅ | | | | |||
| [Amazon Bedrock](https://docs.langchain4j.dev/integrations/language-models/amazon-bedrock) | | ✅ | ✅ | ✅ | ✅ | | | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Context
Feature adds Bedrock Antrhopics Streaming capability #1005
Previous PR #679
Change
Added new streaming model
AbstractBedrockStreamingChatModel
Checklist
Before submitting this PR, please check the following points:
Checklist for adding new embedding store integration