Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track costs for streaming with Cohere #218

Open
brenkao opened this issue May 7, 2024 · 1 comment
Open

Track costs for streaming with Cohere #218

brenkao opened this issue May 7, 2024 · 1 comment
Labels
Feature Request New feature or request good first issue Good for newcomers

Comments

@brenkao
Copy link
Collaborator

brenkao commented May 7, 2024

Is your feature request related to a problem? Please describe.
Many providers are starting to add usage to streaming. This makes it much easier for Mirascope to calculate cost.

Describe the solution you'd like
Add a total_cost property to CohereCallResponseChunk. Read the "event_type": "stream-end" sent by Cohere API and calculate cost using

"token_count": {
    "prompt_tokens": ...,
    "response_tokens": ...,
    "total_tokens": ...,
    "billed_tokens": ...,
}

Update https://github.com/Mirascope/mirascope/blob/dev/mirascope/cohere/utils.py as necessary.

@brenkao brenkao added Feature Request New feature or request good first issue Good for newcomers labels May 7, 2024
@willbakst willbakst changed the title Cost tracking for streaming Cohere Feature Request: Track costs for streaming with Cohere May 8, 2024
@willbakst
Copy link
Contributor

willbakst commented May 8, 2024

See #214 since these are related.

Namely: #214 (comment)

@willbakst willbakst changed the title Feature Request: Track costs for streaming with Cohere [FEATURE REQUEST] Track costs for streaming with Cohere May 8, 2024
@willbakst willbakst changed the title [FEATURE REQUEST] Track costs for streaming with Cohere Track costs for streaming with Cohere May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature Request New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

2 participants