Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OpenAI] Instructor for Batch Requests #604

Open
phatlast96 opened this issue Apr 16, 2024 · 1 comment
Open

[OpenAI] Instructor for Batch Requests #604

phatlast96 opened this issue Apr 16, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@phatlast96
Copy link

phatlast96 commented Apr 16, 2024

Is your feature request related to a problem? Please describe.
Since OpenAI has a bulk feature now and I've been using Instructor's JSON formatting feature a lot, I'd love to combine those 2 since I've been hitting my rate rate limit on the number of requests.

Describe the solution you'd like
I would love use Instructor for JSON formatting of bulk requests. This would be beneficial since it's a lot of cost savings.

Describe alternatives you've considered
Continue hitting my rate limit using Instructor.

Additional context
Add any other context or screenshots about the feature request here.
https://community.openai.com/t/batchapi-is-now-available/718416

@phatlast96 phatlast96 changed the title [OpenAI] Instructor for Bulk Requests [OpenAI] Instructor for Batch Requests Apr 16, 2024
@dosubot dosubot bot added the enhancement New feature or request label Apr 16, 2024
@CarloNicolini
Copy link

This feature should make instructor even more powerful. Having access to OpenAI batch requests allows a 50% discount. The API is something like this

from openai import OpenAI
client = OpenAI()

client.batches.create(
  input_file_id="file-abc123",
  endpoint="/v1/chat/completions",
  completion_window="24h"
)

Clearly there is some difference to be supported, but we'd all love the batch feature to be implemented by instructor ♥️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants