Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: widen support of env vars in OpenAI components #7653

Merged
merged 20 commits into from May 15, 2024

Conversation

CarlosFerLo
Copy link
Contributor

@CarlosFerLo CarlosFerLo commented May 6, 2024

Related Issues

Proposed Changes:

Now you can change the timeout and max retries of your OpenAI clients (OpenAIDocumentEmbedder, OpenAITextEmbedder, OpenAIGenerator and OpenAIChatGenerator) just by using the OPENAI_TIMEOUT and OPENAI_MAX_RETRIES environment variables.

How did you test it?

Enriched the unit tests of all this classes to test how this variables can be set.

Checklist

  • I have read the contributors guidelines and the code of conduct
  • I have updated the related issue with new insights and changes ✅
  • I added unit tests and updated the docstrings ✅
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:. ✅
  • I documented my code ✅
  • I ran pre-commit hooks and fixed any issue ✅

@github-actions github-actions bot added topic:tests 2.x Related to Haystack v2.0 labels May 6, 2024
@github-actions github-actions bot added the type:documentation Improvements on the docs label May 7, 2024
@CarlosFerLo CarlosFerLo marked this pull request as ready for review May 7, 2024 10:53
@CarlosFerLo CarlosFerLo requested review from a team as code owners May 7, 2024 10:53
@CarlosFerLo CarlosFerLo requested review from dfokina and masci and removed request for a team May 7, 2024 10:53
Copy link
Member

@masci masci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

Any reason why we don't also add these settings as __init__ parameter? In other words, we could have

    def __init__(
        self,
        ...
        timeout: float = 30.0,
        max_retries: int = 5
    ):

then in the method:

        ...
        self.client = OpenAI(
            api_key=api_key.resolve_value(),
            organization=organization,
            base_url=api_base_url,
            timeout=float(os.environ.get("OPENAI_TIMEOUT", timeout)),
            max_retries=int(os.environ.get("OPENAI_MAX_RETRIES", max_retries)),
        )

@CarlosFerLo
Copy link
Contributor Author

Thanks for the PR!

Any reason why we don't also add these settings as __init__ parameter? In other words, we could have

    def __init__(
        self,
        ...
        timeout: float = 30.0,
        max_retries: int = 5
    ):

then in the method:

        ...
        self.client = OpenAI(
            api_key=api_key.resolve_value(),
            organization=organization,
            base_url=api_base_url,
            timeout=float(os.environ.get("OPENAI_TIMEOUT", timeout)),
            max_retries=int(os.environ.get("OPENAI_MAX_RETRIES", max_retries)),
        )

Don't you think it would be better to make the priority order this way:

PARAMETER -> ENV_VAR -> DEFAULT

Where -> Means overwrites.

@masci
Copy link
Member

masci commented May 10, 2024

@CarlosFerLo yes please go ahead as you suggest!

@CarlosFerLo CarlosFerLo requested a review from masci May 12, 2024 16:13
haystack/components/embedders/openai_text_embedder.py Outdated Show resolved Hide resolved
haystack/telemetry/_environment.py Outdated Show resolved Hide resolved
@CarlosFerLo CarlosFerLo requested a review from masci May 15, 2024 20:48
@coveralls
Copy link
Collaborator

Pull Request Test Coverage Report for Build 9102969524

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 50 unchanged lines in 5 files lost coverage.
  • Overall coverage increased (+0.03%) to 90.585%

Files with Coverage Reduction New Missed Lines %
telemetry/_environment.py 2 84.21%
components/generators/openai.py 3 96.34%
components/embedders/openai_text_embedder.py 10 75.61%
components/embedders/openai_document_embedder.py 14 79.1%
components/generators/chat/openai.py 21 78.9%
Totals Coverage Status
Change from base Build 9101640205: 0.03%
Covered Lines: 6591
Relevant Lines: 7276

💛 - Coveralls

@masci masci enabled auto-merge (squash) May 15, 2024 21:55
@masci masci merged commit 686a499 into deepset-ai:main May 15, 2024
25 checks passed
@CarlosFerLo CarlosFerLo deleted the issue-7610 branch May 16, 2024 05:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add enviroment variable for openai timeout, backoff and max retries
3 participants