Add Support for Custom Headers in LLM Requests

[k1ng]

Feature Description

To integrate and proxy requests from Zep through Portkey.ai, we need the ability to add custom headers to our requests to the LLM. Please note that Portkey.ai already has an integration with Langchain.

Portkey.ai requires certain specific headers to be passed in every API call, which can be found in their documentation: https://docs.portkey.ai/how-portkey-works/portkey-headers

Value of Feature

This feature will allow logging Zep's requests to LLM and calculating the cost of LLM calls as a part of Zep-Portkey.ai integration. Custom headers support could also be useful for other potential integrations and use-cases.

Possible Implementation

We could provide the ability to set custom headers through environment variables, a .env file, or the config.yaml file. This provides flexibility in choosing how to supply custom headers, depending on user preferences or security requirements.

Thank You.

[danielchalef]

Hey @k1ng , cool idea. Would you be open to contributing this functionality to the project? If so, I suggest the following approach:

Config

1. Add the custom header option to ServerConfig:

   zep/config/models.go

   Line 44 in 90cd50d

   type ServerConfig struct {

2. Add an example and comment to the config.yaml file. Comment out the config option as a default.
   

   zep/config.yaml

   Line 41 in 90cd50d

   server:

3. OR if the header will include a secret, it should not be in the config file. Remove (2) above, and add the option to
   

   zep/config/config.go

   Line 17 in 90cd50d

   var EnvVars = map[string]string{

You may want to add two config options: One the header name and the other a secret, with the first being added in two above, but the secret kept to an environment variable only.

Add the header middleware

Create a new middleware function that pulls the header from appState.Config and adds it to the HTTP Header. See this example:

zep/pkg/server/middleware.go

Line 12 in 90cd50d

func SendVersion(next http.Handler) http.Handler {

Add the middleware to the Router

See here:

zep/pkg/server/routes.go

Line 44 in 90cd50d

func setupRouter(appState *models.AppState) *chi.Mux {

Happy to answer any questions. Also, happy to review a rough PR and work with you to finalize it.

[k1ng]

Hi @danielchalef,

Thank you very much for your detailed explanation, it is appreciated. I'd like to contribute, but unfortunately, I lack the necessary experience with Go to make a pull request at this time. I will definitely keep your guidance in mind and hopefully, in the future, I will be able to make more substantial contributions to the project.

[SpencerWightman]

I'd like to work on this issue. Could you please assign it to me?