Azure OpenAI "Resource not found Error" when using baseURL in YAML config

[hostage007]

What happened?

I have custom librechat.yaml to use Azure OpenAI, but get 404 error when requesting.

2024-05-16T10:15:41.388Z warn: [OpenAIClient.chatCompletion][stream] API error
2024-05-16T10:15:41.390Z error: [handleAbortError] AI response error; aborting request: 404 Resource not found
2024-05-16T10:15:41.400Z debug: [AskController] Request closed

Steps to Reproduce

I have custom librechat.yaml

# Configuration version (required)
version: 1.0.8
# This setting caches the config file for faster loading across app lifecycle
cache: true

endpoints:
  azureOpenAI:
    # Endpoint-level configuration
    plugins: false
    assistants: false
    groups:
    # Group-level configuration
    - group: "OpenAI-EASTUS2"
      apiKey: "${EASTUS2_API_KEY}"
      instanceName: "${EASTUS2_INSTANCENAME}"
      version: "2024-04-01-preview"
      baseURL: "${EASTUS2_BASEURL}"
      additionalHeaders:
        X-Custom-Header: "KevinTest"
      # Model-level configuration
      models:
        gpt-4-turbo-2024-04-09:
          deploymentName: "GPT4-Turbo-Latest-Dev"

It gives 404 error, logs as below.

2024-05-16T10:15:39.836Z debug: [OpenAIClient] chatCompletion
{
  baseURL: "https://${EASTUS2_BASEURL}/openai/deployments/gpt-4-turbo-2024-04-09/chat/comp... [truncated]",
    modelOptions.model: "gpt-4-turbo-2024-04-09",
    modelOptions.temperature: 1,
    modelOptions.top_p: 1,
    modelOptions.presence_penalty: 0,
    modelOptions.frequency_penalty: 0,
    modelOptions.stop: undefined,
    modelOptions.max_tokens: undefined,
    modelOptions.user: "66442d5b7c4c582b181fabe0",
    modelOptions.stream: true,
    // 1 message(s)
    modelOptions.messages: [{"role":"user","content":"test"}],
}
2024-05-16T10:15:41.388Z warn: [OpenAIClient.chatCompletion][stream] API error
2024-05-16T10:15:41.390Z error: [handleAbortError] AI response error; aborting request: 404 Resource not found
2024-05-16T10:15:41.400Z debug: [AskController] Request closed

As you can see, the model name was concatnated into url, but Azure requires DeploymentName there

Ref:https://learn.microsoft.com/en-us/azure/ai-services/openai/reference
POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/completions?api-version=2024-02-01

Ref:https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/azure_openai

models:
  gpt-4-vision-preview: 
    deploymentName: "arbitrary-deployment-name" **//This should not be arbitrary**
    version: "2024-02-15-preview"

What browsers are you seeing the problem on?

No response

Relevant log output

2024-05-16T10:15:36.462Z debug: [AskController]
{
  text: "test",
  conversationId: null,
  endpoint: "azureOpenAI",
  chatGptLabel: null,
  promptPrefix: null,
  resendFiles: true,
  imageDetail: "auto",
  iconURL: undefined,
  greeting: undefined,
  spec: undefined,
  maxContextTokens: undefined,
    modelOptions.model: "gpt-4-turbo-2024-04-09",
    modelOptions.temperature: 1,
    modelOptions.top_p: 1,
    modelOptions.presence_penalty: 0,
    modelOptions.frequency_penalty: 0,
    modelOptions.stop: undefined,
    modelOptions.max_tokens: undefined,
    // 20 openAI(s)
    modelsConfig.openAI: ["gpt-4o","gpt-3.5-turbo-0125","gpt-4-turbo","gpt-4-turbo-2024-04-09","gpt-3.5-turbo-16k-0613","gpt-3.5-turbo-16k","gpt-4-turbo-preview","gpt-4-0125-preview","gpt-4-1106-preview","gpt-3.5-turbo","gpt-3.5-turbo-1106","gpt-4-vision-preview","gpt-4","gpt-3.5-turbo-instruct-0914","gpt-3.5-turbo-0613","gpt-3.5-turbo-0301","gpt-3.5-turbo-instruct","gpt-4-0613","text-davinci-003","gpt-4-0314"],
    // 12 google(s)
    modelsConfig.google: ["gemini-pro","gemini-pro-vision","chat-bison","chat-bison-32k","codechat-bison","codechat-bison-32k","text-bison","text-bison-32k","text-unicorn","code-gecko","code-bison","code-bison-32k"],
    // 3 anthropic(s)
    modelsConfig.anthropic: ["claude-1","claude-instant-1","claude-2"],
    // 12 gptPlugin(s)
    modelsConfig.gptPlugins: ["gpt-4o","gpt-3.5-turbo-0125","gpt-4-turbo","gpt-4-turbo-2024-04-09","gpt-3.5-turbo-16k","gpt-4-turbo-preview","gpt-4-0125-preview","gpt-4-1106-preview","gpt-3.5-turbo","gpt-3.5-turbo-1106","gpt-4-vision-preview","gpt-4"],
    // 1 azureOpenAI(s)
    modelsConfig.azureOpenAI: ["gpt-4-turbo-2024-04-09"],
    // 2 bingAI(s)
    modelsConfig.bingAI: ["BingAI","Sydney"],
    // 1 chatGPTBrowser(s)
    modelsConfig.chatGPTBrowser: ["text-davinci-002-render-sha"],
    // 15 assistant(s)
    modelsConfig.assistants: ["gpt-3.5-turbo","gpt-3.5-turbo-0125","gpt-4-turbo","gpt-4-turbo-2024-04-09","gpt-4-0125-preview","gpt-4-turbo-preview","gpt-4-1106-preview","gpt-3.5-turbo-1106","gpt-3.5-turbo-16k-0613","gpt-3.5-turbo-16k","gpt-4","gpt-4-0314","gpt-4-32k-0314","gpt-4-0613","gpt-3.5-turbo-0613"],
}
2024-05-16T10:15:38.123Z debug: [BaseClient] Loading history:
{
  conversationId: "44e4324f-a17a-4a61-91cf-2a63dfd776ac",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
}
2024-05-16T10:15:39.423Z debug: [BaseClient] Context Count (1/2)
{
  remainingContextTokens: 127982,
  maxContextTokens: 127990,
}
2024-05-16T10:15:39.423Z debug: [BaseClient] Context Count (2/2)
{
  remainingContextTokens: 127982,
  maxContextTokens: 127990,
}
2024-05-16T10:15:39.424Z debug: [BaseClient] tokenCountMap:
{
  e3c5e678-1ec1-4428-920f-8bd7cf3b328f: 5,
}
2024-05-16T10:15:39.424Z debug: [BaseClient]
{
  promptTokens: 8,
  remainingContextTokens: 127982,
  payloadSize: 1,
  maxContextTokens: 127990,
}
2024-05-16T10:15:39.424Z debug: [BaseClient] tokenCountMap
{
  e3c5e678-1ec1-4428-920f-8bd7cf3b328f: 5,
  instructions: undefined,
}
2024-05-16T10:15:39.425Z debug: [BaseClient] userMessage
{
  messageId: "e3c5e678-1ec1-4428-920f-8bd7cf3b328f",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
  conversationId: "44e4324f-a17a-4a61-91cf-2a63dfd776ac",
  sender: "User",
  text: "test",
  isCreatedByUser: true,
  tokenCount: 5,
}
2024-05-16T10:15:39.836Z debug: [OpenAIClient] chatCompletion
{
  baseURL: "https://${EASTUS2_BASEURL}/openai/deployments/gpt-4-turbo-2024-04-09/chat/comp... [truncated]",
    modelOptions.model: "gpt-4-turbo-2024-04-09",
    modelOptions.temperature: 1,
    modelOptions.top_p: 1,
    modelOptions.presence_penalty: 0,
    modelOptions.frequency_penalty: 0,
    modelOptions.stop: undefined,
    modelOptions.max_tokens: undefined,
    modelOptions.user: "66442d5b7c4c582b181fabe0",
    modelOptions.stream: true,
    // 1 message(s)
    modelOptions.messages: [{"role":"user","content":"test"}],
}
2024-05-16T10:15:41.388Z warn: [OpenAIClient.chatCompletion][stream] API error
2024-05-16T10:15:41.390Z error: [handleAbortError] AI response error; aborting request: 404 Resource not found
2024-05-16T10:15:41.400Z debug: [AskController] Request closed

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Please see the documentation on this:
https://www.librechat.ai/docs/configuration/librechat_yaml/ai_endpoints/azure#using-a-specified-base-url-with-azure

The baseURL naming can be confusing, but it refers to "base" as everything in the URL before /chat/completions or /completions.

If you are using the standard Azure OpenAI domain, you should not set baseURL in your config, i.e. https://librechat.openai.azure.com (https://{INSTANCE_NAME}.openai.azure.com)

You can omit it altogether to resolve your issue.

You can use it, however, with dynamic placeholders or without them.

    baseURL: "https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}"

# or
    baseURL: "https://librechat.openai.azure.com/openai/deployments/gpt-35-turbo"

# or, since we are using the standard domain, we omit the field completely

Both the examples work for me in this config:

endpoints:
  azureOpenAI:
    groups:
    - group: "region"
      apiKey: "${REGION_API_KEY}"
      instanceName: "librechat"
      version: "2024-03-01-preview"
      baseURL: "https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}"
      models:
        gpt-3.5-turbo:
          deploymentName: "gpt-35-turbo"
        gpt-3.5-turbo-1106:
          deploymentName: "gpt-35-turbo-1106"
        gpt-4:
          deploymentName: "gpt-4"

Using the dynamic placeholders makes it so that you can keep the model/region switching behavior, which is the chief reason for the azure config via librechat.yaml; otherwise, it will use the hardcoded values no matter what the model values are, which can be unexpected.

deploymentName: "arbitrary-deployment-name" //This should not be arbitrary

This is not arbitrary, it's used to construct the baseURL if placeholders are included, or the standard baseURL if no field for it is included.

Arbitrary in the documentation means to say it's whatever value you defined, in this case, whatever you named the deployment on Azure.

As you can see, the model name was concatnated into url, but Azure requires DeploymentName there

Not sure where in the pipeline this is happening, probably from the legacy config, but it wouldn't happen if your baseURL is configured correctly.

TL;DR

When configuring baseURL in librechat.yaml for Azure, it refers to everything before /chat/completions or /completions. If using the standard Azure OpenAI domain, omit baseURL. You can use dynamic placeholders or omit the field entirely to avoid issues. Properly configuring baseURL ensures dynamic model/region switching, which is crucial for Azure configs. The deploymentName in the model config is necessary and should be what you named the deployment on Azure.

[hostage007]

Thanks for the detailed explanation, this design makes sense,(a good one indeed)
But to be honest, the vague doc and legacy config concatnating the url, make the setup and troubleshooting a hard job to do.

[hostage007]

Sorry to bother again, the config doesn't work. If you make the deploymentname different from model name, you wll see the error.
My librechat.yaml:

version: 1.1.0

cache: true

registration:
  socialLogins: ["discord", "facebook", "github", "google", "openid"]

endpoints:
  azureOpenAI:
    # Endpoint-level configuration
    titleModel: "Azure OpenAI"
    plugins: false
    assistants: false
    groups:
    # Group-level configuration
    - group: "changtsi-openai-swedencentral"
      apiKey: "${SE_CTR_API_KEY}"
      instanceName: "changtsi-openai-swedencentral"
      baseURL: "https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}"
      version: "2024-02-01"
      # Model-level configuration
      models:
        gpt-4-turbo-2024-04-09:
          deploymentName: gpt-4-turbo-2024-04-09
        gpt-35-turbo-0613:
          deploymentName: gpt-35-turbo-0613
        gpt-4-0613:
          deploymentName: gpt-4-0613
        gpt-4-32k-06132:    # typo on purpose 
          deploymentName: gpt-4-32k-0613-dev

---

debug log:

2024-05-21T08:30:26.893Z debug: [BaseClient] userMessage
{
  messageId: "ffbb4e8f-c8b5-45ec-be49-ec947b7ca5df",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
  conversationId: "6eba9edf-62e8-4632-b659-b63d7a1e71a4",
  sender: "User",
  text: "test",
  isCreatedByUser: true,
  tokenCount: 5,
}
2024-05-21T08:30:27.200Z debug: [OpenAIClient] chatCompletion
{
  baseURL: "https://changtsi-openai-swedencentral.openai.azure.com/openai/deployments/gpt-4-32k-06132/chat/compl... [truncated]",
    modelOptions.model: "gpt-4-32k-06132",
    modelOptions.temperature: 1,
    modelOptions.top_p: 1,
    modelOptions.presence_penalty: 0,
    modelOptions.frequency_penalty: 0,
    modelOptions.stop: undefined,
    modelOptions.max_tokens: undefined,
    modelOptions.user: "664ae41876ddc61fea146f12",
    modelOptions.stream: true,
    // 1 message(s)
    modelOptions.messages: [{"role":"user","content":"test"}],
}
2024-05-21T08:30:28.657Z warn: [OpenAIClient.chatCompletion][stream] API error
2024-05-21T08:30:28.659Z error: [handleAbortError] AI response error; aborting request: 404 The API deployment for this resource does not exist. If you created the deployment within ... [truncated]
2024-05-21T08:30:28.666Z debug: [AskController] Request closed

[danny-avila]

When I have the time to test your exact config, I'll post back. But for now, you can resolve this by omitting baseURL.

[danny-avila]

and for the record, the deployment name does not need to match your model name. Have tested this several times and have automated tests for this since it's so comprehensive.

In my example, both of these work. note the discrepancy in names

      baseURL: "https://${INSTANCE_NAME}.openai.azure.com/openai/deployments/${DEPLOYMENT_NAME}"
      models:
        gpt-3.5-turbo:
          deploymentName: "gpt-35-turbo"
        gpt-3.5-turbo-1106:
          deploymentName: "gpt-35-turbo-1106"