You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now the render method has the following function signature:
/**
* The model name to use. Must be OpenAI SDK compatible. Tools and Functions are only supported
* GPT models (3.5/4), OpenAI Assistants, Mistral small and large, and Fireworks firefunction-v1.
*
* @example "gpt-3.5-turbo"
*/
model: string;
/**
* The provider instance to use. Currently the only provider available is OpenAI.
* This needs to match the model name.
*/
provider: OpenAI;
messages: Parameters<
typeof OpenAI.prototype.chat.completions.create
>[0]['messages'];
text?: Renderer<{ content: string; done: boolean }>;
tools?: {
[name in keyof TS]: {
description?: string;
parameters: TS[name];
render: Renderer<z.infer<TS[name]>>;
};
};
functions?: {
[name in keyof FS]: {
description?: string;
parameters: FS[name];
render: Renderer<z.infer<FS[name]>>;
};
};
initial?: ReactNode;
temperature?: number;
It would be really helpful (and useful) to support a wider range of parameters with an arbitrary dictionary (maybe named additionalParams). Namely, max_tokens is missing from this function signature and thus callers can't guide the length of outputs.
Use Case
This allows users to use not just OpenAI models but also a wider range of models hosted by open source inference infrastructure. As an example, SGLang and Lorax have mostly OpenAI compatible APIs but they don't fully support function calling. Instead they have output format specifications for controlled JSON results. Without arbitrary parameters passed through to the completion call, these other services can't be used fully.
Additional context
No response
The text was updated successfully, but these errors were encountered:
Feature Description
Right now the
render
method has the following function signature:It would be really helpful (and useful) to support a wider range of parameters with an arbitrary dictionary (maybe named
additionalParams
). Namely,max_tokens
is missing from this function signature and thus callers can't guide the length of outputs.Use Case
This allows users to use not just OpenAI models but also a wider range of models hosted by open source inference infrastructure. As an example, SGLang and Lorax have mostly OpenAI compatible APIs but they don't fully support function calling. Instead they have output format specifications for controlled JSON results. Without arbitrary parameters passed through to the completion call, these other services can't be used fully.
Additional context
No response
The text was updated successfully, but these errors were encountered: