[Bug] Gemini-Pro 对话格式和 ChatGPT 一样，那 Gemini-Pro-Vision 呢？

[bbb3n]

💻 系统环境

Windows

📦 部署环境

Vercel / Zeabur / Sealos

🌐 浏览器

Chrome

🐛 问题描述

我测试发现，我只填写了 OpenAI Key / Url （第三方中转）
gemini-pro 模型是可以按照 ChatGPT 格式调用的，这很棒
但是 gemini-pro-vision 无法按照 4v 的格式传图（会报错）

🚦 期望结果

希望如果不填写 Gemini 的 key 和 url ，仅填写 OpenAI 的 key 和 url 时：
gemini-pro 和 gemini-pro-vision 均支持 OpenAI 的调用格式

📷 复现步骤

问题描述已明确

📝 补充信息

No response

[lobehubbot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

💻 System environment

Windows

📦 Deployment environment

Vercel / Zeabur / Sealos

🌐 Browser

Chrome

🐛 Problem description

I tested and found that I only filled in the OpenAI Key / Url (third-party transfer)
The gemini-pro model can be called in ChatGPT format, which is great
However, gemini-pro-vision cannot transmit images in the 4v format (an error will be reported)

🚦 Expected results

I hope that if you do not fill in the key and url of Gemini, but only fill in the key and url of OpenAI:
gemini-pro and gemini-pro-vision both support OpenAI’s calling format

📷 Steps to reproduce

The problem description has been clarified

📝 Supplementary information

No response

[lobehubbot]

👀 @B3n-AI

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[arvinxx]

gemini-pro 和 gemini-pro-vision 均支持 OpenAI 的调用格式

gemini-pro-vision 的调用格式和 openai 是不一样的，需要做转换：https://github.com/lobehub/lobe-chat/blob/main/src/libs/agent-runtime/google/index.ts#L67

你填上 Google API key 就能用了，为啥还要走 OpenAI