Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

支持Gemini流式回复 #806

Merged
merged 3 commits into from
May 29, 2024
Merged

Conversation

YouJiacheng
Copy link
Contributor

@YouJiacheng YouJiacheng commented May 18, 2024

#805
顺便,我感觉支持GPT API的代码里的data.result = result.replace(/^\n\n/, "");是没必要的?SSE用"data: "分割之后确实带\n\n,但是这会被JSON.parse忽略。
Gemini API每一个chunk都会有"finishReason": "STOP",所以不能用这个判断停止。
Gemini API不会以data: [DONE]结束,所以只需要特判split("data: ")之后第一个是空串。
顺便,GPT API应该用data: [DONE]判断结束,而不是finish_reason非null。

@YouJiacheng
Copy link
Contributor Author

YouJiacheng commented May 18, 2024

本地测试通过。
有个无关的地方需要修改才能build

diff --git a/src/modules/prompt.ts b/src/modules/prompt.ts
index fd5457d..904342c 100644
--- a/src/modules/prompt.ts
+++ b/src/modules/prompt.ts
@@ -146,7 +146,7 @@ export function registerPrompt() {
                       node
                         .querySelectorAll("span")
                         .forEach((e) => (e.style.backgroundColor = ""));
-                      span.style.backgroundColor = highlightColor;
+                      (span as any).style.backgroundColor = highlightColor;

                       twinNode
                         ?.querySelectorAll("span")

we don't need to remove \n\n, but still need to copy result.
newResponse should not be parsed before splitting
@YouJiacheng YouJiacheng reopened this May 18, 2024
@windingwind
Copy link
Owner

谢谢。请问现在是可以合并的状态吗?(我不使用gemini因此未做测试)

@YouJiacheng
Copy link
Contributor Author

可以的,有一个concern是我这里假设了onProgress的边界和SSE是匹配的,即每次onProgress一定是整数个data,不会出现半个。

这个假设对Google目前测试下来是成立的,而且与GPT流式回复支持的实现蕴含的假设是类似的。

但我发现GPT流式回复支持这边,因为第三方的兼容API比较多,这个假设有时候是不成立的,比如零一万物的API就比较奇怪:

  1. 有时会把data: {...}\n\n分开;
  2. 有时会把data: {...}\n\n分开。

GPT流式回复支持的实现允许第一种情况,把\n\n通过try catch吃掉了,但对于第二种情况会漏数据。

@windingwind
Copy link
Owner

谢谢,那我合并发布了。关于gpt的实现问题欢迎在单独issue/pr讨论,谢谢贡献

@windingwind windingwind merged commit d7a91b7 into windingwind:main May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants