Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: completions api flush after first calling #807

Merged
merged 2 commits into from
May 13, 2024
Merged

feat: completions api flush after first calling #807

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
016e222
feat: completions api flush after first calling
woorui May 9, 2024
c44d2e1
chore: bump deps to last version
woorui May 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ require (
github.com/quic-go/quic-go v0.42.0
github.com/reactivex/rxgo/v2 v2.5.0
github.com/robfig/cron/v3 v3.0.1
github.com/sashabaranov/go-openai v1.23.0
github.com/sashabaranov/go-openai v1.23.1
github.com/second-state/WasmEdge-go v0.13.4
github.com/shirou/gopsutil/v3 v3.24.2
github.com/spf13/cobra v1.8.0
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,8 +169,8 @@ github.com/sagikazarmark/locafero v0.4.0 h1:HApY1R9zGo4DBgr7dqsTH/JJxLTTsOt7u6ke
github.com/sagikazarmark/locafero v0.4.0/go.mod h1:Pe1W6UlPYUk/+wc/6KFhbORCfqzgYEpgQ3O5fPuL3H4=
github.com/sagikazarmark/slog-shim v0.1.0 h1:diDBnUNK9N/354PgrxMywXnAwEr1QZcOr6gto+ugjYE=
github.com/sagikazarmark/slog-shim v0.1.0/go.mod h1:SrcSrq8aKtyuqEI1uvTDTK1arOWRIczQRv+GVI1AkeQ=
github.com/sashabaranov/go-openai v1.23.0 h1:KYW97r5yc35PI2MxeLZ3OofecB/6H+yxvSNqiT9u8is=
github.com/sashabaranov/go-openai v1.23.0/go.mod h1:lj5b/K+zjTSFxVLijLSTDZuP7adOgerWeFyZLUhAKRg=
github.com/sashabaranov/go-openai v1.23.1 h1:b2IsEG9+BdJ3f6G3gGu9Lon2Mw/C0aYqME3YzwBHcls=
github.com/sashabaranov/go-openai v1.23.1/go.mod h1:lj5b/K+zjTSFxVLijLSTDZuP7adOgerWeFyZLUhAKRg=
github.com/second-state/WasmEdge-go v0.13.4 h1:NHfJC+aayUW93ydAzlcX7Jx1WDRpI24KvY5SAbeTyvY=
github.com/second-state/WasmEdge-go v0.13.4/go.mod h1:HyBf9hVj1sRAjklsjc1Yvs9b5RcmthPG9z99dY78TKg=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
Expand Down
10 changes: 7 additions & 3 deletions pkg/bridge/ai/service.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -436,6 +436,7 @@
ToolCalls: toolCalls,
Role: openai.ChatMessageRoleAssistant,
}
flusher.Flush()

Check warning on line 439 in pkg/bridge/ai/service.go

View check run for this annotation

Codecov / codecov/patch

pkg/bridge/ai/service.go#L439 

Added line #L439 was not covered by tests
Copy link
Collaborator Author

@woorui woorui May 9, 2024 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

responser flushes after first calling is completed

}
} else {
resp, err := s.LLMProvider.GetChatCompletions(ctx, req, s.Metadata)
Expand Down Expand Up @@ -487,7 +488,7 @@
ylog.Debug(" #2 second call", "request", fmt.Sprintf("%+v", req))

if req.Stream {
flusher := eventFlusher(w)
flusher := w.(http.Flusher)

Check warning on line 491 in pkg/bridge/ai/service.go

View check run for this annotation

Codecov / codecov/patch

pkg/bridge/ai/service.go#L491 

Added line #L491 was not covered by tests
resStream, err := s.LLMProvider.GetChatCompletionsStream(ctx, req, s.Metadata)
if err != nil {
return err
Expand Down Expand Up @@ -523,8 +524,11 @@

// run llm-sfn function calls
func (s *Service) runFunctionCalls(fns map[uint32][]*openai.ToolCall, reqID string) ([]ai.ToolMessage, error) {
if len(fns) == 0 {
return nil, nil

Check warning on line 528 in pkg/bridge/ai/service.go

View check run for this annotation

Codecov / codecov/patch

pkg/bridge/ai/service.go#L527-L528 

Added lines #L527 - L528 were not covered by tests
}

asyncCall := &sfnAsyncCall{
wg: &sync.WaitGroup{},
val: make(map[string]ai.ToolMessage),
}
s.muCallCache.Lock()
Expand Down Expand Up @@ -595,7 +599,7 @@
}

type sfnAsyncCall struct {
wg *sync.WaitGroup
wg sync.WaitGroup
mu sync.RWMutex
val map[string]ai.ToolMessage
}
Expand Down