Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

.Net: Add model diagnostics to non-streaming APIs #6150

Merged
merged 16 commits into from May 10, 2024
Merged

.Net: Add model diagnostics to non-streaming APIs #6150

merged 16 commits into from May 10, 2024

Conversation

TaoChenOSU
Copy link
Contributor

@TaoChenOSU TaoChenOSU commented May 7, 2024 •
edited

Motivation and Context

According to the ADR, it's essential that SK provides the best developer experience while complying with the industry standards for observability in generative-AI-based applications.

Description

This PR adds instrumentation to the chat completion and text completion endpoints in all AI connectors.

Streaming APIs will be worked on next.
The telemetry sample and the documentation will be updated in a separate PR.

Note that this is an ongoing effort, i.e. metrics and more events will be added as the conventions evolve.

Contribution Checklist

@TaoChenOSU TaoChenOSU requested a review from a team as a code owner May 7, 2024 22:42
@markwallace-microsoft markwallace-microsoft added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel labels May 7, 2024
@github-actions github-actions bot changed the title Add model diagnostics to non-streaming APIs .Net: Add model diagnostics to non-streaming APIs May 7, 2024
@TaoChenOSU TaoChenOSU self-assigned this May 7, 2024
stephentoub
stephentoub reviewed May 8, 2024
View reviewed changes
dotnet/src/Agents/Abstractions/Agents.Abstractions.csproj Outdated Show resolved Hide resolved
dotnet/src/Agents/Core/Agents.Core.csproj Outdated Show resolved Hide resolved
dotnet/src/Connectors/Connectors.Google/Core/Gemini/Clients/GeminiChatCompletionClient.cs Outdated Show resolved Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceMessageApiClient.cs Outdated Show resolved Hide resolved
dotnet/src/Connectors/Connectors.OpenAI/AzureSdk/ClientCore.cs Outdated Show resolved Hide resolved
dotnet/src/InternalUtilities/src/Diagnostics/ModelDiagnostics.cs Outdated Show resolved Hide resolved
dotnet/src/InternalUtilities/src/Diagnostics/ModelDiagnostics.cs Show resolved Hide resolved
dotnet/src/InternalUtilities/src/Diagnostics/ModelDiagnostics.cs Outdated Show resolved Hide resolved
dotnet/src/InternalUtilities/src/Diagnostics/ModelDiagnostics.cs Outdated Show resolved Hide resolved
dotnet/src/InternalUtilities/src/Diagnostics/ModelDiagnostics.cs Outdated Show resolved Hide resolved
@TaoChenOSU TaoChenOSU mentioned this pull request May 8, 2024
Open
@TaoChenOSU TaoChenOSU enabled auto-merge May 10, 2024 17:17
@TaoChenOSU TaoChenOSU added this pull request to the merge queue May 10, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 10, 2024
@TaoChenOSU TaoChenOSU added this pull request to the merge queue May 10, 2024
Merged via the queue into microsoft:main with commit 822a644 May 10, 2024
15 checks passed
@TaoChenOSU TaoChenOSU deleted the taochen/add-model-diagnostics-to-non-streaming-apis branch May 10, 2024 17:39
@TaoChenOSU TaoChenOSU mentioned this pull request May 14, 2024
Merged
4 tasks
github-merge-queue bot pushed a commit that referenced this pull request May 15, 2024
@TaoChenOSU
.Net: [WIP] OTel model diagnostics: streaming APIs (#6242)
b99b77f 
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->
Previously (#6150) we
added support for OTel (LLM semantic conventions) to non-streaming APIs
in the AI connectors. This PR adds that to streaming APIs.

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->
1. Add OTel (LLM semantic conventions) to streaming APIs.
2. Update the telemetry sample to use streaming APIs along with
non-streaming ones..

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alliscode alliscode alliscode approved these changes

@RogerBarreto RogerBarreto Awaiting requested review from RogerBarreto

@lmolkova lmolkova Awaiting requested review from lmolkova

@stephentoub stephentoub Awaiting requested review from stephentoub

Assignees

@TaoChenOSU TaoChenOSU

Labels
kernel.core kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code
Projects
Semantic Kernel
Status: Sprint: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@TaoChenOSU @lmolkova @stephentoub @alliscode @markwallace-microsoft