Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added cosine similarities to metadata for ProvenanceV0 #610

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

yadomkar
Copy link

@yadomkar yadomkar commented Mar 4, 2024

This PR addresses issue #575, enhancing the validate_each_sentence and validate_full_text methods in the ProvenanceV0 validator to include cosine similarity values in the validation metadata, providing insights into how closely each sentence matches the source text

@codecov-commenter
Copy link

codecov-commenter commented Mar 5, 2024

Codecov Report

Attention: Patch coverage is 0% with 8 lines in your changes are missing coverage. Please review.

❗ No coverage uploaded for pull request base (main@ce61fe6). Click here to learn what that means.

Files Patch % Lines
guardrails/validators/provenance.py 0.00% 8 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #610   +/-   ##
=======================================
  Coverage        ?   81.80%           
=======================================
  Files           ?      118           
  Lines           ?     6764           
  Branches        ?        0           
=======================================
  Hits            ?     5533           
  Misses          ?     1231           
  Partials        ?        0           
Flag Coverage Δ
unittests 81.80% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ShreyaR
Copy link
Collaborator

ShreyaR commented Mar 5, 2024

Thanks for creating the PR @yadomkar! Can you update the tests as well? You'll need to modify this test

@yadomkar
Copy link
Author

yadomkar commented Mar 6, 2024

@ShreyaR Thank you for the feedback! Currently, there doesn't seem to be existing test coverage for ProvenanceV0 that I could directly extend to include checks for the cosine similarity metadata. My changes were specifically for ProvenanceV0, and I haven't touched ProvenanceV1 as it uses a different method. Should I proceed with adding coverage for ProvenanceV0, or is there an existing suite I might have overlooked?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants