-
Notifications
You must be signed in to change notification settings - Fork 508
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support BF16 and FP16 - [MOD-6765, MOD-6776] #4584
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #4584 +/- ##
==========================================
- Coverage 86.18% 86.17% -0.01%
==========================================
Files 190 190
Lines 34508 34549 +41
==========================================
+ Hits 29739 29774 +35
- Misses 4769 4775 +6 ☔ View full report in Codecov by Sentry. |
71b41d0
to
1b6295c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice and clean
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool!
What was decided regarding the infinity check? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just a small comment
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 2.10
git worktree add -d .worktree/backport-4584-to-2.10 origin/2.10
cd .worktree/backport-4584-to-2.10
git switch --create backport-4584-to-2.10
git cherry-pick -x 4a6982d248b6c93064bea0f867b33907bc0d7754 90c7f1b3e0a9d8aa29c7b81db8e8e6130626f504 9683d459bc536471dfc49cb48affeaec146b17e4 079cfcdb9609aa47f6ec63af9d8254d95b390873 10ba3e3cf34aa23f50c83b7c29ae9c71309ea9bb f68a4c7d1926fcf07f0f93e1ae4fd0a27fa92333 ee244a9c52517db9057c4897fb217942673f6db3 1b6295c88803c7b18cd4dc8295b4b867e73e64bc 7b6c88cd7be3b8c7ca2d51c3599c5abdd2fca654 45c253b9208409207a09bd486db025d877e5d9df 0e987681502ca3f1a9efec609b94d6e3700cdcc2 c78e0addc7bb2fd6989318f5ef9ab822b3211668 456ba0fcf88e3eee2aa9b4756acabea9cde2fb38 bad8beff13f6e609697b766f604149af382c5b04 3c20817b89263210c8a36a5f6386c0e31b6d3b49 117abae6f09e0c66d367da2a9d85ee1975c340fe e99e1d36486031604240b82f5c1673d881019e9b 39a47ef9084783b94768867628bc04c129c422a2 450a4b0eb21d309eff0f35334e23a3732084c0d0 2f43ccb04ea9d189e76e51b2cda21ee07f7da1e1 f359daa8da18478f1c80a7eb852c55260a9ea6ea c22511933857c442cb3bf54629503c9e1f62ddb7 5403e4b44e21bb12624cd4bbff22bb5a8ab064a7 531a7e8108d927ff6ae33d70319fc0d33e390de0 2d1f4f6fe1e555b5d4e4a5e4e6a65d93241a04f2 fb8edb1ac1f04e1022aed27252cf3d50deba8c7a 740d6487c6a77ee51db76bc96deaa570839f7299 |
* adding support for the bf16 type in the API * support JSON casting to BF16 * improved casting * add support for fp16 * update VecSim tag (supports fp16 and bf16) * remove temporary definitions * added tests * minor fix * review fixes (tests) * more data types in multithread * fix another test * remove NaN/Inf support * replace bfloat16 module with tensorflow * limit urllib3 version * remove cpu only tensorflow * set jsons from numpy array * fix build for arm ubuntu * attempt * attempt * test * attempt * attempt * another attempt * implement bfloat16 myself * revert installation scripts * simplify create_np_array_typed * extend comment
Describe the changes in the pull request
Implementing the support for
BFLOAT16
andFLOAT16
on the RediSearch side.This PR includes a basic casting function for the vector elements when extracting them from JSON.
TODO:
VecSimType_BFLOAT16
andVecSimType_FLOAT16
onvector_index.h
Main objects this PR modified
BFLOAT16
andFLOAT16
types to theFT.CREATE
API for the vector fieldBFLOAT16
andFLOAT16
Mark if applicable