metadata integration with Pinecone #437
Conversation
There was a problem hiding this comment.
@nsadeh thanks a lot for your contribution! Please check my comment.
| .build()) | ||
| .putAllFields(textSegments.get(i).metadata().asMap().entrySet() | ||
| .stream() | ||
| .collect(Collectors.toMap(Map.Entry::getKey, e -> Value.newBuilder().setStringValue(e.getValue()).build())))); |
There was a problem hiding this comment.
With this change, metadata is stored, but not retrieved during read. Please update PineconeEmbeddingStoreIT to extend EmbeddingStoreIT (instead of EmbeddingStoreWithoutMetadataIT). should_add_embedding_with_segment_with_metadata will fail.
There was a problem hiding this comment.
Thanks, I'll try to get to it today
There was a problem hiding this comment.
Changed as requested - is that it? Sorry, I am not familiar with Java testing
There was a problem hiding this comment.
All my tests succeed when I run ./mvnw test except for Neo4J which I doubt is related
There was a problem hiding this comment.
Hi @nsadeh, should_add_embedding_with_segment_with_metadata test still fails because there is no code that reads stored metadata key-value pairs when retrieving embeddings. You have added code for writing metadata, but not for reading back.
There was a problem hiding this comment.
@langchain4j latest commit should do it
There was a problem hiding this comment.
@nsadeh 3 tests are now failing:
should_add_embedding_with_segment
should_add_multiple_embeddings_with_segments
should_add_embedding_with_segment_with_metadata
There was a problem hiding this comment.
I am going to need a lot more hand-holding on the tests here.
When I run ./mvnw test, the pinecone tests succeed.
When I run these tests specifically from IntelliJ, they fail because they're missing an API key. I haven't been able to find the test api key in the repo.
What's the SOP here for running pinecone tests?
There was a problem hiding this comment.
@nsadeh sorry for the delayed response... You can registed on pinecone.io and get a free key for testing
Resolves #435