feat: Astra DB Source Connector Support #2928
Conversation
|
@erichare I'm taking a look at this. Can you give me a description on why someone would use Astra DB as a source connector? We don't have any other examples of a vector database as a source. Unless this is acting as a non-vector db? Just want to know the use case. |
@potter-potter You are spot on, the use case is essentially that Astra DB can act as a non-vector DB, and in fact up until quite recently was solely a non-vector DB. There are customers that do have standard tabular databases that are trying to migrate to vector, and so the source connector would aid in that migration. I think (correct me if i'm wrong) that it's kinda similar to Mongo being a source connector in that sense, and the description of Mongo's source connector is pretty much the same as we were thinking for Astra. I actually had the same question at first and had to clarify as well haha. Let me know if that makes sense or there's any other clarification needed. |
|
@erichare Cool! Thanks for the explanation. I will combine this in a PR later today with the Destination updates. (after I take a close look at it) |
This pull requests adds support for Astra DB as a source connector, pulling in data from Astra DB for use in Unstructured and applying appropriate metadata.