feat: Data Connectors (SQL)

[davidkircos]

Goals
Add a SQL connection
Support Postgres, MySQL
Connection Code Editor
Run code to get data in sheet
Basic schema viewer

• Update branch to main @jimniels
• Merge Connector Service branch @ddimaria

Client

• List connections @jimniels
• Create a connection @jimniels
• Test a connection @jimniels @ddimaria
• Edit a connection @jimniels
  • Done but has security considerations, like which fields are editable based on being sensitive vs. not
• Delete a connection
  • We'll have to handle the lifecycle of a connection, e.g. somebody adds a connection to a file, runs a query, then later deletes that connection so when you run the query again, what should the user see?
• Select a connection in cell type menu @jimniels
• Code editor connection
• Pull data
• Display schema
• Fix all TODO: (connections) in the PR
• Add mix panel events where it makes sense for connection-related interactions

Core

• Add Connector cell type @ddimaria @davidfig

API

• Endpoints for creating, retrieving, and editing connections @jimniels
  • POST /connections for { type: 'postgres' } or { type: 'mysql' }
  • GET /connections
  • GET /connections/:uuid (TODO: should not return sensitive info)
  • PUT /connections/:uuid
  • GET /internal/user/:auth0Id/connections/:uuid (returns sensitive info)
• Permissions for connections
• Tests for above endpoints
• [blocked] Securely store credentials @davidkircos
• Security considerations for sending sensitive info vs. non-sensitive info

Connections Service

• Create connections service (@ddimaria) Axium, Docker, Etc [ similar to Files and Multiplayer ]
• Endpoint for testing connection - Postgres
• Endpoint for retrieving data - fetch credentials from API - Postgres
• Postgres support for schema retrieval
• Mysql support for testing, querying and schema retrieval
• Add supported databases to docker and include in tests

Deployment

• Static IP for server so people can whitelist
• Create/publish docs for connections

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
quadratic	❌ Failed (Inspect)		May 22, 2024 11:04pm