@himadrisingh : Can you get this assigned on Infra side and prioritize it ?

This will increase the size of the cli though?

Implementation proposal:

• To maintain a single binary, we need to use Go's embed package to embed the DuckDB extensions we use in the binary. See runtime/pkg/examples for an example of how we use it to embed example projects.
• We should implement the embedding in our DuckDB driver (runtime/drivers/duckdb) and unpack the embedded extensions before first opening a DuckDB handle.
  • We should probably unpack to ~/.duckdb/extensions, which is DuckDB's default location for extensions (notice a specific nested folder structure is required).
  • We should avoid unpacking an extension if it already exists (for faster startup after the first run).
• We also need a script that downloads the DuckDB extensions we rely on and places them in the embed directory (probably runtime/drivers/duckdb/embed). The script could be placed in the scripts directory and invoked when make cli is executed (in the cli.prepare Make target)
• Lastly, we need to embed different DuckDB extensions for different architectures (they are architecture specific). Since we build for multiple architectures using a cross-compiler, we can't just download extensions for the current system architecture. There's a few ways to solve this issue – here's one proposal:
  • Have the download script download extensions for all supported architectures and place them in runtime/drivers/duckdb/embed/<arch>/<duckdb version>/
  • Use Go build tags to only embed extensions for the current architecture. For example, runtime/drivers/duckdb/embed_darwin_arm64.go would have the build tag //go:build darwin && arm64 and embed ./embed/darwin_arm64/**