Add load_delta support with deltalake
package
#11960
Labels
area/tracking
Tracking service, tracking client APIs, autologging
enhancement
New feature or request
help wanted
We would like help from the community to add this support
Willingness to contribute
Yes. I can contribute this feature independently.
Proposal Summary
Load_delta should be added using the
deltalake
library. I am happy to contribute this myself since I work on delta-rs project.Load delta should return Polars or pandas datasets, and Polars should be added as optional dependency.
Motivation
Spark-delta is way to heavy for most usecases, hence the popularity of Python deltalake.
Support for single node ML usecases with Polars pipelines and deltalake as storage layer.
Would prevent going through pandas to interact with mlflow, improve tracking of deltalake table inputs.
Details
No response
What component(s) does this bug affect?
area/artifacts
: Artifact stores and artifact loggingarea/build
: Build and test infrastructure for MLflowarea/deployments
: MLflow Deployments client APIs, server, and third-party Deployments integrationsarea/docs
: MLflow documentation pagesarea/examples
: Example codearea/model-registry
: Model Registry service, APIs, and the fluent client calls for Model Registryarea/models
: MLmodel format, model serialization/deserialization, flavorsarea/recipes
: Recipes, Recipe APIs, Recipe configs, Recipe Templatesarea/projects
: MLproject format, project running backendsarea/scoring
: MLflow Model server, model deployment tools, Spark UDFsarea/server-infra
: MLflow Tracking server backendarea/tracking
: Tracking Service, tracking client APIs, autologgingWhat interface(s) does this bug affect?
area/uiux
: Front-end, user experience, plotting, JavaScript, JavaScript dev serverarea/docker
: Docker use across MLflow's components, such as MLflow Projects and MLflow Modelsarea/sqlalchemy
: Use of SQLAlchemy in the Tracking Service or Model Registryarea/windows
: Windows supportWhat language(s) does this bug affect?
language/r
: R APIs and clientslanguage/java
: Java APIs and clientslanguage/new
: Proposals for new client languagesWhat integration(s) does this bug affect?
integrations/azure
: Azure and Azure ML integrationsintegrations/sagemaker
: SageMaker integrationsintegrations/databricks
: Databricks integrationsThe text was updated successfully, but these errors were encountered: