DFLib ("DataFrame Library") is a lightweight pure Java implementation of a common DataFrame data structure.
DataFrames exist in Python (pandas), R, Spark and other languages and frameworks. DFLib's DataFrame is specifically
intended for Java and JVM languages.
With DataFrame API, you get essentially the same data manipulation capabilities you may be used to in SQL (such as
joins, etc.), only you apply them in-memory and over dynamically defined "table" objects. While SQL is "declarative",
DataFrame allows step-by-step transformations that are somewhat easier to understand and much easier to compose.
DataFrame is extremely versatile and can be used to model a variety of data tasks. ETL, log analysis, spreadsheets
processing are just some of the examples. DFLib comes with connectors for many data formats:
CSV, Excel, RDBMS, Avro, JSON and can be easily adapted to other formats (e.g. web-service-based ones like
Google Sheets, etc.)
DFLib provides integration with Apache Echarts to visualize DataFrame data. Charts are generated in a form of HTML/JavaScript code and work in Jupyter as well as regular web applications.
While DFLib works in any Java application, it has a special intergation with Jupyter Notebook, a browser-based interactive environment for data exploration and analysis popular among data scientists and data engineers. In fact, our community maintains a Java "kernel" for Jupyter as a sister project to DFLib.
- DataFrame, a Swiss Army Knife of Java Data Processing, Nashville JUG, 03/2024
- Early version of DFLib, Frankfurt WO Day, 04/2019