A flexible data integration tool to help nonprofits connect to their data collection tools and ERP systems
-
Updated
May 11, 2021 - Python
A flexible data integration tool to help nonprofits connect to their data collection tools and ERP systems
This project showcases the implementation of a data pipeline using Apache Airflow. Leveraging the OpenWeather API, it efficiently fetches real-time weather data and performs ETL processing. Results are seamlessly stored in AWS S3 buckets for further analysis. Moreover, the integration of Slack notifications ensures timely alerts to myself.
Ready to go Apache Airflow stack for docker
Data Engineering using Apache Airflow
Projects related to ETL data pipelines, Kubernetes and Go REST API development
An EL pipeline built with Apache Airflow that downloads a file from the web uploads it to Google Cloud Storage, and creates an external table in BigQuery for data storage and analysis.
A comprehensive data engineering pipeline, orchestrates data workflows with Apache Airflow, Python, Kafka, Zookeeper, Spark, and Cassandra. Containerized using Docker: to deploy and scale effortlessly. This Etsy API Data Pipeline extracts, processes, and analyzes Etsy marketplace data—retrieving product listings, shop details, and reviews.
Invoice de-duplication via Azure Form Recognition, OpenAI, Apache Airflow and Redis Enterprise VSS
Data pipelines created and monitored using Airflow to feed data into Redshift
Apache Airflow config scripts for my very stubborn Mac 💻
Data pipeline that fetches recently played songs in the past 24 hours using Spotify API and saves the data in the SQLite database. Scheduled to run daily using Apache Airflow.
This is a tool to collect title and url from videos posted on youtube.
POC do curso Apache Airflow: The Hands-On Guide utilizando dados de paridade de moedas financeiras
This project is destinate a study for create a environment Apache Airflow
How to configure Apache Airflow using CDK (Cloud Development Kit)
Entire ETL pipeline project from data ingestion, transformation and finally analytics with Google Looker Studio
Add a description, image, and links to the apache-airflow topic page so that developers can more easily learn about it.
To associate your repository with the apache-airflow topic, visit your repo's landing page and select "manage topics."