Atividades do curso "Fundamentos de Engenharia de Dados" da DataScienceAcademy.

This repository is the collection point for all of the projects completed during the Udacity Data Engineering Nano Degree program.

Building Machine Learning and ETL Pipelines to categorize emergency messages based on the needs communicated by the sender

Two data frames of different kaggle cases of diesease cases and weather in Brazil. The project aims to clean the DFs and build a new one in order to analyse the correlation of dengue (serious disease transmited by mosquito), rain precipitation and temperature.

Analysis of NYC's citibike data. Technologies: Python , Prefect, dbt, Terraform , Looker data studio

IGTI Enhenheiro de Dados - Módulo 5 Desafio Final

Data ingestion solution using spring batch and postgreSQL as data warehouse.

ETL Pipeline for Music Analysis

Pipelines de Airflow - códigos de exemplo

Experimenting with Data Pipelines in Python

cryptocurrency ticker data pipeline

The Security Reference Architecture (SRA) implements typical security features as Terraform Templates that are deployed by most high-security organizations, and enforces controls for the largest risks that customers ask about most often.

Projects and Exercises for Udacity Data Engineering Nano Degree

Build Data Pipeline with pgAdmin, AWS Cloud and Apache Spark to Analyze and Determine Bias in Amazon Vine Reviews

The purpose of the project is to efficiently collect, process, and store Twitter data using a combination of Apache Airflow, Apache Spark, and Amazon S3.

Building an ETL pipeline for a database hosted on Redshift. Extracting data from S3 to staging tables on Redshift . Transforming data by executing SQL statements that create the analytics tables from these staging tables by start schema. Loading star schema tables to Redshift

An ETL pipeline that extracts, transforms, and loads data from various sources related to electric vehicle (EV) stocks.

A data pipeline that conducts ETL processes to AWS Redshift, utilizing Spark and coordinated by Apache Airflow.

ETL Pipeline for Shopping Data