extract transform and load
Tracks stock prices, commodities, forex rates, inflation rates. Your job is to extract financial data from various sources like websites, APIs and files provided by various financial analysis firms. After you collect the data, you extract the data of interest to your company and transform it based on the requirements given to you. Once the transformation is complete you load that data into a database.
In this project you will:
-
Collect data using APIs
-
Collect data using webscraping.
-
Download files to process.
-
Read csv, xml and json file types.
-
Extract data from the above file types.
-
Transform data.
-
Use the built in logging module.
-
Save the transformed data in a ready-to-load format which data engineers can use to load the data.
Extract transform and load and transform tasks flow in airflow GUI
