DataOps for the Modern Data Warehouse on Microsoft Azure. https://aka.ms/mdw-dataops.
-
Updated
May 27, 2024 - Shell
DataOps for the Modern Data Warehouse on Microsoft Azure. https://aka.ms/mdw-dataops.
Terraform script to deploy almost all Azure Data Services
Azure Container Instances Proxy implemented in Azure Function App (Consumption Plan)
Threat Detection and Visualization
This pipeline is an ETL application, which aims to collect data for a few minutes from Binance's open API for cryptocurrencies: BTCUSDT, ADAUSDT, ETHUSDT, BNBUSDT and LTCUSDT.
A data pipeline project build on databricks and azure to demostrate lifecycle of a cloud data project.
Generic Pipelines / Templates for Data Factory / Synapse Pipelines w.r.t Different MSFT Offering Integrations / Use Cases
Modern Dataware House Demos with Azure Databricks, Azure Data Factory & Azure Dedicated SQL pool (formerly SQL DW)
This repository serves as a sample demonstration of implementing CI/CD practices on Azure Data Factory using Azure DevOps. It provides an example approach to configure a pipeline for continuous integration and continuous deployment (CI/CD) workflows on Azure Data Factory
Interesting_programs_written_in_Python_language
Explore the Tokyo Olympics data journey! We ingested a GitHub CSV into Azure via Data Factory, stored it in Data Lake Storage Gen2, performed transformations in Databricks, conducted advanced analytics in Azure Synapse, and visualized insights in Synapse or Power BI.
PsDataFactory is a PowerShell module to generate DataFactory pipelines and datasets.
This application shows how to use the Azure .NET library to create and execute Azure Data Factory from C# code.
This project showcase the following: 1. REST Connection to CHAT GPT 2. Logging system to Azure file system, storage app and insights. 3. Azure Data Factory Rest implementation using MS Graph client.
A random parameter generator, can generate random query parameters from excel definition or argument objects.
Azure Data Factory template to refresh Power BI Dataset
Data engineering with Azure services
Add a description, image, and links to the datafactory topic page so that developers can more easily learn about it.
To associate your repository with the datafactory topic, visit your repo's landing page and select "manage topics."