D

Data Engineering

ELT pipeline orchestrated with Apache Airflow, extracting YouTube data via the YouTube Data API v3, loading it into PostgreSQL (staging & core schemas), with data quality checks using Soda. Containerized with Docker Compose, tested with Pytest, and automated via GitLab CI/CD.

This repository holds code for the AskAnna Backend. Our backend stack primarily uses Django and the Django REST Framework.

tools for processing semantic data packaged as Nextflow modules

Live data source that can be used for data engineering, data warehouse and etl development.

Live data source that can be used for data engineering, data warehouse and etl development.

Live data source that can be used for data engineering, data warehouse and etl development.