E

ETL

Rustroid Sentinel is a high-performance backend system built to directly integrate with the NASA NeoWs API, continually extracting and analyzing data on potentially hazardous near-Earth objects (NEOs). Built entirely in Rust using the tokio ecosystem, the service delivers a lightning-fast, highly concurrent data pipeline designed for demanding operational environments.

ETL pipeline project using Python, pandas and SQLite to import and structure CSV, Excel and HTML data files.

This project is a realtime streaming data collection system based on kafka and spark.