H

hadoop

1000+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Docker, CI/CD, APIs, SQL, PostgreSQL, MySQL, Hive, Impala, Kafka, Hadoop, Jenkins, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, tmux..

The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, ...

End-to-end design of a Hadoop-based ecosystem for healthcare data at scale (50 TB, IoT streams, medical imaging). Proposed a 10-node cluster architecture integrating HDFS, Spark, Hive, NiFi, Kafka, and Docker with HIPAA-compliant security (Kerberos, TLS, Apache Ranger). Delivered a proof-of-concept Docker deployment and professional proposal document.

Analyzed decades of historical weather station data (1920–1940) using Hadoop MapReduce. Filtered operable stations, computed descriptive statistics (min, max, mean, median), and produced reports/graphs. Designed modular MRJobs to chain tasks together for scalable processing.

Helper image that extends Java build tools container with additional prereqs needed to compile Hadoop from source for ARM64. This includes backported patches for things like protobuf.

Strategic Information Management - Hadoop with Spark

This web app finds the best configuration of a Spark Application given the hardware of the cluster

Workshop de Big Data a cargo de Jimmy Farfán docente del curso online "Desarrollo de Aplicaciones de Big Data en Hadoop". Si requieren más información o cualquier duda pueden ubicarnos en facebook como Data Hack Formation.

This project was an exercise for the Master in Big Data Engineering and Data Science at "Universidad Autónoma de Madrid". See the readme.md for more information.

This project performs SQL operations on a CSV input in HDFS, using Hadoop's Map-Reduce.

Machine Learning examples

A docker image for Halyard SDK (including Apache Hadoop and Apache HBase).

A base docker image for Apache Hadoop and Apache HBase clients.

A coherent introduction to the Hadoop environment and HDFS.

This project aggregates trending data from Ukraine based Twitter accounts. The raw aggregated data is cleansed before analysis using some Big-data methods. The purpose of this project is to familiarize myself with the workings of Hadoop for HDFS and Map-Reduce infrastructure.

Explaining usage of classroom spaces and trending subjects using classroom usage data.