Scrape KPIs via Lambda and scheduled events


This repo contains the source to deploy a bunch of cron like Lambdas written in Node.js. It uses ansible incombination with cloudformation to deploy the whole stack to AWS.


$ make check
$ make clean test-ci
# Edit src files
$ make test # run tests quickly

Directory Structure

├── cloudformation -- Templates to provision AWS resources
├── group_vars     -- Ansible variables
├── script         -- Non-production workflow scripts
├── src            -- Individual Lambda source code
├── templates      -- Ansible templates
├── test           -- Tests used CI
└── vendor         -- Third party code

This project uses a few different tools so the things are bit mixes (Ansible directory + node code). Plus there is the Dockerfile, Makefile, and Ansible playbooks (deploy.yml, teardown.yml). Ingeration tokens are kept in an ansible vault. Set ANSIBLE_VAULT_FILE to edit group_vars/all/vault.yml.


Deploys happen with ansible. The ansible playbook provisions AWS resources with cloudformation, creates a zip file (containing all Lambdas), runs a smoke test on the zip file, uploads the file to S3, then configures all the Lambdas to use the updated file on S3. Finally, a test invocation happens to verify the deploy.