Dev Ops Crash Course - Day Three

Published Wednesday, March 22, 2017

Notes from day one, day two, day four, and day five.

Year in Review Review (continued)


Nagios repo w/ full documentation:

Nagios API that powers our status page:

Status page lives on nagios01 server: root@nagios01:/usr/local/nagios/nagios-api. There’s a cron job that runs updates: crontab -l to view, crontab -e to edit.

Automated SSH Key Propagation

Cron job that runs every 30min that runs chef user_setup recipe on every host.

Every 30min too often? We should scale this back.

Automated Server Provisioning Tools

Script that uses Digital Ocean, Dynect, and Chef APIs to provision the host automatically. New host in under 2 min.

Note: script is a little dated now. DO has since made its own CLI tools that do a lot of this (doctl). There’s also tugboat gem.

DO tools probably better suited since they incorporate all DO’s latest features.

Faster Deploys

Drew and Devin put work in to speed up server-side only deploys. More work needs to done on speeding up asset compilation.

Weekly DB Migrations

See qa-support server. Runs cron job: root@qa-support /etc/cron.d. Why cron.d? Convention Devin likes to use when running cron jobs that need to be accessible to users but run by different user (for example, postgres user).

Takes daily db dump, terminates all qa db connections, drops + restores db from dump.

See full docs on operations wiki.