DevOps: log aggregation
This course covers log aggregation for a Rails application running across several containers and machines. It starts from the existing production logging setup and moves from prose log lines to structured JSON events. It shows how to carry a request id into background jobs and how an agent ships logs off each host. It then looks at how labels, retention tiers and sampling affect storage cost. It also covers what must never be logged, how to handle personal data, and a final configuration that ties the pieces together.
What you'll learn
- ✓Writing structured JSON log events with stable event names and named fields
- ✓Carrying a request id from web requests into background jobs
- ✓Shipping logs through a host agent instead of from the application
- ✓Controlling storage cost with few labels, tiered retention and sampling by request
- ✓Keeping credentials and personal data out of logs
- ✓Configuring Rails logging with lograge and a shared event helper
When tail -f stops working
On one server, debugging is kamal app logs -f and reading. It works, it is fast, and it stops working at a specific and recognizable moment: when the answer to what happened to that request requires knowing which of six machines handled it.
Read the full lesson free →Curriculum
7 lessonsWhen tail -f stops working
Why tail -f fails across many machines, and which four logging decisions this app already gets right.
Structured logs: from prose to data
Turning prose log lines into JSON events with lograge, stable event names, named fields and meaningful levels.
Correlation: following one request
Passing the request id into background jobs and logging business identifiers instead of personal data.
Shipping logs off the machine
Why the app writes only to STDOUT while a host agent parses, enriches, buffers and ships logs.
Storing and searching without a large bill
Estimating log volume, keeping labels low-cardinality, tiering retention and sampling by request instead of by line.
What to log, and what never to log
Which secrets never belong in logs, why to log ids instead of personal data, and where to redact.
Putting it together: a logging setup
A complete logging configuration for this app, from lograge and a shared event helper to tiered retention.