Audit Logging and Traceability
IntermediateHow audit logs differ from application logs, what every audit entry must contain, how to design an immutable append-only audit store, and the operational patterns for querying, partitioning, and securing audit data at scale.
Step 1 of 6
Audit Logs vs Application Logs: Not the Same Thing
Application logs exist to help engineers debug running systems: stack traces, request durations, query plans. They are ephemeral by design: retained for days or weeks, then rotated or deleted. No compliance framework treats application logs as evidence of anything.
Audit logs are different in kind, not just degree. An audit log is the immutable record of who did what to which resource at what time: and whether it succeeded. SOC 2, HIPAA, and PCI DSS all require audit trails that are tamper-evident and retained for 1–7 years. Forensic investigations and authorization debugging both depend on them.
Every audit log entry requires exactly four fields to be useful: who (actor_id : a service account, user ID, or API key, never a display name), what (action name and resource identifier), when (a monotonic server-side timestamp: not client-supplied), and outcome (success or failure, with failure reason). Missing any one of these four fields makes the log entry ambiguous or useless under interrogation.
Application logs and audit logs should live in separate storage with separate retention policies and separate access controls. Conflating them means either your audit logs get rotated with your debug logs, or your debug logs balloon with audit retention requirements.
Key Takeaways
- →Audit logs are compliance evidence: not debug output: and must be treated as immutable records from day one
- →Every audit entry needs exactly four fields: who, what, when, outcome: missing any one makes it non-actionable
- →Separate audit log storage from application log storage: different retention, different access controls, different cost profile