İçeriğe Atla
Mustafa Erbay
Tutorials · 5 min read · görüntülenme Türkçe oku
100%

Privileged Command Monitoring Runbook on Linux with Auditd

A practical approach that makes privileged operations observable and auditable in production using sudo, auditd rules, and log forwarding.

Privileged Command Monitoring Runbook on Linux with Auditd — cover image

In production, the question “who did what, and when?” usually arrives after an incident. The real problem is that the answer to this question doesn’t sit in a single log line; it’s scattered across different channels such as sudo policy, shell history, the process tree, and file access traces. Auditd is no magic wand here; but when designed correctly, it’s one of the most reliable layers that makes privileged operations operationally observable.

In this article, we’ll lift auditd out of the “installed and forgotten” state and bind it to a runbook: which events to collect, how to reduce noise, how to preserve performance, and how to make sense of it on the SIEM side.

Goal: tying privileged operation traces into a single story

I define the minimum target this way:

  • Identity: Which user (and ideally which session/pam context) did it?
  • Action: Which command ran, which files were touched?
  • Context: Which host, which role (prod/stage), which change window?
  • Trail: Did it reach the central collector without log loss?

Auditd doesn’t always give you “the full command string”; however, since it provides process and file access traces at the syscall level, it builds a very valuable evidence chain.

Design choice: not “many rules”, but “few but reliable rules”

The fastest way auditd dies in production is an uncontrolled rule set. Excessive event generation:

  • Creates disk I/O and journald/audit backlog pressure
  • Raises SIEM cost and noise
  • Makes it harder to find what you’re looking for during an incident

So my approach is two-layered:

  1. Baseline privileged trail: root access, sudo usage, critical file changes
  2. Incident-time expansion: “temporary” extra rules opened only during an incident

Installation and service health check

Even if package names vary across distributions, the health check is the same:

sudo systemctl enable --now auditd
sudo systemctl status auditd --no-pager
sudo auditctl -s

In the auditctl -s output, particularly track these fields:

  • enabled 1 (active)
  • backlog_limit (queue capacity under load)
  • lost (number of lost events)

Critical trail areas: sudo, root, and configuration files

1) Sudo usage (identity trail)

Sudo events usually exist in auth logs; but with auditd you capture them more deterministically. The example below focuses on monitoring execution of the sudo binary.

sudo auditctl -a always,exit -F path=/usr/bin/sudo -F perm=x -F auid>=1000 -F auid!=4294967295 -k priv-sudo

Notes:

  • auid carries the real user (even if euid is root via sudo).
  • The -k key is gold for fast filtering on the SIEM side.

2) Writes to critical files (change trail)

The list of critical files in production varies by organization, but a typical baseline:

  • /etc/sudoers and /etc/sudoers.d/
  • SSH: /etc/ssh/sshd_config, /root/.ssh/authorized_keys
  • Service definitions: systemd unit files
  • Network: firewall rule sets, VPN configs

Example:

sudo auditctl -w /etc/sudoers -p wa -k cfg-sudoers
sudo auditctl -w /etc/ssh/sshd_config -p wa -k cfg-sshd
sudo auditctl -w /etc/systemd/system -p wa -k cfg-systemd

This captures “the file was written”; for what the content actually is, you need additional layers like change management (GitOps / config repo) and file integrity.

3) Privileged process chain (suspicious behavior trail)

A critical mindset: not just “which command”, but “which process tree” matters. While correlating auditd events:

  • Derive the process tree from pid / ppid relationships
  • See which binary ran via the exe field
  • Evaluate cwd and comm fields together

Sudo I/O logging: completing the command + output trail

Where auditd falls short, sudo’s I/O log feature steps in. But it’s more sustainable to target this for critical roles instead of enabling it “for everything”.

Example sudoers approach:

Defaults log_output
Defaults iolog_dir="/var/log/sudo-io/%{user}"
Defaults!ALL iolog_file="%Y%m%d_%H%M%S_%{command}"
Defaults iolog_compress

Runbook: rapid verification during an incident

I’ve used the following flow many times in real life:

  1. Scope: Define host + time range + user/auid
  2. Sudo trail: Filter by the priv-sudo key
  3. Config change: Look at the cfg-* keys
  4. Expand the trail: Add a temporary rule if needed (narrow scope!)
  5. Correlation: Join with process chain / auth log / deployment log on the SIEM

Command examples:

sudo ausearch -k priv-sudo --start today
sudo ausearch -k cfg-sshd --start 2026-04-13 --end now
sudo aureport --auth --summary

SIEM side: carry “meaning”, not “events”

Audit logs are noisy in their raw form. Normalize the fields that are useful on the SIEM:

  • key → use-case label (like priv-sudo, cfg-sshd)
  • auid / uid / euid → identity triple
  • exe / comm → executed binary
  • hostname / env → prod/stage separation

Conclusion

Auditd, used with the right targets and a solid runbook, closes the most critical gaps in privileged access: the questions “who did it?” and “on which system?” become clear. When you add complementary layers like sudo I/O logging for the full command and its output, post-incident investigation times drop dramatically. The real win is turning security from a checklist into operational muscle memory.

Paylaş:

Bu yazı faydalı oldu mu?

Yükleniyor...

Bu yazı nasıldı?

ME

Mustafa Erbay

Sistem Mimarisi · Network Uzmanı · Altyapı, Güvenlik ve Yazılım

2006'dan bu yana sistem mimarisi, network, sunucu altyapıları, büyük yapıların kurulumu, yazılım ve sistem güvenliği ekseninde çalışıyorum. Bu blogda sahada karşılığı olan teknik deneyimlerimi paylaşıyorum.

Kişisel Notlar

Bu notlar sadece sizde saklanır. Tarayıcınızda yerel olarak tutulur.

Hazır 0 karakter

Comments

Server-side AI Moderation

Comments are AI-moderated server-side and stored permanently.

?
0/2000

Server-side AI moderation

✉️ Free · No spam · Unsubscribe anytime

Curated digest, hand-picked by me — not the AI

Once a week: the most important post of the week, behind-the-scenes notes, and a "what I actually used this week" section. Less noise, more signal.

  • 📌
    Best of the week Single most-worth-reading post
  • 🔧
    Toolbox notes Real tools I used this week
  • 🧠
    Behind-the-scenes Notes that don't make it to blog

We don't spam. Unsubscribe anytime. · Tracked only by Umami (self-hosted, no Google).

Your Reading Stats

0

Posts Read

0m

Reading Time

0

Day Streak

-

Favorite Category

Related Posts