systemd Guide: Unit Files, systemctl, Timers & journalctl

Q: What's the difference between start and enable?

start runs the unit now. enable hooks it into a target (per [Install]) so it starts automatically at boot. enable --now does both. disable removes the boot hook, and mask prevents the unit from being started at all.

Q: How do I see why a service failed?

systemctl status name shows the exit code and recent logs, journalctl -u name -b shows the full logs, and systemctl show name -p Result,ExecMainStatus,NRestarts gives details. systemd-analyze verify checks unit syntax.

systemd

systemd is the init system and service manager on nearly every major Linux distribution: Ubuntu, Debian, Fedora, RHEL, Arch, and more. It's the first process the kernel starts (PID 1), and it boots the system, starts services in dependency order, restarts them when they crash, collects their logs, schedules timers, and manages resources with cgroups.

For anyone running applications directly on VMs or bare metal, systemd is how you turn a binary into a reliable service: it starts on boot, restarts on failure, logs to the journal, runs as an unprivileged user, and is sandboxed from the rest of the system. All of that comes from a short unit file, with no supervisor scripts or process managers needed.

TL;DR

systemd manages units: .service, .timer, .socket, .target, .mount, .path, and more.
systemctl controls them: start, stop, restart, reload, enable (start at boot), status, daemon-reload.
Service units define ExecStart, User, Restart=on-failure, environment, and dependencies (After=, Wants=, Requires=).
journald captures stdout and stderr; read logs with journalctl -u name.
Timers replace cron with better logging, dependencies, and catch-up of missed runs.
Customize packaged units with drop-in overrides (systemctl edit), and harden services with sandboxing directives.

Quick Example

A production-ready service unit for a web app:

Core Concepts

Units and Their Locations

Unit files have [Unit] (description, dependencies), a type-specific section ([Service], [Timer], …), and [Install] (what enable hooks into).

Service Types

Prefer exec or notify for modern apps that run in the foreground. Don't daemonize or background inside the app.

Restart Policies

Restart= options: no, on-failure (non-zero exit, signal, timeout, watchdog), on-abnormal, always. Tune with RestartSec=, and rate-limit restarts with StartLimitIntervalSec= and StartLimitBurst= in [Unit]. After too many failures, the unit enters a failed state instead of looping forever. WatchdogSec= restarts hung services that stop sending keepalives.

Dependencies and Ordering

Ordering: After= and Before= control sequence only.
Requirements: Wants= (weak: start it too, but don't fail if it fails), Requires= (strong), BindsTo= (stop when the dependency stops).
Targets group units into states: multi-user.target (normal server boot), graphical.target, network-online.target.

Ordering and requirement are independent. You usually need both: Wants=network-online.target plus After=network-online.target.

journald and journalctl

systemd captures services' stdout and stderr into the journal, with metadata (unit, PID, priority, boot):

Make logs persistent across reboots with Storage=persistent in journald.conf (or by creating /var/log/journal), and ship them centrally with a log agent. See log aggregation.

Timers

Timers schedule units, replacing cron:

The timer activates db-backup.service (a oneshot). Benefits over cron: logs in the journal, dependencies, resource limits, sandboxing, randomized delays, and systemctl list-timers to see next and last runs.

Drop-In Overrides

Don't edit vendor units. sudo systemctl edit nginx creates /etc/systemd/system/nginx.service.d/override.conf, where you can change only what you need (say, LimitNOFILE or Restart). Package upgrades then won't overwrite your changes. systemctl cat nginx shows the merged result.

Sandboxing and Resource Control

systemd uses namespaces, cgroups, and seccomp to restrict services: ProtectSystem, ProtectHome, PrivateTmp, PrivateDevices, NoNewPrivileges, CapabilityBoundingSet, SystemCallFilter, RestrictAddressFamilies, and DynamicUser. systemd-analyze security <unit> scores a unit's exposure and suggests hardening. Resource directives (MemoryMax, CPUQuota, IOWeight, TasksMax) use cgroups v2.

Best Practices

Run in the Foreground and Log to stdout

Let systemd handle daemonization, restarts, and logging. Apps should run in the foreground, write logs to stdout and stderr (ideally structured), and handle SIGTERM for graceful shutdown.

Use Dedicated Users and Sandboxing

Set User=/Group= (or DynamicUser=yes) and add sandboxing directives incrementally, checking systemd-analyze security and testing. See Linux permissions.

Keep Secrets Out of Unit Files

Unit files are world-readable. Use EnvironmentFile= with restricted permissions, or LoadCredential=/LoadCredentialEncrypted=, which pass secrets to the service without environment variables.

Always Run daemon-reload After Edits

systemd caches unit definitions. After editing unit files, systemctl daemon-reload makes changes take effect; systemctl status warns when a unit file changed on disk.

Common Mistakes

Backgrounding the Process With Type=simple

Run the server in the foreground (exec ./server in scripts), or use Type=forking with a PID file for legacy daemons.

After= Without Wants= (or Vice Versa)

After=postgresql.service alone doesn't start PostgreSQL; it only orders the startup if both are being started. Combine ordering with a requirement directive when you need the dependency started.

Restart=always Hiding Crash Loops

A service that crashes every few seconds and restarts quietly can go unnoticed. Rate-limit restarts, and alert on systemctl --failed or on restart counts (NRestarts in systemctl show).

FAQ

What's the difference between start and enable?

start runs the unit now. enable hooks it into a target (per [Install]) so it starts automatically at boot. enable --now does both. disable removes the boot hook, and mask prevents the unit from being started at all.

Should I use systemd timers or cron?

Timers are generally better on systemd systems: journal logging, dependencies, resource limits, sandboxing, catch-up of missed runs, and easy inspection with list-timers. Cron remains fine for simple, portable schedules, and it's what many containers and minimal systems have.

How do I see why a service failed?

systemctl status name shows the exit code and recent logs, journalctl -u name -b shows the full logs, and systemctl show name -p Result,ExecMainStatus,NRestarts gives details. systemd-analyze verify checks unit syntax.

Is systemd used in containers?

Usually not. Containers typically run a single process as PID 1 and rely on the orchestrator (Kubernetes, Docker) for restarts and logging. systemd matters on hosts and VMs, including the nodes that run container runtimes.