HON’s Wiki # Debian Server
Home / Linux Servers
Contents
Using Debian 12 (Bookworm).
Basic Setup
Installation
- Always verify the downloaded installation image after downloading it.
- If installing in a Proxmox VE VM, see Proxmox VE: VMs: Initial Setup.
- Prefer UEFI if possible.
- Use the non-graphical installer. It’s basically exactly the same as the graphical one.
- If it mentions missing non-free firmware, take note of the packages so they can be installed later.
- Localization:
- For automation-managed systems: It doesn’t matter.
- Language: United States English.
- Location: Your location.
- Locale: United States UTF-8 (
en_US.UTF-8
).
- Keymap: Your keyboard’s keymap.
- Network settings:
- Whatever is more practical, typically static addresses for static servers and DHCP/autoconfiguration for everything else.
- Certain NICs require non-free drivers (which are not included in the public ISOs) (e.g.
bnx2
). The installer will typically show a warning about the missing firmware, but sometimes it won’t and the NIC will be listed but won’t be able to access the network properly (e.g. DHCP failing). To get around this you may load the driver from removable media, or connect a temporary NIC for Internet access to finish the install and to download/install the non-free driver for the other NIC(s).
- Use an FQDN as the hostname.
- For automation-managed systems: It doesn’t matter, just leave it as
debian
or something.
- It’ll automatically split it into the shortname and the FQDN.
- If using automation to manage the system, this doen’t matter.
- Use separate password for root and your personal admin user.
- For automation-managed systems: The passwords may be something temporary and the non-root user may be called e.g.
ansible
(for the initial automation).
- System disk partitioning:
- Simple system: Guided, single partition, use all available space.
- Advanced system: Manually partition, see system storage.
- Swap can be set up later as a file or LVM volume. It isn’t really that useful anymore.
- When using LVM: Create the partition for the volume group, configure LVM (separate menu), configure the LVM volumes (filesystem and mount).
- Package manager:
- Just pick whatever it suggests.
- Software selection:
- Just “SSH server”, so e.g. Ansible can reach it.
- GRUB bootloader (no longer asked):
- Install to the suggested root disk (e.g.
/dev/sda
).
Prepare for Ansible Configuration (if Ansible)
Do this if you’re going to use Ansible to manage the system.
This is mainly to make the system accessible by Ansible, which can then take over the configuration.
If creating a template VM, run the first instructions before saving the template and then run the last instructions on cloned VMs.
- Upgrade all packages:
apt update && apt full-upgrade
- If anything significant was updated, restart the server.
- Install the required packages:
apt install openssh-server sudo python3 vim
- If PVE/QEMU VM, install
qemu-guest-agent
.
- Setup sudo for Ansible:
usermod -aG sudo ansible
- (Optional, for PVE VMs) Convert the VM into a template:
- Shut down the VM.
- Change to a template.
- Clone it into a new VM to be used hereafter.
- Boot the new VM and continue with the setup.
- (Optional, for non-cloud) Set static IP addresses in
/etc/network/interfaces
(see the example below).
- Reboot.
Example /etc/network/interfaces
:
source /etc/network/interfaces.d/*
auto lo
ïface lo inet loopback
allow-hotplug ens18
iface ens18 inet static
address 10.0.0.100/22
gateway 10.0.0.1
iface ens18 inet6 static
address fdaa:aaaa:aaaa:0::100/64
gateway fdaa:aaaa:aaaa:0::1
accept_ra 0
Manual Configuration (if not Ansible)
The first steps ((Skip)
) may be skipped if already configured during installation (i.e. not cloning a template VM).
- (Skip) Check the system status:
- Check for failed services:
systemctl --failed
- Check that AppArmor is operational:
apparmor_status
- (Skip) Update the root password:
passwd
- (Skip) Localization:
- Check current locale:
locale
should return en_US.UTF-8
.
- Update if wrong:
update-locale LANG=en_US.UTF-8
- Check the keymap:
- Try typing characters specific to your keyboard.
- Update if wrong:
dpkg-reconfigure keyboard-configuration
- (Skip) Set the hostname:
- Set the shortname:
hostnamectl set-hostname <shortname>
(no domain)
- Set both the shortname and FQDN in
/etc/hosts
using the following format, one line for IPv4 and one for IPv6: <ip-addr> <fqdn> <shortname>
- Check the hostname info:
- Shortname:
hostname
- FQDN:
hostname --fqdn
- IP addresses:
hostname --ip-address
- Packages:
- (Optional) Enable the
contrib
and non-free
repo areas: add-apt-repository <area>
- Or by setting
main contrib non-free
for every deb
/deb-src
in /etc/apt/sources.list
.
- Update, upgrade and auto-remove.
- Install:
sudo apt install ca-certificates software-properties-common man-db tree vim screen curl net-tools dnsutils moreutils htop iotop irqtop nmap
- (Optional) Install per-user tmpdirs:
libpam-tmpdir
- (Optional) Configure editor (Vim):
- Update the default editor:
sudo update-alternatives --config editor
- Disable mouse globally: In
/etc/vim/vimrc.local
, add set mouse=
and set ttymouse=
.
- Fix YAML formatting globally: In
/etc/vim/vimrc.local
, add autocmd FileType yaml setlocal ts=2 sts=2 sw=2 expandtab
.
- Add mount options:
- (Not recommended) Setup hidepid:
- (Note) Hidepid breaks certain systemd things. It’s not recommended to use it until that gets fixed.
- (Note) The
adm
group will be granted access.
- Add your personal user to the PID monitor group:
usermod -aG adm <user>
- Enable hidepid in
/etc/fstab
: proc /proc proc defaults,hidepid=2,gid=<adm-gid> 0 0
(using the numerical GID of adm
)
- (Optional) Disable the tiny swap partition added by the guided installer by commenting it in the fstab.
- (Optional) Setup extra mount options: See Storage.
- Run
mount -a
to validate fstab.
- (Optional) Restart the system.
- Setup your personal user:
- If it doesn’t exist, create it:
adduser <username>
- Add the relevant groups (using
usermod -aG <group> <user>
):
sudo
for sudo access.
systemd-journal
for system log access.
adm
for hidepid, to see all processes (if using hidepid).
- Add your personal SSH pubkey to
~/.ssh/authorized_keys
and fix the owner and permissions (700 for dir, 600 for file).
- Hint: Get
https://github.com/<user>.keys
and filter the results.
- Try logging in remotely and gain root access through sudo.
- Setup SSHD:
- In
/etc/ssh/sshd_config
, set:
PermitRootLogin no
PasswordAuthentication no
# Optional, disable TCP port forwarding
AllowTcpForwarding no
GatewayPorts no
# Comment out to avoid locale issues (or fix it some proper way)
#AcceptEnv ...
- Restart
sshd
.
- Update MOTD:
- Clear
/etc/motd
, /etc/issue
and /etc/issue.net
.
- (Optional) Add a MOTD script (see below).
- (10/Buster and older) (Optional) Enable persistent logging:
- (Note) Persistent logging is the default for Debian 11/Bullseye, but not Debian 10/Buster.
- In
/etc/systemd/journald.conf
, under [Journal]
, set Storage=persistent
.
- (Note)
auto
(the default) is like persistent
, but does not automatically create the log directory.
- (Note) The default journal directory is
/var/log/journal
.
Machine-Specific Configuration
Physical Host
- Install extra firmware:
- Enable the
non-free
repo areas.
- Update microcode: Install
intel-microcode
(for Intel) or amd64-microcode
(for AMD) and reboot (now or later).
- (Note) APT package examples:
firmware-atheros -bnx2 -bnx2x -ralink -realtek
- If it asked to install non-free firmware in the initial installation installation, try to install it now.
- Install firmware from other sources (e.g. for some Intel NICs).
- (Optional) To install all common common firmware and microcode, install
firmware-linux
(or firmware-linux-free
) (includes e.g. microcode packages).
- Setup smartmontools to monitor S.M.A.R.T. disks:
- Install
smartmontools
.
- (Optional) Monitor disk:
sudo smartctl -s on <dev>
.
- Setup lm_sensors to monitor sensors:
- Install
lm-sensors
.
- Run
sensors
to make sure it runs without errors and shows some (default-ish) sensors.
- For further configuration (more sensors) and more info, see Linux Server Applications: lm_sensors.
- Check the performance governor and other frequency settings:
- Install
linux-cpupower
.
- Show:
cpupower frequency-info
- Check the boost state should be on (Intel).
- Check the current performance governor (e.g. “powersave”, “ondemand” or “performance”).
- (Optional) Temporarily change performance governor:
cpupower frequency-set -g <governor>
- (Optional) Permanently change performance governor: TODO
- (Not recommended) Mask
ctrl-alt-del.target
to disable CTRL+ALT+DEL reboot at the login screen.
QEMU Virtual Host
- Install QEMU guest agent:
apt install qemu-guest-agent
Networking
Network Manager
Using ifupdown (Alternative 1)
This is used by default and is the simplest to use for simple setups.
- For VLAN support, install
vlan
.
- For bonding/LACP support, install
ifenslave
.
- Configure
/etc/network/interfaces
.
- Reload the config (per interface):
systemctl restart ifup@<if>.service
- Don’t restart
networking.service
or call ifup
/ifdown
directly, this is deprecated and may cause problems.
Using systemd-networkd (Alternative 2)
This is the systemd way of doing it and is recommended for more advanced setups as ifupdown is riddled with legacy/compatibility crap.
- Add a simple network config: Create
/etc/systemd/network/lan.network
based on main.network.
- Disable/remove the ifupdown config:
mv /etc/network/interfaces /etc/network/interfaces.old
- Enable the service:
systemctl enable --now systemd-networkd
- Purge
ifupdown
and ifupdown2
.
- Check status:
networkctl [status [-a]]
- Restart the system to make sure all ifupdown stuff is stopped (like orphaned dhclients).
Prevent enabled (and potentially untrusted) interfaces from accepting router advertisements and autoconfiguring themselves, unless autoconfiguration is what you intended.
- Using ifupdown: Set
accept_ra 0
for all inet6
interface sections.
- Using systemd-networkd: See example config above.
- Using firewall: If the network manager can’t be set to ignore RAs, just block them. Alternatively, block all ICMPv6 in/out if IPv6 shouldn’t be used on this interface at all. This is not needed with ifupdown or systemd-networkd.
Firewall
- Install
apt install iptables iptables-persistent netfilter-persistent
- Don’t save the current rules when it asks.
- Manually add IPTables rules or make a simple iptables script or something.
- Open a new SSH session and make sure you can still log in without closing the current one.
- (Note) If you flush the firewall and reconfigure it, remember to restart services modifying it (like libvirt, Docker, Fail2Ban).
DNS
TODO Setup resolvconf
to prevent automatic resolv.conf
changes?
Using systemd-resolved (Alternative 1)
- (Optional) Make sure no other local DNS servers (like dnsmasq) is running.
- Configure
/etc/systemd/resolved.conf
DNS
: A space-separated list of DNS servers.
- (Optional)
Domains
: A space-separated list of search domains.
- (Optional)
DNSSEC
: Set to no
to disable (only if you have a good reason to, like avoiding the chicken-and-egg problem with DNSSEC and NTP).
- (Optional) If you’re hosting a DNS server on this machine, set
DNSStubListener=no
to avoid binding to port 53.
- Enable the service:
systemctl enable --now systemd-resolved.service
- Link
/etc/resolv.conf
: ln -sf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf
- Check status:
resolvectl
Using resolv.conf (Alternative 2)
The simplest alternative, without any local system caching.
- Make sure
/etc/resolv.conf
is a regular file and not a symlink.
- Manually configure
/etc/resolv.conf
.
- (Optional) Make it immutable to prevent services (like dhclient) from changing it:
chattr +i /etc/resolv.conf
NTP
This is typically correct by default. Note that systemd-timesyncd uses SNTP, where only one NTP server is used.
- Check the timezome and network time status:
timedatectl
- (Optional) Fix the timezone:
timedatectl set-timezone Europe/Oslo
- (Optional) Fix enable network time:
timedatectl set-ntp true
- Configure
/etc/systemd/timesyncd.conf
:
NTP
(optional): A space-separated list of NTP servers. The defaults are fine. Only one is used.
- Restart
systemd-timesyncd
.
- Check status works:
timedatectl
and timedatectl timesync-status
(check which servers are used)
Miscellanea
- Make sure IPv6 and NDP is configured securely (prevent accidental autoconfiguration on untrusted interfaces):
- For ifupdown, set
accept_ra 0
for all inet6
interface sections which should not use SLAAC.
- If configuring the interface to not accept RAs, ICMPv6/NDP may be firewalled on the untrusted interfaces.
- Reboot and make sure everything still works.
Everything here is optional.
- Setup BASH auto-completion:
- This is typically installed by default.
- Install it:
sudo apt install bash-completion
- Enable it globally: Find the commented
bash-completion
block in /etc/bash.bashrc
and uncomment it.
- Setup Fail2Ban:
- Recommended for public-facing servers.
- Fix the firewall first so it configures itself correctly wrt. which firewall is used.
- Install:
apt install fail2ban
- Check status:
fail2ban-client status [sshd]
- See Linux Server Applications: Fail2Ban for more info.
- Set up a swap file:
- (Note) You should have enough memory installed to never need swapping, but it’s a nice backup to prevent the system from potentially crashing if anything bugs out and eats up too much memory.
- Show if swap is already enabled:
swapon --show
- Allocate the swap file:
fallocate -l <size> /swapfile
- Fix the permissions:
chmod 600 /swapfile
- Setup the swap file:
mkswap /swapfile
- Activate the swap file:
swapon /swapfile
- Add it to fstab using this line:
/swapfile swap swap defaults 0 0
- Setup Postfix mail relay: See Linux Server Applications: Postfix.
- Prevent root local login:
- Alternatively, keep it enabled with a strong password as a local backdoor for recovery or similar.
- Add a personal user first.
- Check that the password field (the second field) for root in
/etc/shadow
is something invalid like “*” or “!”, but not empty and not valid password hash. This prevents password login.
- Clear
/etc/securetty
to prevent root local/console login.
- Extra package security:
- Install
apt-listbugs
and apt-listchanges
and run them before upgrading a package.
- Install
needrestart
and run it after upgrading.
- Install
debsums
and run it after upgrading to check deb checksums.
- Install
debsecan
to get automatically alerted when new vulnerabilities are discovered and security updates are available.
- Google Authenticator 2FA:
- Potentially useful for public-facing servers.
- TODO See old notes.
- Install and run Lynis security auditor:
- Install:
apt install lynis
- Run:
lynis audit system
- MOTD:
- Clear
/etc/motd
, /etc/issue
and /etc/issue.net
.
- Download dmotd.sh to
/etc/profile.d/
.
- Install the dependencies:
neofetch lolcat
- Add an ASCII art (or Unicode art) logo to
/etc/logo
, using e.g. figlet <text>
or TAAG.
- (Optional) Add a MOTD to
/etc/motd
.
- Test it:
su - <some-normal-user>
- Setup monitoring:
- Use Prometheus with node exporter or something and set up alerts.
Troubleshooting
network-online.target
is stalling during boot:
- See all services it depends on:
systemctl show -p WantedBy network-online.target
- Disable the unused services which stall.
Firmware for the network card fails to load:
- Causes a syslog record like “firmware: failed to load rtl_nic/rtl8168g-3.fw (-2)” when trying to up the interface.
- Might happen after installation even if working initially (for some reason).
- Realtek solution: Enable the “non-free” repo and install “firmware-realtek”.
Perl complains about a locale error:
- Test with
perl -e exit
. It will complain if there’s an error.
- Check the locale:
locale
- Comment
AcceptEnv LANG LC_*
in /etc/ssh/sshd_config
to prevent remote clients bringing their own locale.
Boot volume is full:
- If this failed during a software upgrade, take note of the error.
- Most of the time
apt auto-remove
should be enough.
- Manually remove old kernels (if there’s still not enough space):
- List installed kernels with
dpkg -l | tail -n +6 | egrep 'linux-image-[0-9]+' | grep -Fv $(uname -r)
rc
means already removed, iU
means it’s queued for install, ii
means eligible for removal.
- Remove all kernels marked
ii
by apt-uninstalling linux-image-X-generic linux-image-X linux-image-X-common
.
- Run another
apt auto-remove
just in case (pointless?).
- Afterwards:
- If it ran out of space during an APT software upgrade, run
apt install -f
to fix any packages which failed and maybe a apt upgrade
in case there’s more upgrades.
- Make sure the initramfs isn’t corrupt (if it ran out of space during an upgrade) by running
update-initramfs -u -k all
.
Miscellanea
Cron
- Don’t use periods (including file extensions) in the hourly/daily/weekly/monthly scripts.
hon.one
| HON95/wiki
| Edit page