Building a hardened personal Linux OS with only proven tools, and why I ultimately abandoned cryptographic rootfs protection

rust dev.to

I build a defensive security app for Linux called RoamSwitch as an independent developer. Lately I've been going a step further and building a hardened Linux OS from scratch, RoamSwitch OS. This post pulls together two things I wrote about it separately in Japanese, why I'm building a secure OS of my own and the real reason I couldn't protect rootfs cryptographically, and digs a bit deeper into the technical side.

Why build an OS instead of just an app

RoamSwitch runs on top of whatever OS is already there. It handles firewall control, port monitoring, ransomware detection, and so on, but as a userland application it can only go so far. Misconfigurations baked into the OS itself, or weak defaults at the kernel level, are out of its reach no matter how well it's built.

So I started RoamSwitch OS out of curiosity about what changes if hardening gets built in from the OS layer down, instead of bolted on afterward. It's early days still, nowhere near ready for anyone but me to run. The base is Arch Linux, packaged as a bootable live/installer ISO via archiso. I'm not building a distro from scratch. The plan is to layer hardening on top of a normal Arch install, using tools that already have years of track record behind them: AppArmor, fapolicyd, auditd, Falco/Tetragon, Landlock, AIDE, TPM2.

One direction I ruled out early was an immutable, image-based OS along the lines of Fedora Silverblue or ChromeOS. Going that route means treating the whole OS as a single image and swapping it wholesale on every update, which is a completely different (and much larger) engineering and operations problem than what I signed up for. The governing principle stayed simple: don't reinvent tools that already work, build on Arch's rolling package ecosystem and existing driver support rather than around them.

The implementation is a Rust workspace with roughly 39 crates under hardening/, each one producing a binary for a specific defensive feature. That's not the same thing as "a pile of CLI tools you run by hand," though. Most of the watcher-style crates ship as systemd services (Type=simple, Restart=on-failure) that start at boot and just keep running. The ones that only need to run occasionally, like firmware checks, compliance scans, or patch status, ride on .timer units instead. There's no single daemon sitting on top coordinating everything, but the modules aren't isolated either: they talk to each other through shared state under /var/lib/roamswitch/, including a unified incident timeline I'll get into below. Phases 1 through 4 and Phase 6 are verified on real hardware and in QEMU. It's still a research and showcase project rather than something with a production track record, third-party pentest, or legal review behind it, so I'm not recommending it for production use yet.

What's already built

A few of the pieces worth naming.

forensic-sweep watches the usual persistence surfaces: cron, systemd user units, shell rc files, udev rules, PAM config. It flags new persistence, /etc/ld.so.preload hijacking, and fileless execution via memfd_create.

honeytokens plants fake credential files and decoy SSH/HTTP/SMB listeners that no legitimate process should ever touch. It goes further than just dropping bait files: it fingerprints connecting decoy clients with JA3/JA3S, runs adaptive decoys that change behavior over time, and even drops beacon-carrying .docx documents as canary tokens.

fapolicyd-mgr is the application-allowlisting layer, using pacman-managed file hashes as the trust anchor. This ends up mattering more than it sounds, since it's the practical substitute for what IMA appraisal would have covered, which I'll get to.

exfil-guard correlates bursts of access to sensitive files with new outbound network connections or large USB writes, tracking actual byte counts through netlink's INET_DIAG/tcp_info. It also ships a DLP classifier with a real Luhn checksum for card numbers, Japan's My Number check-digit algorithm, and Shannon-entropy-based secret detection.

mitm-guard detects ARP cache poisoning (implemented from scratch after finding that bettercap's own gateway-exclusion logic is structurally blind to ARP spoofing) and DHCP spoofing, using a sandboxed, passive-only bettercap child process.

privacy-guard wraps mac-randomize (a macchanger -r wrapper), dns-check (loopback vs. plaintext resolver), and a tor-mode with a fail-closed nftables kill switch.

wireless-guard covers Wi-Fi evil-twin detection (diffing against an airodump-ng baseline) and Bluetooth proximity scanning.

ransomware-rollback watches for rename/rewrite bursts, unfamiliar extensions, and entropy spikes via inotify, and takes an automatic btrfs/LVM snapshot when it sees them.

That cross-module reaction is worth spelling out, because it's a concrete answer to "how does this actually connect together." About 17 modules (exfil-guard, mitm-guard, honeytokens, forensic-sweep, and others) append events to a shared file, /var/lib/roamswitch/os_hardening_incident_timeline.json, through a common library called os-timeline. That file is the unified incident timeline. ransomware-rollback runs a dedicated background thread, ExternalTimelineWatcher, that polls this file and triggers its own snapshot routine the moment it sees a Critical-severity event written by any other module. To avoid triggering on its own writes, it tags its own entries and filters them out, and it shares a cooldown lock with its native detection path so the two don't double-fire. It's a plain shared file and polling, not a message bus, but it's tested and deliberate rather than an accidental side effect of similar-looking features.

All of this comes with matching AppArmor enforce profiles and verification runs across real hardware, Docker, and QEMU. There are also things I've deliberately left out: per-file cryptographic tamper checks, automatic anonymizing-network switching, automatic password-strength enforcement (it stays advisory). The cryptographic tamper check is the one I want to walk through in detail, because it wasn't a matter of running out of time. I looked into it and turned it down on purpose.

What ransomware is actually doing

Before getting into rootfs integrity, it's worth being precise about what we're actually defending against.

Most ransomware encrypts files with a hybrid scheme. A fast symmetric cipher, AES or ChaCha, encrypts the file contents themselves. That symmetric key then gets wrapped in the attacker's public key, usually RSA-2048 or RSA-4096. The only way to unwrap that key is with the attacker's private key, which they never expose. However hard the victim tries, recovery is impossible without that private key surfacing somewhere. This is also why decryption tools like Japan's National Police Agency's decryptor for the Phobos ransomware family only work against specific variants: they only exist because investigators managed to recover that particular attacker's private key.

A trickier development is intermittent encryption. Instead of encrypting an entire file, the malware touches just the first and last few kilobytes, or a handful of randomly chosen blocks. That's still enough to break the file's headers and structure so it's unusable, but it's a fraction of the work, runs faster, and doesn't push the file's overall entropy up nearly as much. It's specifically shaped to slip past the classic "entropy spiked, must be encryption" detection approach. Campaigns combining AES-CTR with RSA-4096 in this intermittent style have been seen hitting Windows, Linux, and ESXi targets in the same run.

There's also a step that happens before the encryption itself: wrecking recovery options. A lot of ransomware deletes shadow copies or snapshots before it starts encrypting. On Windows that's vssadmin.exe Delete Shadows /all /quiet; on ESXi it goes through esxcli to shut down running VMs and clear local snapshots first. Having backups isn't the whole story if the attacker gets to them before you do.

Two candidates for cryptographic verification

Given an opponent like that, being able to cryptographically confirm a file hasn't been tampered with sounds like exactly the tool you'd want. Linux has two mechanisms that get close: dm-verity and IMA (Integrity Measurement Architecture) appraisal.

dm-verity works at the block-device level. It checks data against a Merkle tree and returns an I/O error the moment it detects tampering on read. The catch is that it assumes the underlying block device never changes, which fits something like Android's /system partition, written once and never touched again, but doesn't work for anything that changes regularly.

IMA appraisal works per-file. It stores a signed hash in a security extended attribute (security.ima), and the kernel verifies it on access. Re-sign the file whenever it legitimately changes, and it can, in principle, handle a system that updates regularly.

IMA appraisal looked like the more promising option going in. Arch's pacman supports hooks that run at install/update time, and I figured those hooks could re-sign files automatically on every package update.

Why dm-verity was out

dm-verity didn't need an experiment to rule out. It was a structural mismatch from the start.

RoamSwitch OS is built on the premise of composing existing, proven security tools rather than building an immutable image-based OS. dm-verity needs the rootfs packaged as a single image, a new image and Merkle tree regenerated on every package update, and the bootloader repointed at the new image (typically via an A/B partition scheme). That's effectively rebuilding a Fedora Silverblue or ChromeOS-style immutable OS from scratch, which flatly contradicts the starting design. Arch pushes package updates multiple times a week, so redoing that image-build cycle constantly would also be an unreasonable cost on its own.

Why IMA appraisal was out too

IMA appraisal I actually tested.

I spun up a Docker container on Arch Linux and inspected the shipped kernel configs for both the linux package (the standard kernel) and linux-hardened (the security-hardened one) directly. The result was the same for both:

  • linux: # CONFIG_IMA is not set. Related options like CONFIG_IMA_APPRAISE don't even show up in the config.
  • linux-hardened: same thing, # CONFIG_IMA is not set. The kernel that markets itself as the hardened option doesn't have IMA turned on either.

The supporting infrastructure was there: CONFIG_INTEGRITY, CONFIG_INTEGRITY_SIGNATURE, CONFIG_SYSTEM_TRUSTED_KEYRING, CONFIG_DM_VERITY were all enabled. The groundwork for IMA appraisal exists. IMA itself was just switched off, in both kernels.

Once that was confirmed, the only path to enabling it is dropping Arch's official kernel package and building and maintaining a custom one indefinitely. That's a direct contradiction of the "compose existing proven tools, don't reinvent them" principle the whole project runs on, and it's really a different, much larger project: building and maintaining your own Linux distribution.

As of September 12, 2026, both results together led to a documented decision not to implement this under the project's current constraints, with an explicit note to revisit it if Arch's official kernel ever turns IMA on, or if the project decides to maintain its own kernel build and signing pipeline for some future fully OS-integrated edition. For the practical threat this would have addressed, unauthorized or tampered binaries executing, the already-implemented fapolicyd-mgr (application allowlisting rooted in pacman's own file hashes) covers a realistic chunk of that ground without touching the kernel at all.

Cryptographic guarantees, scoped narrowly

None of this means cryptographic integrity got abandoned everywhere. Scoped down, it's still in use.

AIDE (Advanced Intrusion Detection Environment) is a proper file-hash-based FIM tool, but it isn't pointed at the whole filesystem. It's deliberately scoped to /etc, /usr/bin, /usr/sbin, and /usr/lib/systemd. The reason is simply time: that scope finishes in 1 to 15 seconds in measurement, versus 2 minutes 33 seconds for a much broader one. It's positioned as detection, not prevention, and the documentation is upfront about the residual gap: a fully-rooted attacker can tamper with AIDE's own baseline database or binary and hide the evidence from the next check.

timeline-seal uses TPM2, a security chip physically separate from the CPU and disk, to chain-sign the incident timeline's JSON log with a Merkle-style hash chain. This one gives genuine cryptographic tamper-evidence, but only over that single log. Its own documented limits are candid too: an attacker can silently stop the timeline instead of editing it, can crowd out old entries via the 200-entry cap, and can re-mint a signing key and forge a new history under root unless the public key is pinned somewhere off the box.

Put side by side, the pattern is consistent: detect broadly with hashing (AIDE), and reserve genuine cryptographic tamper-evidence for something narrow (a single log, via TPM2). Nowhere does the design ask for cryptographic tamper-resistance across the whole rootfs.

What actually stops ransomware

For the kind of ransomware targeting a person's own files, what's actually doing the work isn't cryptographic integrity checking. It's behavior.

honeytokens catches the moment a decoy file that no legitimate process would ever touch gets hit. ransomware-rollback triggers a snapshot the instant a rename/rewrite burst, unfamiliar extensions, and an entropy spike show up together. exfil-guard catches the "steal first, encrypt second" double-extortion pattern before the encryption step even starts.

What makes these hold up against intermittent encryption specifically is that they don't depend on entropy at all. However low an attacker keeps a file's entropy, touching a large number of files in a short window is not something they can hide. A honeytoken doesn't care what encryption algorithm was used against it, only whether it was touched. A cryptographic hash check can only tell you a file changed, never why. Telling a legitimate save apart from an attack in progress needs behavioral information: what got touched, how fast, and in what pattern.

Tools that prevent, and tools that notice and roll back

RoamSwitch OS holds itself to a specific constraint: build on an existing distribution (Arch) and existing, proven security tools rather than maintaining a from-scratch kernel or distro. That constraint collided head-on with the idea of protecting the whole rootfs cryptographically. dm-verity assumes an immutable block device, which means building a different kind of OS entirely. IMA appraisal was disabled at the kernel level on both the standard and the hardened Arch kernel, and fixing that means maintaining an entire custom kernel indefinitely.

So cryptographic integrity checking stays reserved for things that are genuinely not supposed to change, package-managed binaries, a single append-only log, while everyday user data that changes constantly gets watched for suspicious behavior and rolled back automatically the moment something looks wrong. Tools that prevent and tools that notice-and-recover are answering different questions, because what they're protecting has fundamentally different properties. That's the part that took longer to land on than I expected, and I'm glad I worked through it rather than reaching for the strongest-sounding option by default.

Source: dev.to

arrow_back Back to Tutorials