devLap

Building ArtifactX: product-ready apt/yum repos in Rust

rust dev.to

ArtifactX is a Rust CLI for signed apt/yum repositories: import existing repos, regenerate metadata under your own key, publish atomically, serve static files, and roll back when a cutover goes wrong.

That sounds like a narrow problem. It is. But Linux package repositories are one of those narrow pieces of infrastructure where small mistakes travel far.

There is a funny split in Linux packaging. The client side is one of the most stable interfaces in infrastructure: apt install, dnf install, yum install. People understand those commands, automation knows how to call them, and operating systems have spent decades making them boring.

The repository side is often the opposite.

A small internal package repo can start as a directory and a web server. Then it grows a signing key, a CI upload job, a few createrepo flags, an rsync target, an emergency rollback script, a second format for Debian machines, and one old client that only accepts a metadata compression format everyone else forgot about.

At some point the repo is no longer a repo. It is folklore.

That is the problem I built ArtifactX for.

What ArtifactX does for signed apt/yum repositories

ArtifactX (arx) is a single Rust binary for building, importing, signing, publishing, serving, and rolling back apt/yum package repositories.

The current wedge is:

Import first. Cut over when ready.

That means ArtifactX is not asking you to rewrite your whole release process on day one. The common path looks like this:

existing apt/yum repo or package drop directory
  -> arx import / arx add / arx publish-dir
  -> regenerate signed metadata under your key
  -> validate staged output with real apt/yum clients
  -> cut over static URLs only when it is boring
  -> roll back by flipping retained published state if needed
Enter fullscreen mode Exit fullscreen mode

The latest release is v0.2.7. It ships static binaries and self-packaged .deb, .rpm, and .apk artifacts.

The repository/publish side is now product-ready: the common import, publish, static hosting, rollback, and CI paths are documented and tested instead of being local folklore. The HTTP API is available and documented, but still intentionally described as beta until compatibility rules are stricter.

Why package repositories still get messy

Historically, a Linux package repository was just metadata plus payload files. That simplicity is a strength: a static HTTP server can distribute software to a fleet without running a custom agent everywhere.

But the details matter:

  • apt has Release, InRelease, Release.gpg, Packages, by-hash paths, and freshness semantics;
  • yum/dnf has repomd.xml, repository metadata signatures, primary/filelists/other metadata, and old-client compatibility traps;
  • repository metadata signing is not the same as package payload signing;
  • cutovers can expose half-written metadata if they are just copy operations;
  • rollback is not safe unless old metadata and old payload files are retained together;
  • CI uploads need authentication, but long-lived tokens are unpleasant;
  • import and migration need to preserve enough identity that clients do not panic at the first apt update.

There are good heavyweight tools for large artifact-management organizations. But a lot of teams do not need a full artifact platform. They need a signed, inspectable, static package repo that can be rebuilt, copied, audited, and rolled back.

That shaped the design.

Requirements from a real Linux fleet

Most of my day-to-day work is systems tooling: Rust utilities, SSH-heavy Linux operations, Kubernetes and monitoring work, and CI/release automation.

The environment that motivated ArtifactX is not hyperscale, but it is messy enough to be useful. Different systems see different slices of it: a monitoring system sees roughly 1.5k hosts and more than 200k enabled monitoring items; Ansible inventory has roughly 18k entries; the VictoriaMetrics side was ingesting about 0.55M rows/sec across roughly 4.2k scrape targets; and an edge HTTP path was showing about 300 requests/sec in a short log sample when checked.

Those are different views of the environment, not a single clean fleet count. That is exactly the point: package repository mistakes cross boundaries between CI, web serving, metrics, old OS clients, and human runbooks.

That kind of fleet teaches you to care about boring things:

  1. Static files should be enough. A package repo should be able to live behind nginx, a CDN, GitHub Pages, or any ordinary static host.
  2. Cutover should be atomic. Clients should not see half-written metadata.
  3. Rollback must include payload safety. Rolling metadata back to a state whose package files were garbage-collected is not rollback.
  4. Old clients are real. CentOS 7-era yum clients still need gzip metadata. Compatibility should be tested, not wished into existence.
  5. Signing material must not leak into public roots. Public files and private repository state are separate things.
  6. No-op publishes should be cheap. If no packages changed, the tool should not re-read and re-parse the entire repository.
  7. Migration should be incremental. Import a slice, validate it, then cut over.

Those requirements are not glamorous, but they are what turn a weekend script into something I am willing to put in a production-shaped path.

Why Rust and why a file-based model

Rust was a natural fit because ArtifactX wants to be a static operations tool, not a service dependency.

The design bias is:

  • one binary;
  • no database;
  • no background daemon;
  • repository state as ordinary files;
  • deterministic publish steps;
  • clear escape hatches such as arx publish --full if a cache ever looks suspicious.

For Debian-style packaging, ArtifactX builds .deb artifacts with Rust crates for the mechanical pieces (ar, tar, compression, checksums). For RPM repository metadata it uses the existing Rust ecosystem where it makes sense rather than pretending every format should be reimplemented from scratch. The project is split into crates so the reusable packaging/repository pieces can keep permissive licensing where possible, while the CLI binary can integrate GPL dependencies required by the current RPM repository path.

The important choice is not “Rust because Rust.” It is Rust because the result can be shipped as a small, static, inspectable tool that behaves the same way in CI, on a server, and in a local migration test.

What product-ready meant here

I try to be careful with the phrase “product-ready.” It does not mean every imagined feature exists. It means the core promise has been hardened enough that users can trust the common path.

For ArtifactX v0.2, that meant the repository side needed to cover:

  • import from existing apt/yum repos;
  • signed metadata generation;
  • atomic publish;
  • rollback/history;
  • package search and GC with rollback-state retention;
  • publish-dir for repeated package drop directories;
  • staged export and cutover preflight;
  • static hosting and GitHub Pages dogfood;
  • arx serve with a write API;
  • token auth and GitHub Actions OIDC push;
  • OpenAPI/Swagger documentation;
  • real apt/yum compatibility tests, including old gzip metadata expectations.

It also meant writing down the boundaries, because product-ready is partly about saying what the product does not own. ArtifactX signs repository metadata. It does not pretend to solve every organization’s package payload-signing or HSM/KMS story. It can trigger external sync, but it does not claim that your CDN, rsync fleet, or downstream mirror succeeded just because arx publish succeeded.

That separation is part of the product.

Dogfood results: faster small publishes and fewer hidden assumptions

Dogfooding changed the project more than planning did.

The first useful version could generate and serve repositories. Real use exposed the less exciting problems:

  • repeated package drops needed a no-op path;
  • old yum clients needed gzip metadata;
  • API workflows needed clear error shapes and examples;
  • release tags had to avoid racing “latest” aliases and Pages deployments;
  • docs had to explain what private state must never be copied into a public static root;
  • rollback and GC had to be described together, because they are coupled in practice.

One concrete performance result: on an isolated copy of a real repository root, a small-add publish path improved from about 18 seconds to about 1 second after a one-time metadata-cache backfill. That is not a universal benchmark, but it proves the direction: publish cost should track the change set, not the total historical repository size.

ArtifactX now also publishes itself. The release flow builds static binaries, creates packages, and publishes a signed dogfood repository on GitHub Pages:

That loop matters. If the tool cannot publish itself, it is too easy to miss the sharp edges.

What ArtifactX intentionally does not build

ArtifactX is not trying to replace Pulp, Nexus, Artifactory, or a full internal developer portal.

It is also not trying to be a distributed CDN, a policy workflow engine, or a universal software supply-chain platform.

The narrow bet is this:

For many teams, a signed package repository should be an inspectable directory plus a boring publish/rollback tool.

If you need global multi-tenant governance, ArtifactX is probably too small. If you have shell scripts that maintain apt/yum repos and you are afraid to touch them, it might be the right size.

What is next for ArtifactX pack

The next major lane is pack ergonomics.

Repository publishing is now boring enough that the packaging side deserves the same treatment. The v0.3 work is focused on:

  • Cargo workspace and crate selection;
  • better bridges from existing Cargo metadata such as cargo-deb and cargo-rpm conventions;
  • config-file marking;
  • reproducibility knobs such as explicit source dates;
  • directory payload entries;
  • clearer packaging docs;
  • eventually, curated packaging recipes for common upstream projects that do not ship reliable .deb / .rpm artifacts.

That last part is intentionally later. A curated repo is only useful if the packaging path is reproducible, boring, and easy to review.

Feedback I would like

If you work with Linux package repositories, I would especially like feedback on:

  • migration and cutover workflows;
  • old apt/yum/dnf compatibility traps;
  • how much arx pack should read from existing Cargo metadata versus ArtifactX-specific config;
  • what would make you trust a small static repo tool instead of another service.

The project is here: https://github.com/artifactx-rs/artifactx

And the docs / dogfood repo are here: https://artifactx-rs.github.io/artifactx/

Source: dev.to

arrow_back Back to Tutorials