Building a Self-Hosted Workflow Engine on PHP and SQLite

php dev.to

Most workflow automation lives on somebody else's server. That is fine until you are moving customer records, API keys or anything with a compliance story attached, and then "where does this data actually go" becomes a question you cannot answer with a shrug.

The alternative is smaller than people expect. A workflow engine is really three things: a way to draw a sequence of steps, a way to move values between them, and a log of what happened. None of that requires a cluster. What follows is how that shakes out when the whole engine is one PHP app with a single SQLite file.

Why The Storage Choice Simplifies Everything

SQLite gets dismissed as a toy database, which misses what it does for a tool like this. A workflow engine writes a run record, a handful of step logs and a final variable snapshot. That is a low-write, read-mostly pattern with one process touching the file.

The knock-on effect is deployment. There is no database server to provision, no connection string to rotate, no separate backup job. Copying the file is the backup. The visual editor ships inside the project rather than loading from a CDN, so the whole thing runs from your own host with nothing phoning out. The getting started guide walks the path from install to a first flow running against a real trigger.

Six Node Types Cover Almost Everything

The temptation when building this kind of tool is to add a node for every integration. That is how you end up maintaining four hundred connectors.

A smaller set holds up better: an HTTP request node, an if condition, a switch, a set variable, a for each loop, and terminal success and fail nodes. HTTP plus branching plus iteration is a general purpose automation language, and any service with a REST API is already reachable without a bespoke connector.

Two conveniences do a lot of work here. Every field accepts placeholders, and every node writes its activity into the run's step log, so debugging is reading rather than guessing. The node reference covers each node's fields and outputs, including the loop protection that keeps a bad condition from spinning forever.

One Flat Variable Pool Beats Nested Context

This is the design decision that matters most, and the one most likely to be argued with.

Instead of each node owning scoped context, a run has a single flat pool of named values. The trigger fills it, any field reads from it with {name} placeholders, HTTP nodes capture response values back into it, and the run history shows the final state when the flow ends.

Flat means visible. You can look at one screen and know everything the run knew, which is not true of nested context objects where a value three levels down is technically available and practically undiscoverable. Loops fit the same model through loopItem and loopIndex rather than a separate scope. The capture syntax that pulls fields out of an arbitrary JSON response and back into that pool is covered in variables and response capture.

The tradeoff is real: no scoping means name collisions are your problem. In practice, for flows in the five to thirty node range, that has been a much smaller cost than the debugging time a nested model adds.

The Takeaway

Self-hosting an automation engine is not the infrastructure project it sounds like. One process, one file, six node types and a visible variable pool covers a surprising share of what teams reach for a SaaS workflow tool to do, and it keeps the data on hardware you control.

If you are evaluating this kind of thing, the question worth asking is not how many integrations it has. It is whether you can tell what a failed run was holding at the moment it broke.

Source: dev.to

arrow_back Back to Tutorials