My package declares support for Node 18 and up. Its own test suite could not run on two of the three versions I claimed to support. Everything passed on my machine.
The script
{"scripts":{"test":"node --test \"test/**/*.test.mjs\""},"engines":{"node":">=18"}}That looks fine. It is quoted, so the shell does not expand it, which is what you want when a tool does its own glob handling. On my Node 22 it ran 35 tests and went green.
What CI said
The workflow was nine jobs: Ubuntu, Windows and macOS across Node 18, 20 and 22.
Could not find '/home/runner/work/pkg/pkg/test/**/*.test.mjs'
>node --test "test/**/*.test.mjs"
Error: Process completed with exit code 1
macOS 22 green. Ubuntu 18 red. Windows 20 red. The pattern was not the operating system, it was the Node version.
Why
Glob expansion in the Node test runner arrived in Node 21. Before that, a path argument is treated as a literal path. So on 18 and 20 the runner looks for a directory literally named **, does not find it, and dies before a single test executes.
It never even reached the tests. The suite was not failing; it was not running.
The fix
{"scripts":{"test":"node --test"}}With no path argument, the runner discovers test files itself using its default patterns, which include **/*.test.mjs and anything under a test/ directory. That behaviour has been there since Node 18, and it does not depend on the shell, so it works the same on Windows.
Same 35 tests, now on all nine jobs.
The part worth keeping
This is the entire argument for a build matrix, and it is not thoroughness for its own sake.
"Works on my machine" is a true statement about exactly one row of that matrix. I had declared support for a range I had never once executed. The matrix did not find a rare edge case; it found that two thirds of my stated support was fiction, about ninety seconds after I pushed the workflow.
If your package declares an engines range, run your tests across that range. Otherwise the range is a guess you are asking your users to verify for you.