How I stopped paying OpenAI to run my test suite
python
dev.to
I was building an AI project and ran into something that kept bothering me. Every test that touched my LLM code was making a real API call. To OpenAI. Every single time. Tests were slow — 3 to 5 seconds each just waiting for a response. Every CI run cost real money in tokens, for code that hadn't even shipped yet. And the tests were flaky: same code, same input, different output. Language models are non-deterministic, so I'd get a passing run, then a failing run, with no way to tell if my code