A calendar library returned the same answer for every year — and my tests agreed with it

I build a Korean saju (BaZi) service. The whole pitch is that the numbers are computed deterministically and only the prose is written by a model, so the calculation layer is the one part that is not allowed to be vaguely right.

Two days ago I found out it had been wrong for five years of birthdays, and that my test suite had been cheerfully confirming it the entire time.

What the function was supposed to do

A saju chart's year and month pillars do not change on January 1st. They change at solar terms — the 24 points where the sun reaches a fixed ecliptic longitude. The year pillar turns at 입춘 (315°), which lands somewhere around February 4th, at a specific minute that differs every year, because the Earth's orbit does not care about our calendar.

So if you are born on February 4th, whether your chart says 癸卯 or 甲辰 depends on what time of day you were born, compared against an astronomical instant.

I was getting those instants from a calendar library:

const terms = getSolarTermsByYear(year)

The symptom

An agent working on unrelated copy flagged a contradiction: a FAQ page claimed minute-level solar term precision, while a comment in the engine said the data only covered 2020–2030. I went to check which one was lying.

I dumped 입춘 for eleven consecutive years and diffed it against an independent astronomical computation:

        library          computed (KST)
2024    Feb 4, 05:02     Feb 4, 17:26
2025    Feb 4, 05:02     Feb 3, 23:10
2026    Feb 4, 05:02     Feb 4, 05:01
2027    Feb 4, 05:02     Feb 4, 10:46

Eleven years, one value. The function took a year argument and ignored it. It happened to be right for 2026 — presumably whenever the table was generated — and wrong for everything else, sometimes by twelve hours, sometimes landing on the wrong day.

Twelve hours is not a rounding error here. It is the difference between two different charts:

2024-02-04 06:00  →  癸卯 乙丑   (before 17:26)
2024-02-04 18:00  →  甲辰 丙寅   (after)

Same date. Opposite answer. Anyone born on a solar term day between 2020 and 2030 — except 2026 — could have been handed the wrong one.

Now the part that actually bothered me

I have a golden test suite. Solar term boundaries are the first thing it locks. It was green the whole time.

Here is the shape of what it asserted:

// probes chosen around the "known" boundary of 05:02
assert.equal(chartFor('2024-02-04', '04:00').year, '癸卯')
assert.equal(chartFor('2024-02-04', '05:03').year, '甲辰')

Both of those passed. Of course they did. The engine read the boundary from the library, and the fixture values were written by reading the boundary from the library. My test and my code shared a single source of truth, so the test could only ever confirm the library's opinion — never reality.

That is the bug class, and it has nothing to do with calendars. A test is only an oracle if it knows something the code does not. The moment your expected values are derived from the same dependency the code under test calls, the assertion degrades into a change detector: it will tell you when behavior changes, and it will never tell you the behavior was wrong to begin with.

The second failure was subtler. Every probe I had chosen — 04:00, 05:01, 05:03 — sat on the same side of the real boundary at 17:26. So even a test written against real-world truth would have passed with the wrong table, because my examples all fell into one bucket. Example-based tests cannot see a parameter being ignored unless the examples straddle something.

The test that would have caught it

One property, no domain knowledge required:

test('different years produce different solar term instants', () => {
  const seen = new Set<string>()
  for (let y = 2020; y <= 2030; y++) {
    const t = solarTermsOfYear(y).find((x) => x.name === '입춘')!
    seen.add(`${t.month}-${t.day}${t.hour}:${t.minute}`)
  }
  assert.equal(seen.size, 11)   // the old table produced 1
})

It asserts a relationship rather than a value: this function's output must vary with its argument. It would have failed on day one, in a single line, without me knowing a single thing about astronomy.

I now think of this as the cheapest test you can write against any parameterized lookup — tables, caches, config resolvers, i18n bundles, feature flags, per-tenant settings. Anything with the shape f(key) => data deserves one assertion that says f actually reads key. It is the class of bug that produces confident, plausible, uniformly wrong output, which is the worst kind to ship.

The fix

I stopped looking the values up and started computing them. astronomy-engine was already a dependency for something else, and solar terms are just a root-find on solar longitude:

// 입춘 = the moment the sun reaches 315° ecliptic longitude.
// Search from 15 days before the nominal date, over a 30-day window:
// wider than the ~15-day gap between terms, narrower than the next crossing.
const start = MakeTime(new Date(Date.UTC(year, 1, 4 - 15)))
const t = SearchSunLongitude(315, start, 30)

Twenty-four terms per year, 1900–2050, cached per year, returned as both wall-clock and absolute instant. The library stayed for what it is good at — sexagenary cycle and lunar/solar date conversion — and lost the job it was silently failing.

Then the golden test got re-anchored to the real boundary, with probes at 17:25 and 17:28 instead of on one side of a fiction, plus the variance property above.

Three things I would do differently, and will next time:

1. Cross-check a dependency against an independent implementation before trusting it as an oracle. Not continuously — once, at adoption, on a spread of inputs. It took twenty minutes and would have saved five years of charts.
2. Choose probes that straddle the boundary you are testing, computed from the truth, not from the code. If your fixtures come out of the system under test, you have written a snapshot, not a test.
3. Assert variance on anything parameterized. One line. Do it before the interesting assertions.

Housekeeping, since someone will ask

Charts already generated are stored as payloads and are not silently recomputed — a saved reading stays what it was when it was issued, and we regenerate on request if a birth date falls in the affected window. Rewriting people's charts underneath them without telling them seemed worse than the bug.

The calendar engine is open source as k-saju (MIT, TypeScript). The astronomical version is published — v0.1.3 and up; 0.1.0 has the stale table, so upgrade if you installed it early. The corrected engine is live in the product at ioreum.com/en.

I am deliberately not naming the library. I have not filed an issue yet, and honestly the interesting part is not that a table went stale. It is that I built a test suite around it that was structurally incapable of noticing.