I Built HaloLog, a Go Logger That Measured 23.9 ns/op. Then I Tried to Break the Claim

Part 1 of Engineering HaloLog: making high-performance telemetry claims
reproducible and falsifiable.

23.9 ns/op is a dangerous number.

It is precise enough to be repeated and incomplete enough to mislead. Without
the work unit, output contract, clock semantics, sink, concurrency model,
hardware, toolchain, and competing configurations, it is not yet an engineering
result. It is only a number with good typography.

I built HaloLog, a structured logger for Go, around a zero-allocation hot path.
On the original benchmark host, its bare-message path encoded and dispatched a
complete JSON record in 23.9 ns. That reciprocal is roughly 42 million records
per second on one goroutine.

The second sentence needs more care than the first. It is arithmetic derived
from a microbenchmark, not a sustained ingestion test. The benchmark writes to
io.Discard; it does not measure disk, network, batching, backpressure, or a
production log collector. And the 23.9 ns value itself is being re-frozen after
a fairness review changed how Zap should be represented.

That review is the reason for this article.

The interesting part of HaloLog is not that one benchmark produced a low
number. It is the attempt to make performance claims executable: allocation
guards fail CI, two encoding architectures are checked against the same byte
contract, optimized paths become ineligible when transforms need structured
data, and competitor configurations remain open to correction.

This is how that system works, where it is strong, and where its claims stop.

First define the operation

The comparative microbenchmark has a narrow job. Each logger must emit one
newline-terminated structured JSON record containing:

• a timestamp;
• a level;
• a message;
• the same scenario-specific field names, types, and values.

The measured operation ends when those bytes have been encoded and dispatched
to io.Discard. Every comparative row uses one goroutine. Setup that an
application would perform once, such as constructing a logger or binding
request context, happens before the timed loop. Dynamic call-site fields remain
inside it.

That definition makes the result useful for one question:

What is the CPU and allocation cost of constructing and dispatching this log
record through each public logging API, excluding destination I/O?

It does not answer:

How many records will my service persist through a particular file,
collector, network, or storage stack under concurrent load?

Those questions need different harnesses. Combining them in one number would
hide more than it reveals.

The comparison currently includes HaloLog, phuslu/log, zerolog, Zap, slog,
and logrus, with dependency versions pinned in the benchmark module. It uses
typed fields where the library provides them. Caller and stack capture are not
enabled for one logger while disabled for another. No comparative row receives
a level-gated or disabled-output shortcut.

Even after making those choices explicit, one assumption remained too weak:
the benchmark documented output equivalence, but did not execute an output
contract.

The maintainer feedback that changed the harness

I opened a narrow discussion with the Zap community. The question was not
whether Zap was “fast enough” or whether HaloLog had won. It was this:

For dynamic fields available only at the call site, is
Logger.Info(msg, zap.Int(...)) the idiomatic public hot path Zap should be
represented by? If not, is there another public pattern with the same
timestamp, level, message, and dynamic-field semantics but less per-call
work?

A Zap maintainer replied that the usage looked basic but idiomatic. The useful
part of the reply was elsewhere: Zap's own benchmark
configuration overrides the production time encoder with
EpochNanosTimeEncoder, which is cheaper than the default EpochTimeEncoder.
The maintainer also suggested asserting the actual output so differences such
as time encoding could not pass unnoticed.

That was a valid criticism.

The committed comparison used zap.NewProductionEncoderConfig(). Its default
timestamp is a floating-point count of Unix seconds. Zap's benchmark override
emits integer Unix nanoseconds. Both records still contain a timestamp, level,
message, and the same dynamic fields, but they do not have the same byte
representation.

I measured the encoder change instead of arguing that it would be too small to
matter. In an exploratory Windows A/B, the change was material in the one-field
and request-scoped cases, small in the ten-field case, and too noisy to call in
the twenty-field case. Allocation counts did not change.

I am deliberately not publishing those exploratory percentages as a corrected
cross-platform result. The final comparison must come from the upgraded,
tagged suite with the same sample count and evidence policy on Linux and
Windows. A local result can justify more investigation; it cannot silently
become a public leaderboard.

The correct response is not to overwrite the old Zap cell under the same name.
Changing the time encoder changes the wire representation. The corrected suite
will therefore report two named configurations:

• Zap production-default timestamp: fractional Unix seconds;
• Zap benchmark timestamp: integer Unix nanoseconds.

The reader can see both the idiomatic production default and the faster variant
used by Zap's own benchmark. Nothing is silently substituted after the fact.

Fairness became a test, not a paragraph

The first new test exercises the same constructors as the benchmark. For each
logger it captures a representative one-field operation and verifies:

1. exactly one newline-terminated record was emitted;
2. the record is valid JSON;
3. timestamp and level are present;
4. the expected message is present;
5. the field value is a JSON number, not an accidentally quoted string.

This is a semantic contract. It deliberately does not require every logger to
use the same key names for built-in metadata or the same timestamp format.
Requiring byte identity across different public logger formats would be a fake
form of fairness.

The Zap encoder comparison has a second, stronger test. It installs a fixed
clock and compares the complete bytes against two golden records. That makes
the timestamp difference visible:

{"level":"info","ts":1700000000.1234567,"msg":"request completed","status":200}{"level":"info","ts":1700000000123456789,"msg":"request completed","status":200}

Same semantic fields. Different wire representation. Both facts matter.

The test does not prove that every benchmark scenario across every library is
byte-equivalent. It proves a representative output contract and pins the exact
Zap difference that triggered the review. Extending that contract is now an
explicit benchmark task, not an assumption buried in methodology prose.

Where the HaloLog hot path saves work

HaloLog's measured path is not one clever instruction. Most of the gain comes
from deciding when work must happen and who owns the resulting bytes.

1. Time is cached, with an explicit accuracy trade-off

The default clock is refreshed by a background goroutine every 10 ms. A log
call reads the cached Unix-nanosecond value instead of acquiring wall-clock time
on every line.

For the default whole-second JSON format, HaloLog caches the entire prefix for
each (second, level) pair:

{"time":"2026-09-01T10:42:17Z","level":"INFO"

On a steady-state cache hit, rendering that prefix is an atomic pointer load
and one append/copy into the line buffer. When the second changes, the next call
rebuilds the immutable entry and atomically publishes it.

This optimization is not free semantics. A cached clock can lag wall time by
approximately its refresh interval, and scheduler pauses can extend that lag.
Whole-second timestamps intentionally discard sub-second detail. HaloLog offers
millisecond, microsecond, and nanosecond formatting, but those settings do not
make the global 10 ms clock fresher. The current public logger configuration
does not inject a different clock. A workload requiring tighter timestamp
freshness should not inherit the default-path claim; configurable clock
freshness is a limitation to resolve and benchmark separately.

2. Request context is encoded when it becomes context

Service logs often repeat the same fields for every line in a request:

reqLog := logger.With().
    WithString("service", "payments").
    WithString("region", "eu-west-1").
    WithString("request_id", requestID).
    WithInt("shard", 7).
    WithString("version", "1.4.2").
    Logger()

reqLog.Typed().WithInt("status", 200).Info("request completed")

The context builder keeps two representations. It stores structured fields for
paths that need transforms, and it encodes their final ,"key":value bytes for
the direct path. After Logger() creates the child, a direct log line appends
the complete bound byte slice once. It does not re-run five key scans and five
value encoders on every record.

Construction is not free. Building the context allocates and copy-on-appends to
keep branched builders from sharing writable backing. HaloLog caps bound context
at 32 fields. Those costs occur when the child logger is derived, outside the
per-line benchmark. A fair evaluation should report both construction cost and
steady-state line cost; they answer different lifecycle questions.

The measured request-scoped scenario uses primitive typed values. Arbitrary
Any values, mutable references, duplicate keys, and transform-heavy
configurations deserve separate ownership and semantic tests; they do not get
to borrow the primitive direct-path result.

3. The fast path is an eligibility decision

With one raw-capable JSON adapter and no masking or sampling, HaloLog can encode
typed fields directly into a pooled byte buffer. The line becomes:

cached header + escaped message + bound context + call-site fields + "}\n"

There is no intermediate LogEntry on that path.

But masking and sampling need structured data. If either is configured, or if
the output shape cannot use the direct JSON encoder, the logger selects the
capture path at construction. Bound context is then prepended as structured
fields, the transform sees the complete record, and the formatter emits it
afterward.

This is the security property I care about: the optimized path is not asked to
remember to call the masker. It is ineligible when masking exists.

That does not prove every regular expression is a sufficient PII policy, that
arbitrary values cannot contain secrets, or that encryption and retention are
solved. It proves a narrower architectural invariant: configuring the shipped
masking transform prevents this direct encoder from bypassing it.

4. The string scanner optimizes the common case and retains an oracle

JSON strings only need special handling for control bytes, quotes, and
backslashes. On the little-endian targets covered by the current implementation,
HaloLog's scanner examines eight bytes at a time with SWAR (SIMD Within A
Register) bit tricks. A clean word advances without eight separate table
checks. A possible hit falls back to the exact scalar escape path. Other
architectures must earn their own correctness and performance result; this
article does not infer one from amd64.

The optimization is guarded by two different kinds of evidence:

• exhaustive placement of every possible byte value across SWAR lanes and boundary positions;
• structured cases and a Go fuzz target compared against the retained scalar implementation.

“Exhaustive” applies to the enumerated byte-position test, not to the infinite
space of possible strings. Fuzzing explores additional combinations; it is not
a proof by itself.

An experimental portable SIMD implementation produced a surprising result on
one Go version and machine: it lost to the shipped SWAR scanner at typical log
string sizes. That result is interesting, but it is not evidence that SIMD is
inherently slower. It belongs to a separately reproducible experiment with the
exact branch, experiment flag, reductions, size distribution, and crossover
points. The full analysis will be a later article rather than a paragraph that
generalizes past its data.

Zero allocation is a regression gate, not an adjective

Benchmark output saying 0 B/op once is weak evidence. A compiler update, an
interface escape, or a seemingly harmless refactor can bring the allocation
back.

HaloLog currently has eight TestZeroAlloc* guards covering selected paths:

• message-only logging;
• interface-style field chains;
• real-adapter dispatch rather than only a discard shortcut;
• direct message encoding;
• bound-context direct and capture paths;
• typed inline and pooled field builders;
• the direct-path field API;
• the level-first line API.

They use testing.AllocsPerRun and fail when the protected operation allocates.
The benchmark workflow treats those tests as a hard gate while reporting
latency without failing on noisy shared-runner nanoseconds.

This claim has a boundary too. Context construction allocates. Configuration
allocates. Long or hostile values can outgrow retained buffers. Some adapters,
arbitrary-value formatting, transforms, and asynchronous ownership have
different cost models. “Eight protected hot paths remain at zero allocations”
is defensible. “The library never allocates” is not.

Allocation tests are not concurrency evidence. That is a separate gate. The
main CI calls a reusable workflow pinned by commit and runs the root module
with the race detector on both Linux and Windows. The otelbridge module runs
its own race-enabled suite. This still does not establish linear scalability or
prove the absence of all concurrency bugs; it means the committed test
executions are race-instrumented on those runners.

Two encoders, one tested byte contract

The direct path is optimized for the simplest eligible JSON configuration. The
capture path exists because real logging systems need transforms, multiple
adapters, sampling, metrics, and less specialized values.

Maintaining two implementations normally creates semantic drift. A quote is
escaped in one but not the other. A negative timestamp crosses an epoch
boundary differently. A bound field appears before call-site fields in one
path and after them in another. A security transform sees only one
representation.

HaloLog keeps a reference-style formatter and compares the direct path against
it. Golden tests pin complete output segments. Context tests normalize only the
timestamp portion and compare the remaining bytes. Key-prefix and string
oracles retain the older scalar form and compare optimized output against it.

This is differential testing applied inside one library. It does not prove that
the chosen JSON contract is universally ideal. It makes unintended divergence
between the two implementations observable.

The same discipline applies to halologgen, the included schema generator. It
regenerates the committed example into a temporary directory and byte-compares
the result. A misspelled field method or wrong value type becomes a compile-time
problem; nondeterministic generator output becomes a failing test. That is a
different thesis from runtime encoding performance, so it receives its own
article in this series.

Why there is no corrected ranking in this article

The repository retains the original published comparison and its methodology.
I am not pasting that table here and quietly changing one competitor's
configuration underneath it.

The corrected evidence release will upgrade Zap, name both timestamp encoders,
and report Linux and Windows separately. It will use at least ten samples per
row, retain raw benchmark output, publish benchstat summaries, and treat cells
inside the declared noise threshold as ties. A delta measured on one operating
system will not be projected onto another.

Until that release is frozen, the defensible result is methodological:

The maintainer feedback identified a faster, semantically different Zap
timestamp configuration. The difference was large enough in exploratory
measurement to require a separately named result, and the harness now has
executable output checks that make the distinction visible.

That conclusion survives even if HaloLog's final median or ranking changes.

What this article does not prove

It does not prove that HaloLog is the best logger for every Go service.
Performance is one selection criterion. API stability, ecosystem maturity,
operational familiarity, integrations, support horizon, and the cost of
changing an existing logging stack may matter more.

It does not measure storage throughput or durability. io.Discard removes the
destination so encoding and dispatch remain visible. A file, socket, collector,
or congested pipeline can dominate the total cost.

It does not show multi-goroutine scalability or p99 caller latency. Those need a
macro harness with controlled sinks, concurrency, queue capacity, drop policy,
and backpressure. That work is planned separately.

It does not prove universal zero allocation. It identifies and guards exact
paths on a named Go toolchain.

It does not prove SIMD is slow, that a 10 ms cached clock is suitable for every
system, or that matching four semantic fields makes all logging formats
interchangeable.

And a new public project with a small external user base has not earned the
ecosystem evidence of Zap, zerolog, or the standard library. Microbenchmarks can
measure code. They cannot manufacture operational history.

Reproduce it, then try to invalidate it

The repository pins dependency versions, test sources, and the historical
methodology. The core checks are ordinary Go tests:

go test ./core -run 'TestZeroAlloc|TestDirectPath|TestContext_' -count=1

go test ./adapters/formatters/json \
  -run 'TestSWAR|TestDirectMatchesFormat|TestAppendKeyPrefix|TestFormatterGolden|TestHeaderZeroAlloc' \
  -count=1

go test -race ./... -count=1

The corrected comparative run uses the normal Go benchmark toolchain. The Zap
timestamp A/B remains a named benchmark instead of being hidden inside a
generic Zap row:

cd benchmarks
go test -run '^$' \
  -bench 'Benchmark(Info|OneField|TenFields|TwentyFields|Context|ZapTimeEncoder)$' \
  -benchmem -benchtime=1s -count=10 . | tee results.txt
benchstat results.txt

There are several useful ways to break the claim:

• produce a record that violates the declared output contract;
• find a protected path that allocates on the release toolchain;
• demonstrate a race or ownership failure;
• show that a competitor configuration performs equivalent work with less overhead;
• reproduce a ranking change on a documented machine;
• find a masking, escaping, timestamp, or context case where the two paths diverge.

Open an issue with the smallest counterexample and the complete environment.
That is more valuable than a benchmark screenshot.

The result I want to preserve

The original result was a low latency number. The more important result is a
process that can survive having that number challenged.

A maintainer found a configuration difference. The benchmark changed. Output
assumptions became tests. The faster competitor path will be published beside
the production default instead of hidden or used to overwrite history.

That is the standard I want HaloLog to meet:

Fast paths may be specialized. Claims about them may not be vague.

The next articles will go deeper into benchmark contracts, bind-once request
context, byte-identity testing, allocation guards, SWAR versus portable SIMD,
compile-checked telemetry generation, and the gap between encoding
microbenchmarks and logging under load.

For now, the invitation is simpler: run the evidence, inspect the output, and
find the condition I missed.

Repository: Go-Gen-Ecosystem/halolog

Benchmark source: comparison_bench_test.go

Request-context benchmark: context_bench_test.go

Published historical methodology: comprehensive_comparison.md

Zap fairness discussion: uber-go/zap discussion #1576

The next article will isolate benchmark fairness itself: timestamp semantics,
output contracts, maintainer feedback, and what belongs inside the timed loop.