Arithmetic Without Numbers – How LLMs Do Math
hackernews
Probe
gcd(84, 36) is already inside the model as vectors. Can a small readout recover the operation and numbers?
hidden vector only
dot
op=gcd
a=84 b=36 decoded arguments
a=84 b=36 decoded arguments
A probe is an external measuring instrument. We freeze the model, collect activation vectors, and train a small readout to answer questions such as "is this gcd?" or "is operand A equal to 84?"
Shows: information is readable. Does not show: the direction caused the model's behavior.
Source: hackernews