Multimodal Models Are Only as Good as Their Weakest Encoder
TL;DR — Multimodal models look like a single system in the demo but are actually a pipeline of independently-trained encoders bolted onto a shared decoder. Each encoder carries its own training distr