Analog LLM: Analog Foundation Models Robust to Hardware Noise. Analog In-Memory Computing

Analog Foundation Models Robust to Hardware Noise

The Generative AI Futures podcat introduces research on Analog Foundation Models (AFMs), a method for adapting large language models (LLMs) like Phi-3-mini and Llama-3.2 to run efficiently on Analog In-Memory Computing (AIMC) hardware. This approach is necessary because AIMC, while offering high speed and power efficiency by bypassing the von Neumann bottleneck, introduces challenges such as noise and strict quantization constraints that degrade standard LLM performance. The authors present a general and scalable Hardware Aware (HWA) training pipeline using knowledge distillation and iterative weight clipping to increase model robustness, achieving performance comparable to high-precision quantized digital models even under hardware-realistic noise. Furthermore, the AFMs retain crucial capabilities, including safety and instruction following, and show improved scaling behavior in test-time compute compared to other quantized models.