How LLMs can be Assisted to do Arithmetic Correctly

Alvaro Videla's Rune project monitors LLM internal states to correct arithmetic errors by injecting correct results back into inference. The system detects arithmetic queries and overrides probabilistic outputs with deterministic answers. It failed to reliably perform calculations despite using JIT compilation for model intervention. The effort shows LLMs are unsuitable for precise arithmetic tasks.