MicroLLM: Hardware-Aware Compression and Runtime Scheduling of Language Models on Microcontrollers

Deploying Large Language Models (LLMs) and Small Language Models (SLMs) on bare-metal microcontroller units (MCUs) enables private, low-latency, and energy-efficient intelligence for edge Internet of Things (IoT) ecosystems. However, standard MCU architectures operate under severe hardware boundaries, typically featuring less than 2 MB of Flash memory, 256 - 512 KB of SRAM, and milliwatt power envelopes. Pushing language model footprints into the sub-megabyte realm requires aggressive co-compression techniques. This survey paper examines the emerging paradigm of MicroLLM deployment, focusing on the synergy between structured sparsity (eg  block pruning) and ultra-low-bit quantization (specifically sub-4-bit and 2-bit representations like ternary/quaternary weight formats). We evaluate hardware-software co-design pipelines, compilation toolchains (e.g., CMSIS-NN, TinyEngine), memory-scheduling techniques for runtime activations and accuracy-recovery mechanisms. Finally, we highlight current performance benchmarks, trade-offs in perplexity versus memory footprint, and critical open directions for microcontroller-class language modeling