Introducing Slim-Llama, an ASIC processor capable of efficiently supporting LLMs with 3 billion parameters while consuming only 4.69mW.

Large Language Models (LLMs) have revolutionized artificial intelligence, yet their immense power requirements pose significant barriers to deployment, particularly in energy-sensitive environments like edge devices. As organizations aim for wider accessibility to these powerful models, there is an urgent necessity for energy-efficient solutions that can handle the complexity of billion-parameter models without incurring prohibitive operational costs.

Slim-Llama, a pioneering Application-Specific Integrated Circuit (ASIC) from the Korea Advanced Institute of Science and Technology (KAIST), addresses these energy challenges head-on. By utilizing innovative binary and ternary quantization methods, Slim-Llama reduces model weight precision to just 1 or 2 bits, significantly decreasing memory usage while maintaining performance integrity. This processor eliminates reliance on external memory, employing on-chip SRAM to achieve bandwidth of up to 1.6GB/s at 200MHz. With capabilities to manage 3 billion parameters and reaching latency as low as 489 milliseconds, Slim-Llama sets a new benchmark for energy-efficient processing in large-scale AI applications. Furthermore, its impressive 4.69mW power consumption showcases an energy efficiency improvement of 4.59 times over predecessor solutions, making it an ideal contender for real-time AI tasks.