初创公司Cerebras推出 WSE-3 AI芯片, 4万亿个晶体管！

Cerebras Systems has announced the launch of Wafer Scale Engine 3 (WSE-3), a groundbreaking AI wafer-scale chip with 4 trillion transistors, 900,000 AI cores, 44GB on-chip SRAM, and peak performance of 125 FP16 PetaFLOPS. This new device is twice as powerful as its predecessor, the WSE-2, and is manufactured using TSMC's 5nm process technology.

WSE-3 powers the CS-3 supercomputer, which can train AI models with up to 24 trillion parameters - a significant leap compared to supercomputers driven by WSE-2 and other modern AI processors. The supercomputer supports external memory ranging from 1.5TB to 1.2PB, allowing it to store large models in a single logical space without partitioning or restructuring, simplifying the training process and enhancing developer efficiency.

(Cerebras image)

In terms of scalability, the CS-3 can be configured in clusters of up to 2048 systems. This scalability enables it to fine-tune a 70 billion parameter model in one day through a four-system setup and fully train the Llama 70B model within the same timeframe. The latest Cerebras software framework provides native support for PyTorch 2.0 and accelerates training with dynamic and unstructured sparsity, which is eight times faster than traditional methods.

(Cerebras image)

Cerebras highlights the superior power efficiency and ease of use of the CS-3. Despite doubling its performance, the CS-3 consumes the same amount of power as its predecessor. It also simplifies the training of large language models (LLMs), requiring up to 97% less code compared to GPUs. For instance, a GPT-3-sized model requires only 565 lines of code on the Cerebras platform.

The company has received considerable interest in the CS-3, with a backlog of orders from various sectors including enterprise, government, and international cloud providers. Cerebras collaborates with institutions such as Argonne National Laboratory and Mayo Clinic, demonstrating the potential of the CS-3 in healthcare.

Cerebras' strategic partnership with G42 will expand with the construction of Condor Galaxy 3, an AI supercomputer featuring 64 CS-3 systems with up to 57,600,000 cores. Together, the companies have already created the world's two largest AI supercomputers, CG-1 and CG-2, located in California with a combined performance of 8 ExaFLOPs. This collaboration aims to provide global AI computing at dozens of exaFLOPs.

Kiril Evtimov, CTO of G42 Group, said, "Our strategic partnership with Cerebras plays a crucial role in driving innovation at G42 and contributing to the acceleration of the global AI revolution. The upcoming Condor Galaxy 3, with 8 exaFLOPs, is currently under construction and will soon increase our system's total AI computing capacity to 16 exaFLOPs."

参考资料：

中国无锡半导体设备年会展览会

CSEAC

举办地区：江苏

开闭馆时间：09:00-18:00

举办地址：无锡市太湖新城清舒道88号

展览面积：48000㎡

观众数量：58000

举办周期：1年1届

主办单位：中国电子专用设备工业协会