Palo Alto-based artificial intelligence chip startup, SambaNova Systems, made waves on Tuesday with the announcement of their latest semiconductor, the SN40L chip.
This breakthrough chip is engineered to empower customers to utilize higher quality AI models at a reduced overall cost, marking a significant advancement in the AI hardware landscape.
SambaNova’s SN40L, manufactured by TSMC, can serve a 5 trillion parameter model, with 256k+ sequence length possible on a single system node. This enables higher quality models, with faster inference and training at a lower total cost of ownership.
The SN40L chip stands out for its capability to run AI models exceeding twice the size of what the advanced version of OpenAI’s ChatGPT utilizes, according to SambaNova Systems. Rodrigo Liang, the CEO of SambaNova, emphasized that the SN40L chip is purposefully tailored for running large language models in enterprise applications.
“SN40L is specifically built for large language models running enterprise applications. We’ve built a full stack that has allowed us to really understand the enterprise use case really well,” Liang stated.
Security, accuracy, and privacy are paramount concerns in the enterprise AI landscape, requiring specialized design considerations. While Nvidia currently dominates the AI chip market, burgeoning interest in generative AI software has sparked a surge in demand, making these chips challenging for certain companies to acquire. In response, companies like Intel, Advanced Micro Devices (AMD), and startups such as SambaNova have stepped in to meet this growing demand and fill the void in the market.
The newly unveiled SambaNova chip boasts the capacity to power a 5 trillion parameter model, incorporating two advanced forms of memory. Memory efficiency is a critical aspect in processing AI data, and SambaNova’s solution addresses this by mitigating memory-related bottlenecks. By combining innovative hardware elements, the SN40L chip empowers customers to run larger AI models without compromising accuracy for model size.
Taiwan Semiconductor Manufacturing Company (TSMC) is the chosen manufacturer for SambaNova’s groundbreaking chip. This collaboration underscores the strategic partnership and commitment to delivering cutting-edge AI hardware solutions that have the potential to reshape the landscape of artificial intelligence.
SambaNova Systems is backed by investors such as SoftBank Vision Fund 2, funds and accounts managed by BlackRock, Intel Capital, GV, Walden International, Temasek, GIC, Redline Capital, Atlantic Bridge Ventures, Celesta, and several others.
