Trustbit ML Lab Welcomes Grayskull e150 by Tenstorrent

This is an AI hardware from Tenstorrent - Grayskull e150. Two cards are anticipated to be delivered to the Trustbit ML lab within the next few days.

Grayskulls are the first generation of AI/ML accelerators from Tenstorrent - Canadian company lead by the industry veteran Jim Keller.

Right now, you could be reading these words on a device that was influenced by his work. Jim Keller has made significant contributions across the tech industry:

• The Architect Behind x86-64: Keller's work on the x86-64 instruction set is foundational for CPUs in today's home computers and servers.

• Innovation at Apple: At Apple, he was instrumental in developing the A4 and A5 processors, predecessors of the modern M1, M2, and M3 processors.

• Advancements at AMD: Keller's involvement with the Zen architecture has powered Ryzen and Epyc processors.

• Pioneering Auto-Pilot Hardware at Tesla: He was also behind the auto-pilot hardware at Tesla, enhancing vehicle automation and safety. (but we do hope that you are not reading these words while driving).

And for the last few years Jim Keller was busy building new generations of AI processors at Tenstorrent.

The unique capability of Tenstorrent AI hardware - ability to run AI/ML workloads on a reconfigurable mesh of individual energy-efficient processors.

For example, we are expecting e150 Grayskull cards. Each card has 120 Tensix cores. Each Tensix core hosts 5 smaller (RISCV) cores with dedicated accelerators. Peak performance (fp16) is around 98Tflop/s.

When joined together via network-on-a-chip (NoC), they form a reconfigurable data processing pipeline on a chip. It can even run smaller specialized ML models like BERT, Whisper and some smaller flavours of GPT.

All this can happen locally, in your own data-center, without sending data to a third party in the cloud. This is what really interests business customers of Trustbit.

Yes, Nvidia can already do that.

The unique selling point of Tenstorrent hardware is about 3 things:

1. You will not be laughed at, when you try to buy a pack of their cards (to compare, try buying a couple of NVidia A100/H100 GPU cards on an open market today - it is nearly impossible).

2. Next generations of Tenstorrent cards are designed to natively connect to each other, forming large compute meshes. This mesh can elastically grow in your datacenter according to your needs, at a fraction of NVidia costs.

3. Tenstorrent cards have a compute profile that is different from NVidia cards. Their architecture lends itself better to low-latency real-time applications. Energy consumption and TCO are also lower in some cases. Hyundai, Samsung and LG - are already working with Tenstorrent. They might have an idea or two about that.

The main catch is about the new software stack, that deviates from the industry. It might take a bit of time to get used to.

That’s why, at Trustbit, we are already looking into first generations of these cards. As we build expertise in building AI-driven solutions with Tenstorrent hardware, all of our European customers will be able to benefit from lower costs and better hardware availability. Especially, when it comes to privacy-sensitive enterprise  cases and local deployments.