Specialized Compute Infrastructure, Part 2

Compute infrastructure, the collection of hardware and software elements needed to enable cloud computing, has been developing at an astonishing pace (see here for Part 1 of Specialized Compute Infrastructure).

Silicon Processes

The semiconductor industry is continuously fine-tuning processes for manufacturing ASICs, and silicon fabrication is becoming increasingly centralized and capital intensive. TSMC, the leading pure-play foundry, and Samsung have the biggest market shares. They are advancing towards increasingly smaller processes – 3nm is currently in production, with plans for 2nm in 2025 and 1.4nm in 2027.

Supply & Demand

As industries move towards more application-specific hardware, and compute infrastructures shift from homogenous to heterogeneous computing, the need for novel next-gen hardware infrastructure is growing exponentially, while supply lags.

The CapEx and OpEx involved in manufacturing semiconductors are increasing in line with the complexity of the product. In the early 2000s there were over 30 foundries manufacturing semiconductors, and most were integrated with chip design companies (like Texas Instruments). Due to this consolidation, today there are only four advanced chip foundries with high volume manufacturing capacity – TSMC, Samsung, Intel, and Global Foundries.

In response to the astronomical costs of building chip manufacture infrastructure, coupled with the US’s interest in moving more processes and supply chain nodes to US soil, the Biden administration’s 2022 CHIPs and Science Act allocated a $208B investment in the US semiconductor industry for manufacture and R&D. This came at the same time that TSMC announced an allotment of $40B towards fab in Arizona, Samsung announced $17B for fab in Texas, and Intel is also targeting building a new fab in Arizona.

Specialized Compute: Modernizing Infrastructure

In recent years, the biggest data centers, hyperscalers, and cloud service providers have been looking at their supply chain for opportunities to add specialized hardware innovations to their offerings. Their aim is to differentiate, compete and gain market share in this $500B annual cloud services market, expected to more than double by 2030.

Specialization is the driving factor in winning massive markets.  The emerging, ever-changing tech landscape has proven in many cases to represent a tipping point for many giants. ChatGPT is a most recent example of this.

We see that aggressive M&A strategies that add specialized compute infrastructures to their product serve as a huge value added. AWS acquired Annapurna Labs in 2016, fueling growth by integrating ASICs into AWS’s EC2 offering. AMD acquired Xilinx in 2020 and Pensando in 2022 to add FPGAs and smartNICs to its product line. Qualcomm acquired Nuvia, headed by the Chief Architect of Apple’s revolutionary M1 chip. As businesses increasingly rely on TTM solutions to be competitive, compute capabilities become more competitive and specialized solutions are winning market share.

Building ASICs

While hyperscalers and enterprise companies traditionally focused on, and excelled in software solutions, today they are aggressively developing their own specialized hardware – some more successfully than others.

Companies including Block, Meta, Google, Apple, and Amazon are expanding their in-house teams of chip design engineers dedicated to hardware infrastructure. Though many companies struggle with building hardware, success can translate into massive revenues; developing the right hardware can be the ‘make it or break it’ for expansion. For example, Google Cloud’s TPU accelerates dense vector and matrix computations (for ML) and contributed to Google Cloud increasing its cloud computing market share in 2022.

Cloud providers such as Microsoft Azure, Oracle and others seek next-gen compute solutions that will help them diversify and maximize their offerings and grow market share. Security and privacy are a top concern for them as they seek to appeal to sectors that cannot modernize their workloads to the cloud but that represent hundreds of billions of dollars of potential revenue. Once robust hardware and software solutions are available to implement scalable solutions for complex operations, such as FHE, ZKP and others, this market will quadruple in size.

Conclusion

Disruptive new compute technologies will continue to reshape our present and future, affecting every aspect of our lives. With a barrage of novel software and hardware breakthroughs across Blockchain, Quantum, AI and Privacy, we’re certain to see almost every global industry affected, hopefully for the better.