3D stacking

For the semiconductor industry to flourish, all chip ecosystems must embrace closer ties, greater openness, and more flexibility to co-innovate solutions in today’s rapidly unfolding global digital transformation.

As monolithic scaling slows down, the semiconductor industry is increasingly relying on advanced packaging technologies to extend Moore’s law through heterogeneous integration. Higher on-package bandwidth, improved yield resiliency and the need to integrate diverse IP from multiple foundries are driving demand for advanced packaging technologies that address these issues but introduce challenges of their own such as efficient power delivery to all the different domains in a heterogeneous system. SEMI spoke with Kaladhar Radhakrishnan, Intel Fellow at Intel, about heterogeneous system integration trends and new developments in the semiconductor industry. Radhakrishnan shared his views ahead of his keynote at the SEMI Connecting Heterogeneous Systems Summit, 1-3 September 2021, an online event. Join the summit to meet experts from Intel and other key industry influencers. Registration is open. SEMI: What is driving the adoption of electronics and semiconductor devices nowadays and why is the development of new and innovative technologies important? Radhakrishnan: We are living in an increasingly data-driven world where devices have become an integral part of our lives. A recent study estimated that in the United States alone, 13.6 connected devices per capita consume an average of 300 gigabytes worth of data every month. In the workplace, COVID-19 has driven fundamental business changes that has sped up the adoption of digital technologies such as virtual conferencing, remote work, and e-commerce. Organizations are realizing that a high-quality video conference can be an adequate substitute for many in-person meetings. As a result, businesses are accelerating the digital transformation in order to adapt and thrive in this new environment. Five decades of sustained exponential growth in semiconductor performance has conditioned the average digital consumer to expect more from their devices. However, there are some headwinds ahead as traditional scaling slows down and power density rises. Because consumers and businesses are now generating data at a faster rate than they can consume it, technologists need to scale compute, storage, and bandwidth even faster to keep pace. Without investments in research and development of new and innovative technologies to address these challenges, the full potential of this data will go unrealized. SEMI: What forces are heightening the importance of heterogeneous system integration? What are the implications for increased on-package bandwidth, improved yield resiliency and the need to integrate diverse IP from multiple foundries? Radhakrishnan: The semiconductor industry increased transistor density and scaled performance through classical Dennard scaling until the turn of the century. By then, the gate oxide thickness had scaled down to atomic dimensions and the exponential increase in sub-threshold leakage signaled the end of scaling through traditional methods. Since that time, the chip industry has been relying on innovations in transistor materials and structures such as high-k metal gate, strained silicon, and FinFETs to keep pace with Moore’s law. However, this alone will not be sufficient to continue scaling and the industry needs to explore other vectors to augment improvements in transistor technology. Heterogeneous integration through advanced packaging is one key technology that can help drive these gains. Technologies like Foveros can enable device density scaling by creating a 3D stack of multiple die using high-density interconnects. Heterogeneous integration enables chipmakers to move from a monolithic system designed on a single large chip to a heterogeneous system comprised of a number of smaller chiplets. The main benefit of using smaller chiplets is that they improve yield and enable application based customization of the foundry processes. However, if the disaggregation to smaller chiplets is not accompanied by an increase in on-package bandwidth, the power and performance penalties associated with chiplet-to-chiplet communication will hobble system performance. This is why advanced packaging technologies that improve die-to-die communication are key enablers for heterogeneous integration. SEMI: What are some of the key technology challenges in developing heterogeneous systems? Radhakrishnan: The obvious challenge that most people focus on is the need for improved on-package bandwidth. However, as we rely on 3D stacking to continue device scaling at the package level, it is important to comprehend power delivery and thermal challenges as well. Power to the top die has to be delivered through TSVs on the bottom die, which not only adds resistance but also reduces the useful area available on the bottom die. This problem is further exacerbated when we stack more than two die. Excessive noise on the power delivery network can cause timing issues that limit the maximum operating frequency of the transistor. Similarly, when we stack multiple die, we must take into account associated thermal challenges. For example, each interface of the multi-die stack adds thermal resistance, which makes it harder to cool the chips at the bottom. SEMI: What are some of the key global market trends that driving demand for heterogeneous and system-level integration? Radhakrishnan: The number of artificial intelligence (AI) and machine learning applications have grown dramatically due to their ability to solve highly complex problems across a wide range of segments. AI and machine learning models require more memory bandwidth and compute capabilities that are difficult to achieve without some form of heterogeneous integration. Another market trend driving demand for heterogeneous integration is the increasing reliance on custom hardware accelerators. To combat the slowdown in frequency scaling and single-core performance, we have moved to multi-core architectures by tackling the inherent parallelism in our workloads. However, Amdahl’s law tells us that such an approach will hit a bottleneck when we reach the limits of the serial portion of the workload. As these constraints slow the performance of general-purpose processors, the reliance on custom hardware accelerators to boost performance for specific workloads is growing. Heterogeneous integration at the system level with a combination of CPUs, GPUs, FPGAs and other accelerators can optimize system power and performance. SEMI: What solutions is Intel developing to address these market needs? Radhakrishnan: Intel is actively involved in the development of the industry ecosystem for heterogeneous integration. We have developed a number of innovative advanced packaging solutions such as the EMIB and Foveros that are used in products today. Intel is also developing the next generation of advanced packaging technologies, Foveros Omni and Foveros Direct, which will dramatically scale the IO density by using direct Cu-Cu bonding technology. Foveros Omni is a crucial building block technology to enable high-voltage power conversion on the package for efficient power delivery. Intel is uniquely positioned to predict the design needs for future systems and deploy its resources to develop the technology building blocks needed to continue performance scaling. Our IDM 2.0 strategy enables us to leverage our leadership in packaging technologies to design the best products and use the best IP to deliver leading products across a broad range of categories. SEMI: What do you expect from your participation at SEMI Connecting Heterogeneous Systems Summit? Radhakrishnan: I’m hoping to shed some light on some of the new technologies we have been developing at Intel to enable heterogeneous system integration. I also want to bring awareness to the power-related challenges we are facing with heterogeneous systems. I also look forward to listening to what other industry leaders have to say on the topic. Kaladhar Radhakrishnan is an Intel Fellow and a Power Delivery Architect with the Technology Development group at Intel. He plays a significant role in shaping and driving power delivery technologies for Intel microprocessors. His areas of expertise include integrated voltage regulators, advanced packaging and passives technologies. Kaladhar is a two-time recipient of the Intel Achievement Award, the highest Intel honor an individual or small team can receive. He has authored four book chapters, over 40 technical papers in peer-reviewed journals, and has been awarded 35 U.S. patents. He has also served as an adjunct professor at Arizona State University. Kaladhar joined Intel in 2000 soon after receiving his Ph.D. in Electrical Engineering from the University of Illinois at Urbana-Champaign. Serena Brischetto is senior manager of marketing and communications at SEMI Europe.

In the long unfolding arc of technology innovation, artificial intelligence (AI) looms immense. In its quest to mimic human behavior, the technology touches energy, agriculture, manufacturing, logistics, healthcare, construction, transportation and nearly every other imaginable industry – a defining role that promises to fast track the fourth Industrial Revolution. And if the industry oracles have it right, AI growth will be nothing shy of explosive.“The gains these days are not incremental,” said Ajit Manocha, SEMI president and CEO, said to a gathering in July of the Chinese American Semiconductor Professional Association (CASPA) for its Summer Symposium at SEMI’s headquarters in Milpitas. “They are hockey stick – exponential – with AI semiconductors growing in market size from $4 billion this year to $70 billion in 2025.”Manocha left little doubt that AI is remaking the semiconductor industry and, in the process, the world at large. Internet of Things (IoT) and 4G/5G, both key AI enablers, will account for more than 75 percent of device connections by 2025.“Today, 30 billion devices worldwide are connected,” Manocha said, citing an Applied Materials prediction that the number of connected devices globally will grow to between 500 billion and 1 trillion by 2030. Those devices will generate stunning amounts of data collected, interpreted and used to reason, solve problems, learn and plan, leading to the holy grail of autonomous machine behavior.To process this colossal amount of data central to the promise of AI, the industry must break through the limits of a key technology: memory. Memory a Critical AI BottleneckThe challenge for memory starts with performance. Historically, every decade gains in compute performance have outpaced improvements in memory speed by 100 times, and over the past 20 years that gap has grown, said Steven Woo, a fellow and distinguished inventor at Rambus, presenting at the symposium. The upshot is that memory has bottlenecked compute and, in turn, AI performance. The industry has responded with new ways to implement memory systems on AI chips. Each is suited to unique performance requirements and, of course, comes with trade-offs. Among the frontrunners: On-chip memory delivers the highest bandwidth and power efficiency but is limited in capacity. HBM (High Bandwidth Memory) offers both very high memory bandwidth and density. GDDR balances trade-offs among bandwidth, power efficiency, cost and reliability. Since 2012, AI training capability has grown 300,000 times, besting Moore’s law by 25,000 times in doubling every 3.5 months, a blistering pace compared to the 18-month doubling cycle of Moore’s law, Woo said. The staggering improvements have been driven by parallel computing capacity and new application-specific silicon like Google’s Tensor Processing Unit (TPU).These specialized silicon architectures and parallel engines are key to sustaining future gains in compute performance and combatting the slowing of Moore’s Law and the end of power scaling, Woo said. By rethinking the way processors are architected for certain markets, chipmakers can develop dedicated hardware capable of operating with 100 to 1,000 times greater energy efficiency than general purpose processors to overcome another big limiter to scaling compute performance – power.For its part, the memory industry can improve performance by signaling at higher data rates and using stacked architectures like HBM for greater power efficiency and performance, and by bringing compute closer to the data.Memory scaling for AIA key challenge is scaling memory for AI. Demand for better voice, gesture and facial recognition experiences and more immersive virtual reality and augmented reality interactions is tremendous, said Bill En, senior director at AMD, speaking at the symposium. These capabilities require more processing power across both high-performance computing (HPC) for big data analytics and machine learning as it relies on AI and machine intelligence to generate meaningful insights. Emerging machine learning applications include classification and security, medicine, advanced driver assistance, human-aided design, real-time analytics and industrial automation. And with 75 billion IoT-connected devices – all generating data – expected by 2025, there will be no shortage of data to analyze, En said. The wings alone of a new Airbus A380-1000 feature some 10,000 sensors.Mountains of this data are stored in massive data centers on magnetic hard drives, then transferred to DRAM before moving to SRAM within the CPU for the handoff to the compute hardware for analysis.With data growing at an exponential clip, the question is how to make sure all other memory systems can handle the flood of data. AMD’s answer is a chiplet architecture featuring eight smaller chips around the edge that drive the compute and a large chip in the center that doubles the IO interface and memory capability to in turn double chip bandwidth.AMD has also moved from a legacy GDDR5 memory chip configuration to HBM to bring memory bandwidth closer to the GPU for more efficient processing of AI applications. The HBM provides much higher bandwidth while reducing power consumption. Compared to DRAM, AMD’s HBM delivers a much faster data rate and far greater memory density, En said.Over the next decade, look for more performance improvements from multi-chip architectures, innovations in memory technology and integration, aggressive 3D stacking and streamlined system-level interconnects, he said. The industry will also continue to drive performance gains in devices, compute density and power through technology scaling.Michael Hall is a global marketing communications manager at SEMI.