The headlines are dominated by the unprecedented growth in data centers and their rising energy consumption, along with the skyrocketing user cost of AI for inference at scale. Perhaps less obvious is the direct connection between the two – reducing energy consumption reduces AI cost. This is because both require smart use of AI system capability at maximum efficiency, which is often not the case today.
Energy efficiency must be a strategic priority for all companies using AI inference, not just for data center builders. Smarter AI usage with lower energy consumption and less cost will help enhance the financial bottom line. Leadership in energy efficiency innovation will also generate a topline reward for companies expanding into AI’s next frontier, physical AI – when AI ventures off the computer screen into the real world. A robot, a drone, or a wearable typically run on a limited battery, so energy efficiency is a must-have, not optional!
Each of these challenges can turn into opportunities.
Energy
AI training compute is growing at an estimated 4–5x per year, with the latest large language models (LLMs) reportedly using 5-50 trillion parameters. The rapid growth of AI inference at scale amplifies this massively and generates enormous energy demand. In the US, data center energy consumption has tripled over the past decade and by most projections, it will triple again in just 5 years! The International Energy Agency (IEA) estimates that globally, data centers consumed ~460 terawatt-hours (TWH) of electricity in 2025, which will more than double to 945 TWH by 2030. For perspective, this is more power than the entire country of Japan uses today!
The global energy infrastructure was not designed to absorb this level of demand. Communities across the world are pushing back against this strain on their power grid and water supply, as prices rise and shortages occur. Nuclear energy may help augment the traditional grid, but it is a distant hope, at least a decade away. This challenge cannot be ignored and could become a major roadblock for the future of AI. It is no surprise that energy is now the strategic currency for AI globally, with AI business transactions now being announced in Gigawatts, rather than Megaflops.
Cost
Investment in AI system infrastructure is rising at a dizzying pace: top players invested ~$100B for data centers in 2020, and this will increase TENFOLD to ~$1 trillion in 2026! Broader global investments are even higher. But there is no free lunch: everyone must pay! AI users are now feeling the pinch as they use more AI for inference, and even big players have reportedly blown through their annual AI budgets in just months. To make matters worse, only a small fraction of AI applications currently generate financial value and provide the expected return-on-investment (ROI). In part, this is because basic AI capabilities are becoming table stakes: companies invest in expensive AI tools that may enhance a product or service, but competitors do the same; so there may be no price premium to be had.
AI tokens have become the basic unit for quantifying workloads and cost, as AI providers typically bill usage in millions of input and output tokens. Daily token consumption in the US is projected to increase TENFOLD from today’s ~200 trillion units per day (TUs/day) to ~2250 TUs/day by 2030. Token prices are falling, but not fast enough to keep up with the rising token usage; so AI costs will continue to rise sharply on the current trajectory. Little wonder that CFOs and finance departments are sounding alarm bells and beginning to ration AI budgets.
Physical AI
AI has grown from data analytics to generative AI and is now crossing another frontier by stepping off the computer screen into the physical world. “Physical AI” – intelligent systems like robots, drones, or medical wearables – require fusion of software intelligence with mechanical, electrical, and optical systems like sensors, actuators, cameras, radar, LIDAR etc. Physical AI systems must be autonomous because they cannot rely on data center communication, which adds latency and security risks. So, they must possess “edge intelligence” for real‑time decision‑making to sense, think, and act continuously in the physical world. Each of these functions consumes energy, and an autonomous unit usually operates on a limited battery. Without breakthroughs in energy efficiency, these physical AI systems would either become tethered to power sources or operate with short duty cycles of limited usefulness.
The Path Forward
Innovation is the magic wand that can turn these challenges into opportunities. The technology industry excels at innovation, but the spotlight needs to shift from “ever-larger” to “smarter and efficient” systems. Further, point solutions in silos are no longer sufficient – radical efficiency improvement requires system-level optimization of the entire AI stack.
Join SEMI’s Smart Data-AI Initiative with our Alliance Partner, City of San Jose, for an insightful workshop on September 9 where we will explore how to bend the curve for energy, cost and performance for future AI computing. We unite leading industry experts across the whole AI ecosystem to see the big picture for future materials, devices, and systems; with deep dives on photonics, hardware-software co-optimization, and chip-to-grid enhancements. This is not yet another AI conference – it is a workshop to explore practical solutions and amplify your business strategy by connecting the dots across the AI stack. You’ll learn about cutting-edge innovations, network with experts, and explore meaningful collaborations.
Frequently Asked Questions
Why is energy efficiency becoming a strategic priority for AI computing?
AI training and inference are driving massive increases in data center energy use, putting pressure on power grids, water resources, and operating budgets. Mitigating the skyrocketing cost for AI users is also a growing strategic priority. Reducing energy consumption directly reduces cost of AI for users, since it implies that AI system capability is being used at maximum efficiency.
How are AI tokens connected to rising AI costs?
AI providers typically bill usage based on input and output tokens, making tokens a key measure of workload and cost. As daily token consumption grows, total AI spending can rise even if token prices fall, prompting companies to look for more efficient ways to use AI.
Why does physical AI make energy efficiency even more important?
Physical AI systems such as robots, drones and medical wearables must sense, think, and act in real time, often while running on limited battery power. Without major gains in energy efficiency, these systems could face shorter operating times, reduced usefulness, or dependence on external power sources.
How can companies and industry stakeholders help drive innovation in smarter, more efficient systems?
Companies and industry stakeholders can engage with SEMI’s Smart Data-AI Initiative by joining workshops, connecting with experts across the AI ecosystem, and participating in discussions on materials, devices, systems, photonics, hardware-software co-optimization, and chip-to-grid innovations. These forums are design to help organizations identify practical solutions, explore meaningful collaborations, and shape a more energy-efficient future for AI computing.
Dr. Pushkar P. Apte is Global Lead for Smart Data-AI Initiative & Strategic Technology Advisor at SEMI.
