In-depth analysis by SemiAnalysis founder: How long can AI investments last? Can NVIDIA's moat be maintained? When will the memory shortage end?

Investment in AI infrastructure still has room for growth, but rising financing costs, constraints in the semiconductor supply chain, and intensified competition in self-developed chips are collectively testing the sustainability of this round of AI boom.

At the 35th Annual Conference of CASPA held in the United States on October 23, SemiAnalysis founder and CEO Dylan Patel, Bernstein Managing Director and Senior Analyst Stacy Rasgon, and Austin Lyons discussed the prospects of the semiconductor industry, focusing on AI infrastructure investment, supply chain bottlenecks, and the competitive landscape of chips.

Patel pointed out that the current demand for AI computing power remains strong, but capacity constraints in key areas such as substrates and memory are limiting infrastructure expansion. Meanwhile, financing costs have surged, making it increasingly difficult for emerging cloud service providers without long-term customer orders to secure funding. Whether AI investment can continue to advance depends not only on demand but also on the financing capacity and returns of the projects.

Regarding NVIDIA, Rasgon believes that its product performance improvements can still support high prices, but the self-developed chips from cloud providers are changing the competitive landscape. As companies like Google with TPU, Amazon with Trainium, and OpenAI advance their self-developed chip plans, NVIDIA may face dual pressures on market share and pricing power even if it continues to grow.

The memory market faces another challenge: while HBM demand is strong and industry profit margins are high, manufacturers' capacity expansion may gradually change the supply-demand relationship after 2028. In the next two years, financing capacity, supply of key components, and competition from self-developed chips will become critical variables affecting the speed of AI infrastructure expansion and the distribution of profits in the industry chain.


Key points summarized by Wall Street Insights are as follows:

AI investment still has growth potential, but financing costs are becoming a significant constraint. The yield on 10-year U.S. Treasury bonds has surpassed 5%, and the yield on unsecured bonds from some emerging cloud service providers is approaching 15%. Companies lacking long-term customer orders face even greater financing difficulties, and the pace of project construction will increasingly depend on financing capacity and investment returns.

Short-term semiconductor supply chain bottlenecks are difficult to resolve, with substrates and memory being key constraints. Substrates, HBM, optical interconnect components, and cleanroom capacity all face supply limitations. In contrast, power shortages can be partially alleviated through on-site generation, while semiconductor supply chain expansion requires more time.

Memory shortages may persist until new capacity is released, with a turning point in supply-demand dynamics expected after 2028. HBM demand is strong, and the memory industry's gross profit margin is reportedly at 88%—90%; however, if new capacity is concentrated in production after 2028 while demand growth slows, memory prices and profit margins may come under pressure.

NVIDIA still has a performance advantage, but self-developed chips are challenging its pricing power. NVIDIA continues to support high prices through performance improvements, but the self-developed chip strategies of companies like Google with TPU, Amazon with Trainium, and OpenAI provide alternatives for large customers. Even if NVIDIA's revenue continues to grow, its market share and profit margins may not increase in tandem.

The cost of AI infrastructure continues to rise, and investment returns will become the focus of the next phase. The cost of building a 1-gigawatt computing power data center has risen from about $40 billion in 2024 to over $60 billion. Cloud providers are accelerating self-developed chip initiatives to reduce unit computing power costs, and whether companies can continue to expand their model of purchasing tokens through cloud services will impact the long-term returns on infrastructure investment.

### How Long Can AI Investment Last? Financing Costs and Supply Chain Become Key Constraints

The financing threshold for AI infrastructure construction is significantly increasing. The yield on 10-year U.S. Treasury bonds has surpassed 5%, and the yield on unsecured bonds from emerging cloud service providers like CoreWeave is approaching 15%. The bond issued by Volulta for building clusters for Anthropic has a coupon rate of 12% but was sold at a price of 95%, resulting in an actual yield of about 15%.

Rasgon pointed out that previously cash-rich hyperscale cloud providers, such as Meta and Google, are also consuming significant free cash flow in AI infrastructure construction, and some companies may even need to support investments through equity financing in the future.

Patel further noted that the construction demand undertaken by non-hyperscale cloud providers accounts for nearly half of the entire market, but these companies typically lack sufficient balance sheet support. Without long-term procurement commitments (offtake agreements) from hyperscale cloud providers, projects are unlikely to secure financing. To address this, NVIDIA has begun to support the construction of some computing power clusters through revenue-sharing arrangements, while Broadcom has provided chips to Anthropic in a similar installment payment manner.

This means that financing capacity is becoming a critical dividing line for the expansion of AI infrastructure. Strong demand does not guarantee that all projects can be realized on schedule; for emerging cloud service providers that rely on external financing and face high capital costs, long-term orders and cash flow will directly impact construction speed.

### Substrates and Memory Are Harder to Expand Rapidly Than Power

In terms of supply chain bottlenecks, Patel listed substrates as one of the current key constraints. Japanese suppliers are relatively cautious about capacity expansion, limiting the acceleration of production for chips, CPUs, and other products.

Memory is another major bottleneck. Rasgon pointed out that the number of wafers required to produce the same capacity of HBM is about 3—4 times that of DDR5, and the growth in AI chip demand has intensified competition for memory capacity. Meanwhile, components such as lasers and optical fibers required for optical interconnects are also facing shortages, and even if CPO plans accelerate mass production around 2028, related supply may still struggle to meet demand.

The wafer fabrication equipment market is also constrained by cleanroom capacity. Rasgon believes that the forecast for the global wafer fabrication equipment (WFE) market to exceed $200 billion by 2026 is overly optimistic. By improving the utilization of existing factories, the market size may increase from the early expectation of about $120 billion to about $155 billion, but new cleanrooms will not be put into use until 2027, with more capacity expected to be released in 2028—2029.

In contrast, power shortages have more coping strategies. Patel expects that by 2027, the scale of data centers under construction globally will approach 50 gigawatts, with about 35 gigawatts in the United States. On-site gas turbines, reciprocating engines, and fuel cells can partially alleviate power constraints.

Therefore, the bottlenecks faced in AI construction are not simply due to a lack of electricity or space, but rather the difficulty of synchronously expanding various key materials and manufacturing processes. In the short term, supply chain constraints and financing costs may jointly affect project delivery speed.

### When Will the Memory Shortage End? Supply Release After 2028 Is Key

The current high profits in the memory industry reflect the imbalance between AI demand and limited capacity. Rasgon stated that the gross profit margin in the memory industry has reached 88%—90%, but the market has not given related stocks extremely high valuations, partly due to investors' concerns about a concentrated release of new supply after 2028 leading to price declines.

The uniqueness of HBM lies in its manufacturing requiring more wafer resources. As demand for AI accelerators grows, HBM capacity expansion not only needs to increase its own capacity but also affects the supply of other DRAM products. In the short term, capacity allocation and expansion speed are unlikely to keep pace with demand; however, as new capacity gradually comes online, the supply-demand relationship may also reverse.

When the memory shortage will end is not determined by a fixed date but by when new capacity begins to exceed the increase in demand. If expansion is concentrated after 2028 while AI demand growth simultaneously slows, the current high prices and profit margins may be difficult to maintain.

Investors need to pay attention to the progress of HBM and DRAM capacity expansion, advanced packaging, and wafer manufacturing capabilities, as well as changes in cloud providers' capital expenditures. Before supply significantly increases, memory may still constrain the delivery of AI computing power; however, over a longer period, expansion may also become a factor that depresses prices and erodes profits.

### Can NVIDIA Maintain Its Moat? Performance Advantage Faces Challenges from Self-Developed Chips

NVIDIA's core advantage lies in continuously enhancing chip performance and strengthening customer loyalty through its software ecosystem and development tools. However, as cloud providers increasingly focus on computing power costs, its high prices also provide room for the development of self-developed chips.

Rasgon pointed out that NVIDIA's valuation relative to the Philadelphia Semiconductor Index is at a historical low, reflecting market concerns about the sustainability of AI investment and competition from self-developed chips.

From a product iteration perspective, NVIDIA continues to support pricing through performance improvements. From Ampere to Hopper, the average selling price has roughly doubled, with performance improving about tenfold; from Hopper to Blackwell, prices have increased by about 50%, with performance reportedly improving by 3—30 times. As long as performance improvements continue to exceed price increases, customers will be motivated to pay higher prices for the new generation of products.

However, competitors are accelerating their catch-up. Patel's research shows that NVIDIA's share of related co-processor wafers and HBM demand has begun to decline. Meanwhile, Google TPU has started selling to external customers, Amazon Trainium continues to enhance support for the PyTorch ecosystem, and OpenAI and Anthropic are also advancing their self-developed chip plans.

These developments may not immediately replace NVIDIA but can provide alternatives for large customers, enhancing their bargaining power. For cloud providers with substantial computing power needs, the appeal of self-developed chips lies not only in reducing procurement costs but also in decreasing reliance on a single supplier.

Therefore, the key test for NVIDIA is not whether it can continue to grow, but whether it can consistently justify its price premium through performance advantages. Even if the overall AI market expands, NVIDIA's revenue, market share, and profit margins may not necessarily grow in tandem.

### Enterprise AI Implementation: Purchasing Tokens Is More Realistic Than Building Computing Power

On the enterprise application side, the three analysts believe that most companies will not purchase large amounts of hardware to run private models but will instead buy tokens through cloud services, utilizing cloud-based AI inference capabilities and integrating them with their own data and business processes. This trend is similar to the past shift in enterprise IT from building infrastructure to cloud computing.

Currently, AI-assisted programming is one of the more mature enterprise applications. Rasgon shared that with Claude, a financial model that originally took a week to complete was built in just about 1 hour and 20 minutes, involving logical deductions from 10 years of historical data, multiple business reorganizations, and management conference call content.

The value of such applications primarily manifests in time savings and reduced repetitive labor, rather than immediately reflecting in revenue. Their investment returns still need to be validated through actual business performance, but companies calling models through the cloud also mean that the demand for AI infrastructure will increasingly depend on the usage of inference services.

If enterprise applications continue to expand, cloud providers will have the incentive to keep investing in computing power; however, if construction costs and financing costs rise faster than revenues, the returns on infrastructure investment will still be under pressure.

Here is the full content:

Host Austin Lyons:

Let’s start from a more macro perspective, first discussing demand and then supply. On the demand side, AI revenue is growing very quickly, but the growth rate has begun to slow. Currently, the total revenue of the entire AI industry is still just a small part of the total industry expenditure, so the market is scrutinizing every revenue data point very carefully.

On the supply side, the computing power capacity for the next few years has been fully booked, and prices remain strong. From the perspective of semiconductor analysts, how should we understand these two signals? Ultimately, where will the value in the industry chain flow?

Today, we have invited two important figures from the semiconductor industry: Dylan Patel, the founder and CEO of SemiAnalysis, one of the most influential research institutions in the field of AI infrastructure; and Stacy Rasgon, Managing Director and Senior Analyst at Bernstein, who has been involved in semiconductor analysis on Wall Street for over a decade. I am your host today, Austin Lyons, currently the Chief Strategist and Chief Analyst, and a former Chief Engineer at Intel.

Today, we will discuss the future of the semiconductor industry, looking ahead to 2027 and 2028, covering areas such as financing, GPU, CPU, and wafer foundry.

### 1. AI Infrastructure Investment: Will Financing Costs Slow Down Expansion?

Host Austin Lyons:

Recently, capital costs have become the focus of market discussions. Hyperscale cloud providers are investing huge amounts of money, and free cash flow has turned negative. Meanwhile, the yield on the 10-year U.S. Treasury bond has surpassed 5%, and the bond market yields for companies like CoreWeave are approaching 15%.

Should we be worried about this? As companies continue to increase investments and financing costs rise, how much impact will this have on AI infrastructure construction? Let's start with Stacy.

Stacy Rasgon:

There are indeed many limiting factors currently slowing down the pace of AI infrastructure construction, including power, supply chains, semiconductor manufacturing equipment, and now possibly financing.

From the stock performance perspective, while the stock prices of related companies have risen significantly from previous lows, they are still well below their peaks. The market is sending a signal: investors are concerned about the speed and sustainability of infrastructure construction, and they worry that industry growth may be nearing its peak. Financing is clearly one of the important issues, and it may even be one of the most critical.

You mentioned earlier that hyperscale cloud providers were once among the most cash-generating companies in human history, but now they are consuming cash at a rapid rate and may need to issue stock for financing in the future. Meta recently adjusted its equity structure, partly to create space for such financing arrangements. Companies like Google have also engaged in financing.

Take CoreWeave as an example; its bond yield is close to 15%. It is important to note that this does not represent the actual financing cost of all the company's current debt, but rather the trading yield of its unsecured bonds in the market. However, if the company were to re-enter the market today to issue unsecured bonds, the financing cost would likely be around this level. This is quite a high cost.

However, I believe that at least for the next year or two, the industry can generally maintain operations. More importantly, related revenues must continue to grow to gradually alleviate market concerns about financing.

For instance, in the computing power leasing market, the current supply is extremely tight, and the common saying we hear is that no one can obtain enough computing power. Some companies are leasing computing power through short-term contracts, and according to certain estimates, the revenue per gigawatt of computing power could reach around $50 billion.

Therefore, it is not hard to imagine a scenario where, even with high construction costs, the related revenues are still sufficient to cover costs, or even exceed them. But the question is, how long can this level of revenue be sustained? Ultimately, it depends on how strong demand is relative to supply and how long the supply-demand gap can be maintained.

These questions may require Dylan to provide a deeper judgment from the perspective of industry supply and demand. However, financing is undoubtedly one of the many limiting factors currently.

Another point worth noting is that limiting factors slowing down growth may not necessarily be a bad thing. It may prevent the industry from over-expanding and avoid uncontrolled investments.

A similar situation occurred during the internet bubble. At that time, a large amount of dark fiber was laid, which refers to fiber that has been installed but not yet put into use; many infrastructures were ultimately overbuilt, and some are still not fully utilized today.

Today's construction constraints may actually lead to a more reasonable supply-demand match, allowing industry growth to continue for a longer time. If there were to be unrestricted and reckless expansion today, the future could face a more severe risk of collapse.

So, I do not believe that all limiting factors are bad. Of course, how the market views these constraints is another matter, and they have clearly affected the short-term performance of related stocks.

Dylan Patel:

I agree with Stacy's point. Looking back at infrastructure construction over the past few decades, every time the industry expands, companies often repeat orders, sometimes doubling or tripling them, ultimately resulting in excessive capacity. The supply chain's ability to forecast demand has always been inadequate.

Now, financing is playing a certain self-regulating role. A year or two ago, as long as a cloud service provider was willing to pay a sufficiently high interest rate, they could usually obtain funding; but now, without a long-term procurement commitment of sufficiently high credit quality (offtake agreement), some emerging cloud service providers cannot build computing power clusters.

Hyperscale cloud providers can certainly push their free cash flow down to nearly zero and continue to take on debt, and their profits are still growing, so their construction capacity will also increase. But they cannot invest indefinitely. Next year, they cannot raise capital expenditures to over $2 trillion, or even $2.5 trillion. Ultimately, funding capacity will limit the scale of expenditures.

Looking at emerging cloud service providers (Neocloud), the construction demand they undertake is close to half of the entire market. If they do not have procurement commitments from hyperscale cloud providers, how can they obtain financing?

Today, even if they obtain procurement commitments from clients like Anthropic or OpenAI, the market may require related debt to provide yields of 13% to 15%. Volulta is an example: the company is building a computing power cluster for Anthropic, with a bond coupon rate of about 12%, but the issuance price is only 95% of par, so the actual financing yield is close to 15%.

If a company does not have procurement commitments from hyperscale cloud providers or large AI labs, how can it build computing power to serve small and medium-sized enterprises and startups? Frankly, without the support of these large clients, financing will be very difficult.

This is also one of the strategies Elon Musk has adopted: first build a large amount of computing power, and then lease it at a high price to startups and AI labs through short-term rental contracts. But overall, financing constraints will indeed suppress industry expansion.

NVIDIA has already recognized that this is a significant issue. There are only a few large clients in the market who can purchase as much computing power as their funds allow; but there is another half of the market demand—how to meet these clients?

As a result, NVIDIA has begun to adopt some innovative financing structures. For example, in projects like Sharon AI and Fermis, NVIDIA supports project financing by sharing computing power cluster revenues. The arrangement is roughly as follows: NVIDIA rents the cluster at a price that guarantees the cloud service provider makes little to no profit, and if the cloud service provider rents the computing power to other clients at a higher price, NVIDIA shares a portion of the revenue.

Similar arrangements are increasing. When Lambda builds computing power clusters for Anthropic, it also needs to sign a revenue-sharing agreement with NVIDIA, with a sharing ratio of about 10%.

This indicates that whether emerging cloud service providers can obtain sufficient capital, complete construction, and ultimately even expand to the point of oversupply is constrained by financing conditions.

We also see that the industry's financing methods are shifting from long-term loans and long-term debt to more flexible arrangements. When Broadcom provided chips to Anthropic, it used a leasing-to-purchase approach, extending the payment period to several years.

For companies that cannot rely on their balance sheets for financing like hyperscale cloud providers, obtaining computing power requires support or credit guarantees from companies like NVIDIA, Broadcom, or AMD for the clusters.

Therefore, financing is an important factor limiting the expansion of AI infrastructure. Hyperscale cloud providers themselves also have large on-balance-sheet and off-balance-sheet commitments. For example, Meta's commitments in computing and data centers are reportedly as high as $1.4 trillion.

These projects can obtain funding in various ways: companies can borrow to purchase data centers, or lease data centers or related equipment from other companies, and the party providing the infrastructure can then finance based on this. But making the entire industry continue to expand significantly on this basis is becoming increasingly difficult.

The capital markets have even begun to feel uneasy about Meta's financial arrangements. Meta's own financial condition does not mean it cannot bear these investments, but such a large scale of commitments is enough to raise market vigilance.

Ultimately, the construction of AI infrastructure has three interrelated core conditions: land, power, and shell facilities, chips, and financing. Any constraint on one link may slow down overall expansion.

### 2. Where Are the Bottlenecks in the Semiconductor Supply Chain in 2027-2028?

Host Austin Lyons:

The situation regarding financing is already quite clear: hyperscale cloud providers are relatively capable of bearing investments, but emerging cloud service providers face greater pressure, and the industry is trying new financing structures.

If we temporarily set financing aside and return to traditional supply chain bottlenecks, which links will limit industry growth in the next two years?

ASML has stated that capacity for 2028 is already sold out; TSMC's demand has consistently exceeded supply, even as it expands its 2nm process capacity. The market also generally believes that power is an important bottleneck.

Dylan, you recently mentioned that nearly 50 gigawatts of data centers may come online next year. Can you rank the various links in the semiconductor supply chain according to the degree of constraints?

Dylan Patel:

Within the semiconductor supply chain, apart from memory, I believe the biggest bottleneck is substrates, including packaging substrates, PCB materials, and other related materials.

These links have historically seen insufficient investment. Many related material suppliers in Japan have a conservative business style and are unwilling to expand production on a large scale, so in the next two years, whether it is AI accelerators, CPUs, or other components in the supply chain, there will be significant limitations.

In my view, substrates are currently one of the most severe bottlenecks, with constraints even exceeding those of land, power, data center shell facilities, and some semiconductor manufacturing equipment.

The second biggest bottleneck is memory.

The third category of bottlenecks is optical interconnects, including Co-Packaged Optics (CPO) and Near-Package Optics (NPO). It is not easy for the related industry to significantly increase production. Even if all companies hope to adopt CPO by 2028, or if a large enterprise hopes to deploy it comprehensively at that time, they will not be able to obtain enough lasers, fiber optic connection components, and other supporting parts.

As for land, power, and data center shell facilities, power is indeed an important constraint. However, our research has found that the market may be overestimating the constraints faced by data center construction next year.

It is expected that nearly 50 gigawatts of data centers will be under construction globally next year, with about 35 gigawatts in the United States and the rest distributed in regions like Australia, India, and Malaysia. In fact, data centers are being rapidly constructed all over the world.

Although power is still tight, companies can take various measures to alleviate constraints, such as deploying gas turbines, reciprocating engines, or fuel cells on-site at data centers.

Therefore, while there are indeed constraints in power and data center construction conditions, their impact may be less than that of substrates, memory, lasers, and other optical supply chain components.

Power supply can be increased in a relatively direct manner in a short time, while the manufacturing processes and material systems involved in the semiconductor supply chain are much more complex, and the time required for capacity expansion is longer.

Stacy Rasgon:

This judgment is very interesting. NVIDIA and Broadcom have both provided quite high growth guidance. Jensen Huang has stated that NVIDIA's revenue is expected to grow by at least 70%; Broadcom's growth expectation even reaches 100%. However, these guidelines themselves are already constrained, and the main constraints considered by companies seem to be land, electricity, and data center construction conditions.

Perhaps this is because they are semiconductor companies themselves and have a clearer understanding of how many chips can be produced, thus focusing more on data center construction.

I agree that there are some solutions regarding electricity and land. However, even on-site power generation may not be realized quickly. How long does it actually take to order a gas turbine today? Elon Musk has even stated that he is prepared to manufacture turbine blades himself to bypass supply constraints.

The memory market is particularly noteworthy. Memory stocks currently have very low valuations, with many stocks having forward price-to-earnings ratios of only two to three times or three to four times, while memory prices have already risen significantly.

I am not directly responsible for memory stock research, but taking SanDisk as an example, its quarterly earnings per share guidance is even higher than the company's stock price about two years ago. This indicates that the current price increase is very substantial.

However, memory stocks generally only have single-digit price-to-earnings ratios because the market usually prices stocks this way when the industry cycle approaches its peak. Investors are concerned that especially after entering 2028, when new supply begins to be released, memory prices will drop significantly.

I have been thinking: what if this situation does not occur?

Looking inside AI racks, about 70% to 80% of the silicon area is actually memory, mainly DRAM. High Bandwidth Memory (HBM) has a special issue: producing the same capacity of HBM requires about three to four times the number of wafers compared to ordinary DDR5 DRAM.

This means that HBM production will occupy a large amount of wafer resources, squeezing the capacity that could originally be used to produce ordinary memory needed for PCs, smartphones, and other products.

### III. Memory and Manufacturing Equipment: Shortages May Last Until 2028

Stacy Rasgon:

The demand for HBM is very high, which also leads to another bottleneck: semiconductor manufacturing equipment, which is the equipment used to produce these chips.

This week, a company released a preliminary outlook for the wafer fabrication equipment (WFE) market for next year, expecting the market size to exceed $200 billion. I find this prediction quite surprising because I believe it is difficult for the market size to reach a higher level, and even whether this number can be achieved is questionable. The reason is simple: we do not have enough fabs.

Clean rooms are almost fully occupied. Companies have been trying to maximize the utilization of existing facilities, which is why the market's estimate for WFE size at the beginning of the year was about $120 billion, while it may ultimately reach $155 billion this year. The growth mainly comes from better utilization of existing facilities rather than large-scale new construction.

However, there is ultimately a limit to the utilization of existing facilities. Companies must build new clean rooms, which takes several years. New clean rooms will start to be used in small quantities in 2027 and will further increase in 2028 and 2029. Only then can we better assess whether supply and demand will become imbalanced.

If demand continues to grow at the rate currently expected by the market, memory may still remain in a state of tight supply for a long time.

Substrates, PCBs, and other materials also face similar issues. Equipment suppliers have stated that short-term supply constraints have affected their ability to increase production. The constraints are not limited to PCBs and semiconductor chips but also include seemingly ordinary components like valves and mass flow controllers. The shortage of these basic components will also impact other links in the supply chain.

During the pandemic, semiconductor equipment companies and most companies in the supply chain have learned some lessons, and their management of supply issues is now more mature than in the past. However, many links are still constrained, and these issues must be resolved first before the capacity constraints in other links can be alleviated.

Therefore, the supply chain actually has layer upon layer of constraints. I believe the shortage situation will not disappear in the short term. This year the issues are not significant, but next year may still be tight, and 2028 is unlikely to be relaxed. As for 2029, it will depend on the demand situation at that time. Overall, I expect the supply tightness to continue for a while.

### IV. The Pricing Power in the Semiconductor Industry is Changing

Host Austin Lyons:

If supply remains tight in the coming years, and the profit margins in the memory industry have reached over 80%, even 88% to 90%, will companies across the entire supply chain continue to raise prices and obtain higher profits?

Stacy Rasgon:

The semiconductor industry has an interesting characteristic: in the past, people almost took continuous price declines for granted. In fact, the semiconductor industry has historically been a sector that severely destroyed economic value, unable to cover capital costs for a long time, with most of the value created flowing to consumers.

Many years ago, when I worked at McKinsey, I participated in a study on value creation in the semiconductor industry. We used the classic definition of economic profit, comparing the return on invested capital of companies with the weighted average cost of capital, and summarized the entire industry. The study covered about 150 companies, 12 segments, and a complete industry cycle.

The initial time frame of the study was approximately from 1996 to 2004. The results showed that the entire industry cumulatively destroyed about $5 billion in economic profit. If Intel is excluded, the destruction scale reaches about $50 billion. At that time, this was quite an astonishing number.

In other words, the value created by the industry was not enough to cover capital costs, and economic value actually flowed to consumers. That was a time when purchasing semiconductor products was very cost-effective.

After 2008, the industry underwent consolidation and concentration, and companies began to gradually improve profitability, at least slowly being able to cover capital costs.

The pandemic changed the industry's perception of its own pricing power. Companies found that they actually had strong pricing power. Previously, the industry did not truly realize this.

During the pandemic, the costs of inputs like raw materials continued to rise, but many companies I tracked did not see a decline in gross margins due to rising costs. They all raised prices, and the price increases were sufficient to offset the rising costs.

For example, if a company's original gross margin was 60%, and costs increased by 100 units, it could raise prices by 160 units to maintain its original gross margin. Customers also accepted these price increases.

I believe that the inflation during the pandemic actually helped the semiconductor industry improve its profitability. In 2023 and 2024, the industry's shipment volume is roughly flat compared to 2019, but the overall average selling price (ASP) is about 50% higher, even excluding memory.

Now, this pricing power is more evident. It is expected that the total revenue of the semiconductor industry will reach about $1.5 trillion this year, or even higher, with a year-on-year growth rate close to 100%. About 70% to 80% of the revenue increase comes from the rise in average selling prices of memory, with a significant portion also coming from price increases in non-memory products.

In contrast, the growth rate of shipment volume is much smaller. The industry is making fuller use of its pricing power, and customers generally accept these prices because they often have little choice in the face of tight supply.

Therefore, as long as supply remains tight, I believe this situation may continue.

Dylan Patel:

In the semiconductor industry, different companies use pricing power in different ways. Some companies significantly raise prices during booms but suffer severe blows during industry downturns; others can maintain profit margins relatively stably throughout the cycle.

Taking TSMC as an example, over the past decade, its gross margin has steadily increased from about 45% to around 60%. During booms, it can indeed raise prices, but during industry downturns, it does not significantly give up the profits it previously gained.

The memory industry is different. Just about two years ago, the industry was in one of the worst downturns since the internet bubble, with gross margins even being negative. Now, however, memory manufacturers' profits have significantly improved.

Therefore, the pricing power of different links in the supply chain depends on their respective market structures. TSMC has a near-monopoly position and maintains long-term cooperative relationships with customers. Even if it has the ability to raise prices, it has not significantly increased prices like some other companies.

The situation for memory manufacturers is different. If customers like Apple exert significant pressure on suppliers during industry downturns, then when the cycle reverses and supply is insufficient, suppliers will also raise prices. What we are seeing now is this cyclical change in bargaining power.

### V. The Cost of Computing Power Construction Soars, Who Will Bear the High Costs?

Dylan Patel:

In 2024, the total cost of building 1 gigawatt of computing power is about $40 billion, covering various aspects such as chips, land, electricity, and data center facilities. Now, this cost has exceeded $60 billion.

Of course, there are not many companies that will actually build 1 gigawatt of computing power in 2024, but by 2026, more and more companies will begin to build such large-scale computing facilities. Meanwhile, the computing power that can be provided per gigawatt is also increasing.

But the problem is that such a gigawatt-scale computing cluster may have very considerable profitability.

Previously, I mentioned that cloud service providers often need to obtain long-term procurement commitments from ultra-large cloud vendors to finance the cluster. This means they need to lock in prices in advance, which also limits their ability to obtain excess profits.

Elon Musk has taken another strategy: if chips, land, electricity, and data center facilities are all constrained, then he can first build the infrastructure and sign lease contracts only when it is truly put into use. This way, when computing power supply is very tight, he has the opportunity to rent it out at a higher price without having to lock in lower prices in advance.

At the same time, AI labs themselves can also generate huge revenues from inference services. According to relevant disclosures, the revenue scale corresponding to each gigawatt of inference computing power for Anthropic could reach $70 billion to $80 billion.

However, these revenues do not mean that profit margins can remain high indefinitely. If a company obtains computing power at a cost of $15 billion per gigawatt, the profit margin may reach 75% to 80%; if the cost rises to $50 billion, the profit margin will drop significantly.

Therefore, who will bear these costs remains a question.

NVIDIA must also face this pressure. Although its profit margins have generally remained stable, even slightly declining, it has to absorb some of the costs from rising memory prices as well as other supply chain links like substrates and optical components.

The result is that the prices NVIDIA charges customers have risen significantly, but its profit margins have not increased proportionately.

This makes the distribution of profits in the supply chain particularly important: who can maintain profit margins, who can further expand profits, and who has to bear the pressure of rising costs? These questions are changing the industry's pricing landscape in real-time.

In the future, customers may sign similar contracts at different prices. Companies with strong sales capabilities that can seize opportunities in tight supply may obtain very high prices.

Host Austin Lyons:

This is similar to what Stacy just mentioned about the changes during the pandemic: the semiconductor industry is gradually realizing that it can charge for the value it creates.

Dylan, you also mentioned that system suppliers like NVIDIA must decide whether to pass on rising costs to customers or absorb the costs themselves to maintain profit margins when the prices of various components are rising.

The issue is that infrastructure construction will obviously continue in the coming years, with limited supply and strong demand, but financing is becoming increasingly difficult. So, will large cloud vendors and model companies like Meta one day think that computing power has become too expensive to bear and start resisting further price increases?

Stacy Rasgon:

When evaluating these investments, one cannot only look at costs and profit margins but also at what the company gains after spending money.

Taking NVIDIA as an example, the price of each generation of products is rising, but the performance improvements that customers receive are often greater. This is very important.

From Ampere to Hopper, the average selling price of GPUs has roughly doubled, but performance has increased by about 10 times. From Hopper to Blackwell, the situation is harder to compare directly due to significant architectural differences between the two generations. However, it can be roughly estimated that the average selling price of a single GPU has risen by about 50%, while performance has improved by 3 to 30 times. Considering subsequent software optimizations, the actual improvement may be even greater.

From Blackwell to Rubin, the rack cost may rise significantly again. According to reference data provided by NVIDIA, the rack cost for Blackwell is about $25 billion, while for Rubin it is about $40 billion, but the performance improvement is about 10 times.

Therefore, the key is not whether the construction cost per gigawatt of computing power rises from $25 billion to $40 billion, or from $40 billion to $60 billion, but rather how many Tokens and computing capabilities customers can obtain for every dollar spent.

As long as the performance gained per unit cost continues to improve, such investments remain attractive.

However, if NVIDIA's next-generation products only see a 20% performance increase, I would start to worry significantly about its pricing power and profit margins.

NVIDIA's gross margin exceeds 70%, and it is built on a massive revenue scale. The market has been asking: how does it continue to maintain such high profit margins in the face of competitors and other GPU suppliers?

My view is that as long as NVIDIA can continue to ensure that the performance increase significantly exceeds the average selling price increase, there is no need to overly worry about profit margins. Because as product complexity rises and costs increase, prices must also go up; the key is that the relationship between performance and cost must always be favorable to customers.

If the pace of innovation slows down, I would be very concerned.

I remember attending the Hot Chips conference around 2023. At that time, NVIDIA's Chief Scientist Bill Dally gave a keynote speech, and one point left a deep impression on me: over the past decade, GPU performance has increased by about 1000 times. NVIDIA achieved this progress through various technological means, and Dally stated that the goal for the next decade is to increase performance by another million times.

If NVIDIA can continue along this trajectory, I wouldn't be as worried about its profit margins. But if it cannot maintain the pace of innovation, or if other competitors catch up quickly, the situation would be completely different.

Ultimately, someone has to bear these costs. Ideally, customers are willing to pay for the value they receive. But if costs rise at an unsustainable rate, suppliers will have to start bearing part of it, and the competition for profits among different segments of the industry chain will become more intense.

### 6. The Rise of Self-Developed Chips: How Long Can NVIDIA's Competitive Advantage Last?

Host Austin Lyons:

Dylan, more and more companies are starting to design their own chips. The case of OpenAI is particularly noteworthy. How should we view the development of GPUs and the rapid growth of various XPUs and AI accelerators in the next two years? Many startups are trying different chip architectures; how will this change the competitive landscape?

Dylan Patel:

More and more companies are trying to develop their own chips, which is a clear trend. OpenAI has demonstrated an important fact: a company can produce very high-performance products even when designing chips for the first time.

OpenAI once invited us into their lab. We used SemiAnalysis's open-source inference benchmark InferenceX to test various open-source models, and the results showed that their first-generation chip outperformed NVIDIA's Blackwell.

This chip made extensive use of AI tools during the design and programming process. Anthropic is also advancing its own chip project and hopes to complete development in half the time of OpenAI. These goals are quite ambitious, but if the model can significantly accelerate the design process, it is not entirely impossible.

Meanwhile, Google has begun selling TPUs, and Amazon's Trainium has also matured significantly and supports PyTorch well. Meta and Microsoft also have very proactive chip roadmaps.

There is a strong economic incentive behind this. As mentioned earlier, the cost of building 1 gigawatt of computing power has risen from $40 billion to over $60 billion. If the profits of chip design companies like NVIDIA, Broadcom, and Marvell could be eliminated, the cost for the same scale of computing power might only be around $20 billion.

In other words, if the profit costs of chip design can be significantly reduced, theoretically, the funds saved from one gigawatt of computing power could support the construction of more computing power. This economic benefit is very clear for companies that need to run a large number of inference tasks.

Therefore, NVIDIA must maintain a very fast pace of innovation. It cannot just achieve a doubling of performance per dollar each year; otherwise, other competitors may quickly catch up.

If NVIDIA can continue to maintain this pace of performance improvement, it certainly has reason to continue earning high profits. But if large customers can design chips themselves, deploy them, and scale production while acquiring memory, TSMC wafers, substrates, and optical components at prices close to supply chain costs, then they have significant room for price reductions.

If the industry chain further disaggregates, and NVIDIA or Broadcom no longer hold the same dominant positions as before, the situation could change dramatically. This is also one of the reasons the market is concerned about the historically low valuation multiples of the two companies.

For example, will Meta reduce its reliance on Broadcom? Will Google do the same? Will these large customers eventually stop purchasing NVIDIA chips and completely shift to internal designs?

Currently, component suppliers themselves may also gain strong pricing power. The real question is whether NVIDIA and Broadcom can continue to maintain their astonishing profit margins. At least for the next two years, I believe they still have the opportunity to do so, but it will be hard to say after that.

Chip design itself remains very complex, but AI models are significantly lowering the difficulty of software development and accelerator programming.

For example, our InferenceX benchmark took about six months to get NVIDIA's inference system running; AMD took six to nine months; Google and Amazon only took a few months; while OpenAI's chip took about a month and a half.

The process from getting the chip to work to having software run on the chip is rapidly shortening. Now, companies are starting to focus their attention on chip design itself.

Will chip design gradually become a more widespread and standardized capability like software development? If the answer is yes, the number of companies capable of designing chips could increase significantly.

Of course, chip design still requires fixed costs such as mask and electronic design automation (EDA) tools, and just these investments could reach $50 million. But if AI continues to lower the barriers to design and programming, the number of companies participating in chip design may still increase.

Once chip design becomes more widespread, industry profit margins may decline. We have already seen some software-as-a-service (SaaS) companies face market revaluation; meanwhile, NVIDIA and Broadcom's profits are still growing at over double each year, but their stock prices have remained relatively stagnant.

This is one of the most concerning issues for the semiconductor industry in the future.

### 7. Can XPU Startups Challenge GPUs? NVIDIA's Market Share is Changing

Host Austin Lyons:

So, how will the GPU and XPU markets develop in the next two years? More and more AI accelerator startups are trying different architectures; will the competitive landscape see significant changes?

Dylan Patel:

For XPU companies, I see two different paths.

If you are a hyperscale cloud provider, you can take a relatively stable design route, creating products similar to those of Google or NVIDIA, and leverage your own scale to drive applications.

But if you are a startup, you must attempt very aggressive and distinctly different technological routes from existing solutions. Looking back at the first generation of AI hardware startups, one important reason for the relative success of Cerebras and Groq is that they took bolder technological paths.

For the emerging startups now, I believe only those with sufficiently unique technical solutions that cannot be easily replicated by NVIDIA, Google, or AMD truly have the opportunity to stand out.

The good news is that there is currently a large influx of capital into this field, and various radical architectures and design ideas are being tested.

Meanwhile, NVIDIA's share in the industry chain has already begun to change. From 2023 to 2024, NVIDIA occupied a significant proportion of HBM and advanced packaging capacity; by 2025, this growth began to slow, and in 2026 its share further declined. According to our judgment, this trend may continue into next year.

But this does not mean NVIDIA's profits will decline immediately. The entire market is growing too quickly; even if its share declines, NVIDIA's profits next year may still double.

Google, Amazon, and AMD are all gaining share. One cannot just look at the number of chips, as different chips vary greatly in size and value. More meaningful metrics include the usage of advanced packaging wafers, HBM, and expenditures on wafer foundry and memory.

These metrics indicate that competitors are gradually expanding their market share.

Looking ahead to 2028 and 2029, NVIDIA's growth rate may slow down, and it may not be able to maintain a 100% annual revenue growth rate. At that point, the market will need to assess whether the slowdown in growth will further lead to a decline in its share, or whether NVIDIA can continue to lead with new architectures after Rubin.

NVIDIA must continue to accelerate its pace of innovation. Its annual chip update rhythm is not only for its own development but also to make it difficult for competitors to catch up.

NVIDIA's moat is not just hardware. CUDA software, a large developer ecosystem, already deployed infrastructure, and its own balance sheet and financing capabilities all constitute competitive advantages.

NVIDIA is also leveraging AI to accelerate its chip design. It is not unaware of competitive pressures, nor has it failed to adopt new technologies. Ultimately, this is a race.

Jensen Huang is clearly highly vigilant about competition. He has learned some lessons from Intel's history and always pays attention to competitors' movements.

Stacy Rasgon:

This also explains why NVIDIA's valuation is at such a low level.

About nine to ten months ago, I wrote a research report comparing NVIDIA's relative valuation with that of the entire semiconductor industry. At that time, NVIDIA's relative valuation had only been lower than that for 13 trading days over the past decade.

And now, its relative valuation is even lower than it was then.

Investors are worried not only about whether AI demand can be sustained. If the entire AI demand disappears, both NVIDIA and other companies would be severely impacted.

But within the semiconductor industry, NVIDIA's biggest threat clearly comes from competition. It currently leads its peers by a wide margin, but all competitors want to capture its market.

This is also a consistent characteristic of the semiconductor industry: whenever a company succeeds, there will always be challengers. NVIDIA must ensure it remains ahead, and as the industry develops, this will only become more difficult.

### 8. Enterprise AI Applications: Will More Companies Buy Chips Themselves or Directly Purchase Tokens?

Host Austin Lyons:

Earlier, we discussed GPUs, XPUs, and competitors expanding their market shares. Our discussion mainly focused on data centers and hyperscale cloud providers.

So, for companies like NVIDIA, is local deployment of computing power also an important growth opportunity?

Stacy Rasgon:

NVIDIA now distinguishes between revenue from hyperscale cloud vendors and non-hyperscale cloud vendors, but this classification is not entirely accurate. Some AI labs may be categorized as non-hyperscale cloud vendors, while the spending of hyperscale cloud vendors may also indirectly support these labs, leading to overlap between the two.

Roughly speaking, the two types of customers currently account for about half each. The revenue contributed directly or indirectly by AI labs may account for about 20% of NVIDIA's total revenue. This is still a considerable proportion.

Therefore, NVIDIA's revenue sources may be more diversified than we previously imagined.

Ultimately, the adoption of AI by enterprises is key to the sustainable development of this industry. If enterprises are not genuinely using AI, the long-term growth of the entire industry lacks support.

We are still in the early stages, but I believe that at least one core application with clear value has emerged on the enterprise side: programming.

The revenue from AI programming agents is growing rapidly. Nowadays, almost all software developers are using AI-assisted programming, and very few rely entirely on manually writing code.

The value of enterprise applications may not necessarily be directly reflected in revenue growth; it may also manifest as increased productivity, which is harder to measure.

I myself am a financial analyst and need to build financial models. A few months ago, our team tried using Claude to complete this type of work. The initial results were not ideal, but later a colleague believed the tool had made significant progress.

So, we arranged a team meeting to have it build a financial model from scratch for a company we had already covered.

The results were surprising: it took only one hour and twenty minutes to complete a model. The model covered ten years of historical data, handled multiple business reorganizations and segment adjustments of the company, and attempted to make predictions based on the CEO's statements during the earnings call, rather than simply randomly generating numbers.

This task, if assigned to an analyst in the team, might take a week; if I were to do it myself, it could take three months. But Claude only took one hour and twenty minutes.

This means the same team can accomplish more work, even completing more tasks with fewer personnel.

Of course, this also brings other issues. If AI becomes widely adopted in enterprises, could it lead to a significant rise in unemployment? I do not know the answer, but it is indeed a question worth considering.

Based on current practical experiences, I am increasingly convinced that AI applications can bring real returns. However, this is still an early market. Many enterprises are still in the experimental stage, unclear on how to use AI, and lack sufficient expertise to assess specific needs, so widespread adoption will take time.

Overall, I hold a positive attitude toward the current situation.

Dylan Patel:

I would like to add one point: regarding enterprise AI applications, I do not believe most enterprises will purchase hardware in large quantities and run their own models internally.

There are significant economies of scale in model inference. For most enterprises, a more reasonable approach is to directly purchase tokens, that is, to call the inference services of cloud models, and then combine these capabilities with their own data.

I believe this will be the primary way most enterprises consume AI in the future.

This is very similar to the development of enterprise IT over the past twenty years. Twenty years ago, enterprises typically purchased infrastructure and deployed systems themselves; today, most IT spending has shifted to cloud computing. In the future, most enterprise spending on AI may also be used to purchase tokens, rather than buying computing power and running open-source models themselves.

Even pharmaceutical companies and hedge funds may primarily pay for direct use of models, rather than purchasing hardware to run open-source models themselves. Of course, there will still be some vendors in the market that specialize in purchasing computing power, deploying models, and providing services to customers.

There are many more topics worth exploring in this discussion, but we have exceeded our time. Today's exchange has been very interesting, and I appreciate everyone's participation.

Risk Warning and Disclaimer

The market has risks, and investment requires caution. This article does not constitute personal investment advice and does not take into account the specific investment goals, financial situation, or needs of individual users. Users should consider whether any opinions, views, or conclusions in this article align with their specific circumstances. Investing based on this is at one's own risk.