All podcasts / Invest Like the Best / Summary

Etched - Building AI Hardware to Make Inference Faster and Cheaper

2026-06-30 - 89 min - source - Read full transcript

Patrick O'Shaughnessy (host)Gavin UbertiRobert Wachen

Key insights

Etched treats the full rack - chip, boards, power delivery, interconnects, and manufacturing - as the product, not just the chip.

Gavin and Rob argue that 'production is the product': since inference is a manufacturing problem at scale, controlling every layer down to the cold plates and assembly line is what actually determines how many tokens a company can bring online, not chip performance alone.

vertical-integration

Etched's core technical bet is that GPU architectures never optimized for the actual physics constraints of inference-only workloads.

Because standard EDA tooling and chip sign-off assume general-purpose, worst-case conditions (like sub-freezing operating temperatures that AI data centers never hit), Etched found that relaxing those inherited constraints unlocked large compounding gains - each individually modest (20 percent here, 2x there) but additive across the full stack.

ai-inference-hardware

Low-voltage inference lets Etched pack more flops into the same silicon without thermal throttling.

Since power scales with the square of voltage, running chips at under half the voltage of a typical AI chip is the lever that lets them add more transistors and flops without hitting the thermal ceiling that forces GPUs to self-throttle at high utilization. They cite Bitcoin miners, which already run at roughly a quarter of GPU voltage, as proof the physics allows it.

ai-inference-hardware

Interconnect latency, not raw chip flops, is the binding constraint on decode throughput at scale.

On NVIDIA Blackwell chips, point-to-point chip communication takes about 4,000 nanoseconds, which means an 8x scale-up in chip count yields far less than an 8x gain in tokens per second per user. Etched built its own custom interconnect stack above layer two of Ethernet to cut this by more than 5x, treating the whole cluster's memory as one shared pool ('cluster-scale memory').

ai-inference-hardware

Etched's talent model pairs recognized industry veterans ('legends') with young, inexperienced but obsessive early hires.

They deliberately recruited a small number of people who had done the exact hardest version of a problem before - for example, Brian Loyler, who built NVIDIA's HGX and DGX rack systems - and paired them with founders in their early twenties who had no industry pedigree but extreme drive, arguing the combination outperforms either type alone.

talent-and-recruiting

Pre-fetching non-chip-dependent work compressed their chip-to-working-rack timeline to 40 days versus an industry example of 10 months.

Etched built and validated the entire rest of the product - software stack, racks shipped to customer data centers without chips installed, 700+ FPGAs emulating the full chip to run real inference workloads, and thermally accurate mock chips to validate cold plates - all before their actual silicon came back from the fab, so almost nothing was left to build once it arrived.

vertical-integration

Etched deliberately chose hand-written kernels over a general graph compiler, betting the number of architectures that matter would stay small.

Three years ago they decided not to support arbitrary PyTorch, CUDA, or ONNX graphs, betting instead that fewer than 100 model architectures would matter and that hand-optimized primitives for those would outperform a flexible compiler; they now expect AI models themselves to increasingly write and tune these kernels.

ai-inference-hardware

Etched intentionally avoids vertically integrating into data centers or model building, drawing the line where economies of scale actually exist.

They only vertically integrate where there is a large economies-of-scale effect (chip design, fabrication, rack production) and stay out of parts of the stack - like building their own data centers - where doing so would not add more token capacity to the world, even though their hardware advantage could tempt them to expand.

vertical-integration

A company built around one existential product out-competes internal chip projects at larger companies, because the larger companies do not need the chip to succeed.

The guests argue NVIDIA remains the best standalone chip company precisely because chips are its only product, whereas Google's TPU, Meta's MTIA, Microsoft's internal chip effort, and OpenAI's chip project are all survivable failures for their parent companies - so those teams cannot match the intensity, talent draw, or supplier partnerships of a company where the chip is do-or-die.

semiconductor-economics

Etched deliberately built its first-generation chip on a different process node and memory supply chain than NVIDIA's next generation to avoid competing for the same scarce capacity.

Their gen-1 chip uses 4-nanometer fabrication and a different HBM supplier than NVIDIA's Rubin (3-nanometer), which they frame as a positive-sum choice: customers deploying at scale aren't choosing between a gigawatt of NVIDIA or a gigawatt of Etched, they can get both, since the two products don't draw on the same limited wafer and memory allocations.

semiconductor-economics

Etched's Series A nearly failed to close; every major VC passed before a roughly $103 million round assembled from smaller checks and supplier goodwill.

After a 30-page technical memo failed to convince top-tier investors (who cited the founders' age, lack of a taped-out chip, and skepticism that inference would be a large market), the founders exhausted their personal networks for smaller checks while TSMC and Synopsys extended favorable, loan-like payment terms before any of that capital had closed, effectively functioning as early believers alongside the eventual investors.

founder-and-startup-resilience

A clock-domain-crossing bug that required aligning two signals to within 50 picoseconds, two billion times per second, was treated as solvable by assumption before it was solved in practice.

When their taped-out chip produced wrong results due to a back-pressure logic failure across a clock domain boundary, some team members quit believing it unsolvable; the founders' standard approach - assume a solution exists, then work backward from what it would require - led to a drift-and-lock clock-phase mechanism that fixed it within about two scary weeks.

founder-and-startup-resilience

Companies

Etched - The company being profiled - builds vertically integrated AI inference chips and full server racks.
NVIDIA - Reference chip maker throughout - Blackwell latency and flop density used as the benchmark Etched compares itself against.
TSMC - Etched's foundry partner; praised for collaborative customer service and for extending favorable terms before Etched had raised its Series A.
Cypress Semiconductor - Former employer of Etched's CEO Mark Ross, who joined as an early advisor and skeptic-turned-believer; Cypress sold for about $9 billion.
Synopsys - EDA/emulator vendor that let Etched pay for tooling on deferred, loan-like terms while the company was still pre-revenue and unproven.
Google - Cited as an example of a chip project (TPU) that is not existential for its parent company, unlike Etched's single-product focus.
Meta - Cited as an example of a chip project (MTIA) that is not existential for its parent company.
Microsoft - Cited as an example of a chip project (internally referred to by the guests as Maya) that is not existential for its parent company.
OpenAI - Cited as pursuing its own custom AI chip project, framed by the guests as another example of a non-existential in-house chip effort.
xAI - Referenced via its Colossus training cluster (100,000+ networked GPUs) as the current scale of training infrastructure, contrasted with the smaller scale-up domains still typical in inference.
Positive Sum - Patrick O'Shaughnessy's investment firm; he discloses being a multi-round Etched investor and asks the guests to treat him as biased.

Techniques and frameworks

Low-voltage inference - Etched's custom power-delivery mechanism that runs chips at under half the voltage of other AI chips, exploiting the fact that power scales with the square of voltage - something GPU architectures were never designed to exploit because they weren't purpose-built for inference.
Cluster-scale memory - Treating the full-cluster interconnect, not just on-chip memory, as the addressable memory pool; achieved via a custom interconnect stack built above layer two of Ethernet to cut chip-to-chip latency by more than 5x versus stock NVIDIA interconnects.
PD (pre-fill/decode) disaggregation - Running pre-fill and decode on physically separate clusters of servers, transferring the KV cache between them, since the two phases have different bottlenecks (flops vs. memory bandwidth).
Project-based recruiting - Mapping every hard technical problem to the specific person who has solved something like it before, then repeatedly re-approaching them - most people say no on the first ask but yes by the twentieth conversation.
Pre-fetching the schedule - Parallelizing every piece of work that does not require the chip itself - software stack, rack assembly, cold plates, production line - so the full product is ready the moment silicon returns from the fab, cutting chip-to-working-rack time to 40 days versus an industry example of 10 months.
Kernel-first software strategy - Betting on hand-optimized kernels for a bounded set of expected model architectures instead of a general-purpose graph compiler, trading early flexibility for peak performance and later leaning on AI models themselves to help write kernels.

Summary

Patrick O'Shaughnessy sits down with Etched co-founders Gavin Uberti and Robert Wachen roughly three years after his first conversation with Uberti, when Etched was two Harvard dropouts with an idea that "everyone" told them was impossible: beating NVIDIA at inference on a chip built by people in their early twenties. In the years since, Etched has taped out a working chip on its first attempt, raised over $800 million, and shipped Sohu, a complete inference rack, not just a chip, built around two core technical bets: low-voltage inference, which packs far more flops into the same silicon by running at under half the voltage of typical AI chips, and cluster-scale memory, a custom interconnect stack that treats an entire cluster's bandwidth as one shared memory pool rather than optimizing chip-to-chip latency as an afterthought.

The conversation traces both founders' personal paths into the company: Wachen's stage-four bone cancer diagnosis in high school and the moment GPT-4's vision model correctly flagged the tumor on his back from an old photo, which convinced him the world lacked the infrastructure to serve AI at scale; and Uberti's early kernel-engineering career (X-Nor, acquired by Apple; Octo AI, acquired by NVIDIA) and a competitive robotics background that shaped Etched's "win by shipping" culture. They describe a deliberately narrow talent strategy - "legends" like ex-NVIDIA rack architect Brian Loyler paired with young, inexperienced, obsessively driven early hires - sourced through "project-based recruiting," which maps the hardest problems in the world to the specific people who solved something similar before, then re-approaches them repeatedly until a no becomes a yes.

Much of the episode is a case study in vertical integration and velocity under extreme capital and schedule constraints. The founders describe relocating a dozen engineers to Bangalore for six months to unblock a stalled vendor, running 24-hour day/night shift handoffs, and "pre-fetching" every part of the product that didn't require the actual chip - software stack, racks shipped chip-less to customer data centers, 700-plus FPGAs emulating the full design, thermally accurate mock chips for cold-plate validation - so that when silicon finally returned from the fab, the company went from chip-back to working rack in 40 days, versus a competitor's publicly reported 10 months. They also recount a near-catastrophic clock-domain-crossing bug that required synchronizing two signals to within 50 picoseconds two billion times per second, and a Series A that nearly didn't happen: every major VC passed on a 30-page technical memo before the founders assembled roughly $103 million from smaller checks and favorable terms extended by TSMC and Synopsys while the company was still unproven.

The back half turns toward market structure and long-run theses. Uberti and Wachen argue vertical integration should stop wherever economies of scale stop - they build their own chip, boards, and rack because that determines token throughput, but avoid building data centers or models because it wouldn't add capacity. They contrast Etched's single-product focus with in-house chip efforts at Google, Meta, Microsoft, and OpenAI, arguing those teams can't match the intensity of a company for which the chip is existential, which is also why NVIDIA remains the best standalone chip maker. On architecture, they lay out a "machines don't think like people" thesis: because moving data is expensive and math is cheap for silicon (the reverse of biological neurons), future models should lean into using far larger, more dynamic amounts of compute per token rather than mimicking brain-like memory efficiency. They close on a decade-plus thesis that inference will become one of the largest markets in the world, framed around metrics like "agents per megawatt," and a prediction that agents could outnumber humans doing knowledge work as soon as 2027.

Notable Quotes

"The best availability is availability. If I have a thousand chips today, someone's going to use them." - Robert Wachen

"You kind of have to be sick in the head to join our company." - Robert Wachen

"It is completely unsurprising that the best chip in the world is built by a company that only builds that chip." - Gavin Uberti

"Everything is solvable and we're just going to work at it until we figure it out." - Robert Wachen

"If you invested on base rates, you should do something other than what we and I do. There's always the index fund." - Patrick O'Shaughnessy