SAP: Bringing the 'Operating System' of a Company into the AI Era with CTO Philipp Herzig

Key insights

Media referenced

• No Priors episode with Andrej Karpathy - podcast - Herzig references a prior No Priors episode where the guest (Andrej Karpathy) argued developers should write evals first; he ties this to how SAP is rethinking test-driven development for agentic coding.

Companies

• SAP - Herzig's employer; enterprise software vendor with 400,000 customers running finance, HR, supply chain, and customer-facing processes end to end.
• Concur - SAP's travel and expense product, cited as an early example of consumptive/outcome-based pricing and live travel-booking and expense agents.
• Fieldglass - Cited alongside Concur as one of SAP's few products that were not seat-based licensed even before the AI-driven pricing shift.
• Anthropic - Cited with OpenAI as building the coding models that made agentic coding reliable because code has a built-in verification signal (compiles, passes tests).
• OpenAI - Same context as Anthropic - credited for the code-generation models behind agentic coding's productivity gains.
• Sierra - Referenced as an example of a company already selling outcome-based (rather than seat-based) AI pricing, which SAP is moving toward.
• LiteLLM - Cited as a recent example (roughly two weeks before the taping) of a security vulnerability that leaked API keys and credentials, used to argue why enterprises can't just deploy open-source AI tooling without hardening it first.

Techniques and frameworks

• Generative UI - SAP's term for dynamically generated interfaces that replace fixed click-through UIs, driven by the user's question rather than a pre-designed screen.
• Agent mining - The evolution of process mining: recording every decision trace and clarifying question an agent asks a human, to detect anomalies or promote a local practice into a new standard operating procedure.
• RPT-1 (Relational Pre-trained Transformers) - SAP's transformer-based architecture, published at NeurIPS and other venues after two years of research, built to make predictive/tabular forecasting (classification, regression, time series) scale the way LLMs scaled unstructured-text tasks.
• Evals (e-vals) as agent harness boundary conditions - Herzig's framing for why enterprise agents are harder to make reliable than coding agents: you must define the correct output and boundary conditions (security, privacy, code quality) up front so the agent's actions can be verified against them, since finance/HR domains lack code's built-in compile/test signal.

Summary

Sarah Guo interviews Philipp Herzig, CTO of SAP, about how a 50-year-old enterprise software company is re-architecting itself for the AI era while trying to avoid the fate of prior technology transitions that unseated incumbents. Herzig's throughline is that SAP has survived mainframe, client-server, internet, mobile, and now AI shifts by staying obsessively focused on customer outcomes rather than the underlying technology, and that AI is forcing a rebuild on three layers simultaneously: the UI (moving to dynamic, "generative UI" that responds to a user's question instead of a fixed click path), the business process layer (agents blending rigid standard operating procedures with unstructured judgment), and the data layer (a harmonized semantic model across SAP's traditionally siloed tables of financial, HR, and supply-chain data).

A central theme is that scale, not raw AI capability, is SAP's real engineering problem. A retrieval chatbot over ten documents or a single MCP server is trivial to demo; doing the same thing reliably across thousands of documents, localized policy answers, and roughly 20,000 internal APIs turns into a hard orchestration and disambiguation problem. Herzig contrasts this with why agentic coding took off so fast: code has a built-in verification signal (it compiles, tests pass), while enterprise domains like finance don't. His answer is that SAP has to build explicit "evals" and boundary conditions up front, and to capture "tribal knowledge" - the decisions living in people's heads or Slack threads that never make it into the system of record - through what SAP now calls "agent mining," a descendant of process mining that turns every human clarification into training data and, eventually, updated standard operating procedures.

Herzig spends significant time on a contrarian point: large language models are the wrong tool for predictive, tabular problems like demand forecasting or cash-flow prediction, because next-token generation isn't suited to regression and classification. Classical machine learning (XGBoost, AutoML) works but doesn't scale - one pharmaceutical customer operating in 90 countries needed 180 separately trained models. SAP's response, after two years of research published at NeurIPS and elsewhere, is RPT-1 (Relational Pre-trained Transformers), an attempt to bring LLM-style democratization to structured data prediction.

On the business side, Herzig confirms AI is forcing a pricing model transition for SAP, from seat-based licensing toward consumption-based and eventually outcome-based pricing (citing Sierra as an example of the latter), but says customers are pulling the company toward a hybrid model because they want cost predictability and don't yet fully trust agent outputs. He frames enterprise AI adoption more broadly through an "innovation race versus outcome race" lens - the pace of new AI capability is outrunning customers' ability to actually capture value from it - and ranks fragmented/siloed data (often the legacy of purchasing decisions or M&A) as the top blocker, ahead of scale complexity and security concerns like the recent LiteLLM credential-leak vulnerability. He predicts most enterprise agents will interact with systems via structured tool-calling and APIs rather than browser-based computer use, and that knowledge workers will be "uplevelled" from doing tasks to supervising agents, similar to how junior developers moved from writing code to reviewing agent-generated code.

The conversation closes on a personal note: Herzig describes his day as reviewing team progress across the stack, prototyping with multiple command-line agent instances, and working closely with customers to stay grounded in real problems rather than pitching technology. He flags SAP's early-stage quantum computing research as a long-horizon, hardware-agnostic bet aimed squarely at classic hard optimization problems - route planning, logistics, traveling-salesman-style problems - motivated by concrete cost and emissions savings rather than any near-term AI application.

Notable Quotes

"It's actually not the AI so much, but it's actually teaching the AI to do the right thing at scale." - Philipp Herzig

"I always tell my developers all the time, our job at SAP is to make the technology disappear. We need to get the outcome in front of the customer." - Philipp Herzig

"Everybody who works today, maybe in the finance shared service center, it's for me the equivalent of a junior developer today with Cloud Code. Now they actually get one level higher - they're not so much tasked with writing a lot of the code, but they start supervising the code, give feedback." - Philipp Herzig

"There's this AI innovation race and then there's this AI outcome race, and the gap almost increases versus getting narrow." - Philipp Herzig

"The problem is large language models are not made for [prediction]. They're language models, so they do this phenomenally well [with text]. But if you still want to do these predictions, you have to go back to these classical machine learning approaches." - Philipp Herzig