SAP: Bringing the 'Operating System' of a Company into the AI Era with CTO Philip Herzig

Key points

• SAP's durability story (1972 → today, multiple tech-cycle survivor): 'Customers always seek outcomes and ROI. The technology layer changes, the outcome bar doesn't.' Same logic that made standard software work in the 1970s makes the AI layer matter today.
• **'AI is a business-model transition, not a technology transition.'** The on-prem-to-cloud playbook applies: people initially treated cloud as 'on-prem on the internet,' then learned CI/CD, multi-tenancy, scaling. Same multi-year arc happening with AI now.
• Three layers being re-engineered simultaneously: (1) UI — 'the time is over where you design software that requires intelligence to sit in front of the computer.' Generative UI replaces button-clicks. (2) Business processes — agents blend structured (ERP) and unstructured (email, docs) workflows for outcome-as-a-service. (3) Data layer — single harmonised semantic view of all enterprise data, as fuel for the AI.
• Concrete shipped products: **Joule for Consulting** — fastest-growing AI product in the SAP suite, reduces consultant effort by ~30% on complex landscape moves. Concur's travel and expense agents live in production.
• The real engineering challenge: **scaling AI past the demo.** RAG on 10 docs is a CEO-impressive demo. RAG on 1,000 docs is engineering. RAG on 100,000 docs and 20,000 APIs is a different problem class. Add master-data coupling (Sarah is a US employee, Philip is German — same query, different correct answer) and you need real architecture, not vibe code.
• MCP scale challenge: 'Last year everyone could build an MCP server, super simple. 10 APIs is fine. 100 APIs you get context bloat. We have 20,000 APIs.' The composition problem at Fortune 500 scale is non-trivial.
• Test-driven development is **back**, but for evals. 'You used to remember the Google guys saying they go home at 5pm because they wrote their tests. Nobody actually did it because the requirements were too messy.' Now agents demand the discipline because verification = evals = boundary conditions.
• On the SaaS-apocalypse market panic: 'Last year AI was in a bubble. This year SaaS is dead. Reality has always been more durable than that. Companies need to get managed end-to-end.' Continuation of McDermott's framing from Issue 01.
• On LLMs and predictive analytics: LLMs alone are not enough for the forecasting and operational-research problems that ERPs solve. The traveling-salesman class of problem still wants a hybrid: LLM for natural-language interface and reasoning + classical optimisation under the hood.
• Real-time enterprise relevance bar: 'Tariffs being introduced. New taxes. Strait of Hormuz. What does this mean for my supply chain?' These are queries an oil and gas customer wants answered against their ERP data within minutes — not the kind of thing Perplexity or ChatGPT can do today.

Notable quotes

Themes

• AI as business-model transition (the on-prem-to-cloud parallel)
• Generative UI and the end of click-to-teach interfaces
• Scaling agents past the demo (20,000 APIs problem)
• Eval-driven development as the new discipline