Compute is eating the cap table: AI startup gross margins, and why a million in ARR can be a vanity metric

Sunny Ray · Jun 28, 2026 · 6 min read

An AI startup can hit a million in ARR and be raising money just to pay for its own tokens. Two guests on my show in June made that warning from opposite ends of the table, a week apart.

On June 19 I interviewed a London fractional CFO who calls his firm commercial architects. I read a line of his back to him that I have not stopped thinking about: "compute is eating the cap table." His argument was that many AI companies still price on the old SaaS model, built on margins above 80%, while compute and API costs have pushed some of them down to around 40%. Founders scale ARR without modeling the operating cost underneath, then raise rounds just to cover compute.

So what are AI startup gross margins, really? Much lower than the SaaS habits founders bring to the pitch. And lower margins change what investors need to see.

The 80% margin habit

For fifteen years software founders learned one set of numbers. Build once, sell many times, keep 80% or more of every dollar as gross margin. Investors learned to value companies on revenue multiples because the margin underneath was assumed.

AI breaks the assumption. Every query costs something. Every heavy user costs more. The product gets more expensive to serve the more people love it, which is the opposite of the software math everyone grew up on.

The public data backs the guest. Bessemer's State of AI report for 2025 put its fastest-growing AI cohort, which it calls Supernovas, at about 25% gross margins on average, trading profit for distribution in the short term. Its steadier cohort, the Shooting Stars, ran around 60%, still below SaaS peers. Neither number looks like the 80% many founders still put in their models.

Why a million in ARR can be a vanity metric

On air I put his position back to him bluntly: "hitting a million in an ARR can be a vanity metric." Teams celebrate the milestone, and sometimes what they are celebrating is fragile demand.

A million in ARR at 80% margin is a business. A million at 25% margin, with customers who would leave if you raised prices to cover the compute, is a subsidy. The revenue line looks the same on a slide. The company underneath could not be more different.

His fix was specific: build the pricing model and the unit economics model separately, before you scale. Pricing answers what the customer will pay. Unit economics answers what it costs you to serve them. If you only build the first, you find out about the second when the cloud bill arrives.

The true cost of tokens isn't known yet

The second warning came on June 12 from a seed investor who spent 21 years at an investment bank before backing startups. He was talking about the flood of venture money into AI and said: "we're not paying the true economic costs of tokens today."

His point was that nobody knows whether tokens will get cheaper through efficiency or more expensive as demand for power grows. He compared it to startups that ran on investor subsidies, where the economics only worked until the subsidy ended. And he added a line every founder replacing headcount with AI should hear: the AI tooling that replaces an employee may end up costing more than the employee.

He was also contrarian in a way I liked: accounting matters. Cash accounting alongside the standard statements. Contribution margin by product line. Accrued expenses do not make payroll.

AI spend is a number to watch, in both directions

The same seed investor shared how his firm watches AI inside its portfolio: it tracks AI spend per employee as a productivity signal. High spend means a team is either extremely proficient or extremely inefficient. Zero spend means someone needs a nudge.

That is the right instinct for founders too, pointed at your own product. Your compute bill is not just a cost line. It tells you which customers are getting real value and which ones are burning tokens you cannot charge for. The fractional CFO made the parallel point about readiness: understanding your unit economics and your data room matters as much as growth when you are trying to scale, especially in a capital-heavy business. Investors are not asking for perfect margins. They are asking whether you know exactly where yours come from.

Build the two models, then pitch

If you are raising for an AI company in 2026, the investor who matters will find your real margin in diligence. Better they find it in your deck, already explained.

• Gross margin by customer segment, not blended. Heavy users and light users are different businesses.
• Cost to serve per unit of usage, and how it moved over the last six months.
• What happens to margin if your model provider raises prices, and if they cut them.
• Pricing that tracks usage, so margin does not collapse when a customer loves the product.
• Contribution margin by product line, in cash.

We are living this with our own investor tool. On a June 16 call we were weighing simple monthly tiers against token-based pricing with a margin built in on usage. That conversation would not have happened in a pure SaaS company. In AI it is the first conversation.

What to show investors instead of top-line ARR

In May I saved a post listing weak founder traits, and one of them was confusing valuation with validation. ARR without margin is the same confusion. The number validates that people will use the thing. It does not validate that you can afford to let them.

Lead with ARR, sure. Then show the slide most AI founders skip: what that revenue costs to deliver, and why it gets better with scale instead of worse. If you can show margin improving as you grow, you are pitching a business. If you cannot, you are pitching a fundraising treadmill, and the smart investors will know before you do.

If you want to raise with numbers that survive diligence, and with the investors who actually ask these questions, raise with numbers that hold up.

Frequently asked questions