Half my design system metrics don't work anymore

Which metrics to keep, delete and which evals you need to set up

4 years ago I compiled my list of useful design system metrics. Well, a lot of them are not useful anymore. Not just total number of components and detachments, but many others as well.

Historically, adoption was the only metric that mattered, but today I would focus more on getting proper design system evals.

We have to think about a new user. Agents read our docs every single time and we need to set up rules to see if they are doing good job or not.

How do you know if your design system works or if it is getting improved every month? How do you share with stakeholders the numbers that give you budget? If you do this right, you will be able to see your progress.

Which metrics from the list I kept

Number of Snowflakes: How many one-off components you have? Not just in Figma, in code as well.

New Linter Warnings: Are there new violations coming up? Trend should go down.

Accessibility Pass Rate: What’s accessibility audit score?

Support Questions and Time to Answer: Since getting help from AI agents, are you faster? How much time do you need to answer design system questions?

Time to Adopt a New Release: How long does it take for product to run the latest version?

System-wide Change Effort: How much time does it take to re-brand, add new subrand or just change tokens?

Interviews with Designers and Developers: This one is oldie, but goldie. So simple, but very effective. You hear directly from design system users what they about it. Repeat twice a year.

Stop spending time with these metrics

These numbers can only go up, but that’s all you find out.

Total component count: The right number of components is when there is enough of them to cover for everything you need for building your products. Some systems can have 50, some 80, but having more is not a sign of success.

Installs and downloads as adoption: Number of npm downloads is just a number. What are people doing afterwards is important. Are they changing the code, adding more components? Ha, you don’t know.

Number of MCP calls: More MCP call doesn’t mean your agents are doing more work, maybe they are just confused.

Agents reporting their own success: You have to be careful with “too much“ success. A lot of times it might look like everything goes smooth and only when you push back, you get a real answer. Have in mind what was you end goal with task or eval, so you can compare the results.

Update the meaning of these metrics

These metrics are still useful, but I adjusted them a bit.

Detach Rate: How often are components detached? Are teams exploring new components or because the solution is not good enough. Imagine you have a multi-brand design system. The main one works great, but the number of detachments in other brands is significantly higher. What does this tell you? Maybe you did not think enough about surfaces, different icons, more complex components or just more elements.

Docs page views: Which pages are the most read? What are keywords people use when searching for something? Think about improving content, adding help, improving components, etc.

Adoption by Product: Which products use the system, and are they using components the way they were designed to be used? Treat it as a health check.

Design and Code Parity: Do your Figma components and coded components match in props, states and variants?

Trust Pulse: Get answers from design system user. Ask them if they would still use the design system it if it was optional? On the other hand, you also need to rate (and adjust) the trust levels for the agents.

Contributions from other teams, suggestions from the AI: How many contributions do you get? What kind of them? Do agents (AI tools) give you suggestions for improvements? Do you accept them?

Time to First Component: How long does to ship the first screen with the system? (human vs agent)

Lead Time Against a Baseline: How much faster do teams ship with the system? Compare with a baseline or a team without it, or the number means nothing.

Overrides and Hard-coded Values: How many style overrides and raw values sit in each product? This is your design and tech debt in one number.

UI Bugs per Product: How many UI and consistency bugs are reported for each product? Track the trend.

Now add these to your report

Conformance by surface: How much of your product really uses your components and tokens?

Agent adherence pass rate: How often an agent builds something that follows your rules?

Cost per shippable output: How much you really pay for something you can ship?

Accessibility pass rate: How much of your UI passes accessibility checks? Is it improving?

Component friction list: Which components don't do their job?

Setting up evals

Have in mind that having metrics look over design system health is totally different than measuring ROI or measuring the quality of the output.

The main thing today is to know if the system makes the output better and better? That’s for both users, people and agents. The output needs to be the same quality whether a person or agent prompt it.

Now we need our most needed layers again: intent + implementation. Intent tells us why and how components are use, and implementation is everything that changes over time anyway. The model, tools, skills, tokens.

To test how everything works together, we set up evaluations. In short, we call them evals. If these were users, we would call it user testing.

An eval is some kind of the bridge. It’s a real task, tested through our tools, models, components, scored againts your intent. For task we give points. When something changes, we run it again. And again.

How do you choose your evals?

First, go through past issues and challenges. Make a list. Then write down what would hurt your production the most if an agent (or human) broke the rules/guidelines?

Create a script that will automate these tasks. You can also write a list of prompts. Run each task or prompt at least 3 times.

When you get a report, read it, check the code, check the outcomes and of course the look and feel.

The trick is to try with cheaper models as well, because in the end, you want the save money and keep the speed.

You can start with 10-15 tasks only.

When you see costs going down, and pass rates up, you won’t need much math to show your design system is doing well.