Back to Devoured - November 18, 2026
DEVOURED
How should we define “capabilities activities” for the purpose of pacing?

How should we define “capabilities activities” for the purpose of pacing?

AI Redwood Research
Researchers propose a standardized, auditable framework for defining 'capabilities activities' to pace the frontier of AI development.
What: Alek Westover and Nathan Sheffield provide a taxonomy for AI compute usage, suggesting that companies categorize work into 'capabilities' (by default) and specific, narrow exceptions for safety, products, and customer serving.
Why it matters: Without a clear, third-party verified way to measure compute spend, voluntary safety commitments from AI labs remain unverifiable and potentially susceptible to gaming.
Takeaway: If you are involved in AI policy or infrastructure, advocate for the inclusion of external, third-party audits in company compute-spend disclosures.
Deep dive
  • Defines a 'capabilities' activity broadly as any compute used to improve model performance or general intelligence.
  • Proposes four narrow non-capabilities categories: Idle compute, product development, serving customers, and narrow safety research.
  • Narrow safety research is limited to a small whitelist: alignment evaluation, security patching, interpretability, CoT informativeness, model persona, model organisms, and theoretical alignment.
  • Emphasizes that activities should be classified by their potential for dual-use (improving capabilities) rather than just their stated intent.
  • Advocates for a regime where aggregate compute spend on capabilities is legally or voluntarily capped.
Decoder
  • Dual-Use: Technology or research that has both beneficial applications and potential to be repurposed for harmful or more advanced capabilities.
  • Scalable Oversight: Research aimed at using AI models to help humans evaluate, supervise, or steer other, potentially more powerful AI systems.
Original article

How should we define “capabilities activities” for the purpose of pacing?

We’d like to pace the frontier by restricting the amount of compute AI companies use to improve AI capabilities, perhaps via legislation or voluntary commitments. For this purpose, we’d need a standardized definition of which uses of compute count as “capabilities activities”. This document gives one option for such a definition; a rough breakdown of our classification is as follows:

We define and elaborate on each of these categories below. First, though, we discuss the desiderata this definition is meant to satisfy.

Thanks to Thomas Larsen, Zach Stein-Perlman, Alex Mallen, Eric Gan, Jeff Wu, Girish Gupta, Adam Kaufman, and others for feedback on drafts of this document.

What our classification is meant to accomplish

In order for such a definition to be an effective tool for pacing, it needs to satisfy some basic properties:

  • Substantially restricting the amount of compute spent on “capabilities activities” should substantially decrease the rate of AI progress.
  • The classification should be robust to adversarial pressure, in that the above point should hold even if companies are willing to adjust or reframe their activities to avoid being classified as “capabilities”.
  • The classification should be fairly simple and tractable to adjudicate. Reasonable experts should largely agree on how this definition classifies activities.

There is a simple definition that satisfies all of the above desiderata: one could classify all work by AI companies as “capabilities activities”. The goal of this document is to propose a more scoped notion, allowing many types of work to be excluded from the “capabilities” total, while hopefully remaining pretty robust.

We stress that our definition is not intended as a perfect metric for whether an activity will genuinely end up improving capabilities. Instead, we adopt the following philosophy:

  • False-positives are better than false-negatives. (I.e., it is much more acceptable to classify something as “capabilities” that doesn’t accelerate AI progress, than to classify something as “non-capabilities” that does.)
  • Computationally non-intensive activities are negligible. (I.e., it is not very important how this definition classifies an activity if that activity only represents a very small amount of compute.)

We now give our definition. We start by defining non-capabilities activities. Later in the document, we give examples to clarify what capabilities activities are (although they are defined as “everything other than non-capabilities activities”).

Non-capabilities activities

Activities are considered capabilities activities by default. The only exceptions are idle compute, narrow product development, serving customers, and narrow safety activities. These exceptions apply only if the activity doesn’t improve general model capabilities.

Idle compute

Compute that isn’t performing any computations. Note: the whole GPU has to be idle—an underutilized core doesn’t count as an idle GPU. It should also be idle for a non-negligible amount of time.

Narrow product development

Taking the general purpose AI systems that companies have made and turning these into products counts as a non-capabilities activity, assuming that this doesn’t make general AI systems more capable or enable faster capabilities progress. Work on separate narrow models also counts as a non-capabilities activity, again, assuming this doesn’t make general AI systems more capable or enable faster capabilities progress.

The activities that count as narrow product development are using AI labor to develop products or narrow models, and running R&D experiments for developing the product or narrow model. Examples include integrating AI into software or physical devices, training protein-folding models, recommendation algorithms, engagement algorithms, or using AI to generate ads.

Serving customers

Most forms of serving inference to external customers are non-capabilities activities. The main exceptions are:

  • If the AI company outsources capabilities-relevant tasks to customers.
  • If the AI company trains on its interactions with customers.
  • If the customers themselves use models to help with AI R&D tasks.

Serving inference to company employees should generally be considered a capabilities activity. However, inference for internal company employees who aren’t scientists and aren’t working on AI R&D (namely: company lawyers, marketing teams, HR teams, and communications teams) is considered a non-capabilities activity.

Narrow safety activities

Some forms of safety work seem distinct enough from capabilities work that we consider them non-capabilities activities. We define narrow safety activities to be any of the following: (1) evaluating model alignment, (2) testing and improving company security, (3) research that could improve safety and isn’t too dual-use, (4) monitoring, (5) running misuse classifiers, or (6) incident investigation.

  • Evaluating model alignment. Alignment evaluations aim to determine whether a model would exhibit some behavior, as opposed to whether it could solve some task.
  • Testing and improving company security. This applies only to security work for company-internal infrastructure. If companies are iterating against the results of “pen tests” to make models more capable, then that counts as a capabilities activity.
  • Research that could improve safety and isn’t too dual-use. We specify a whitelist: interpretability research, CoT informativeness research, model persona research, model organisms research, and theoretical work on alignment without near-term applications to improving capabilities.
  • Monitoring. Running monitors to detect bad behavior is a safety activity, provided they are not used as part of an optimization loop to select model improvements.
  • Running misuse classifiers. Running classifiers to check if users are trying to use LLMs to cause harm.
  • Searching for and investigating misalignment incidents. Searching for and investigating egregious misalignment incidents counts as a non-capabilities activity.

Capabilities activities

AI company activities are considered capabilities activities by default, unless they fall into any of the categories above. This includes:

Central capabilities activities

  • New production pretraining and post-training runs.
  • Distillation.
  • Building and running capabilities evaluations and benchmarks.
  • Improving kernels or optimizing inference.
  • AI R&D experiments not classified as safety or narrow product development.
  • Designing scaffolds, prompts, or skills that could make models better at coding or AI R&D.
  • Designing hardware accelerators.
  • Obtaining, filtering, grading, or creating data to use in AI R&D.

Safety work that might improve capabilities

We classify the following as capabilities activities because they are hard to distinguish from capabilities-driven work:

  • Work on debate or “scalable oversight”.
  • Work on conceptual reasoning capabilities or solving fuzzily specified tasks.
  • Running control, monitor, or dangerous capabilities evaluations.
  • Running “constitutional AI” / RLAIF.
  • Training monitors or misuse classifiers.
  • Training for jailbreak robustness.
  • Improving RL environments or patching broken ones.

Product work that might improve capabilities

Some work on consumer product development that might yield generally-applicable intelligence or algorithmic improvements should be classified as capabilities work, such as building products that require training general-purpose models for specific domains (like math or legal research), scaffolding models for coding/research, robotics, or work on voice/video/image models.

Unknown or ambiguous activities

If an activity doesn’t clearly fall into any of the non-capabilities categories, it is presumptively considered a capabilities activity.

Conclusion

Pacing the frontier is urgent. We urge AI developers to let third-party auditors measure and report how much compute AI companies are allocating to capabilities and non-capabilities activities, using a definition of these activities broadly similar to the one given in this document.

Explore all 55 articles in today's digest

Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi