How MetaRoles Measures How Much of a Role AI Does

1. Roles and tasks: what gets counted

Every MetaRoles role post is anchored to an occupation in O*NET, the US Department of Labor's occupational database, keyed by its O*NET-SOC code (Standard Occupational Classification), such as 43-4051.00 for customer service representatives. O*NET describes each occupation as a list of task statements: the 31.0 release, the current one, holds 18,838 of them, and each task is linked to exactly one occupation.¹²¹⁴

O*NET sorts those tasks into two types. A core task is one "critical to the occupation": at least 67% of workers surveyed rate it relevant, and its mean importance rating is 3.0 or higher. Every other task is supplemental, either because fewer than 67% rate it relevant or because its importance falls below 3.0.¹² The ratings behind that split (importance, relevance and frequency) come from surveys of people in each job and are published for 911 occupations.¹³

A scorecard's rows are core tasks, quoted close to verbatim from O*NET. Supplemental tasks can come up in the narrative, but they are never rows. Some scorecards list every core task; others list only the core tasks AI is being applied to, and their table note says which were left out and why.

2. Evidence grades: how mature each case is

Each case on a scorecard carries one of four grades, as defined on our About page:¹¹

• In production a named organisation runs it in live work.
• Pilot a named organisation is trialling it in real work.
• Prototype shown in a lab, a paper or a demo, not in real work.
• Vendor claim stated only by the seller, with no confirmation from the organisation using it or from an independent source.

The grade comes from the sources, not from our judgement, and a case moves up only when a new source supports it. A vendor's announcement doesn't count as a deployment until the operator or an independent source confirms it. A deployment that has ended is reported as ended, with its dates. Where a task has several cases, the task's grade is the most mature one.

3. Extent and the "X of Y" count

Each task row also gets one of three extent labels. Does it needs a production case in which AI or a robot performs the task without a person doing it in that instance, so a "Does it" row always carries the production grade. Assists means AI helps a person do the task, in production or in a pilot: a person still reviews, signs or finishes the work. Not yet means the only cases are prototypes or vendor claims, or that we found none.

The headline on every scorecard counts the "Does it" rows against all rows: "3 of 8 core tasks AI does in production", for example. That count has narrow limits, and it is worth being exact about them:

• It means AI completes the task for some of the work at the named operators, not for every customer, patient or shift, and not at every employer.
• It is a count of tasks, not a share of working time. Tasks aren't weighted by how long they take, so 3 of 8 does not mean three-eighths of the job.
• It is not a share of jobs, a count of workers affected or a measure of job loss.
• It is not a forecast. It reports what sources show today, and it changes only when a new source does.

4. Videos, costs and benefits

A scorecard's "Seen at work" block shows two to four videos of real use, published by the operator or by the maker on its own channel. Each video ID is checked against YouTube's oEmbed service before it ships, and the caption names the channel and says "vendor video" when the maker published it. When no genuine footage of real use exists, the block is left out.

Costs and benefits are figures exactly as their source states them, with who stated them: "vendor-stated", "operator-stated", or the type and funder of a study, such as a randomized trial. When nothing is published, the tile reads "Not disclosed" rather than an estimate. Vendor-stated figures are never mixed with independent measurements, and we don't add up separately reported figures into totals of our own.

5. How other indices measure AI and work

The best-known measures of AI and occupations ask a different question from the scorecard, and each answers it with different data. They appear side by side in the table above, never combined.

Capability exposure. Eloundou et al. rate tasks on whether an LLM could cut the time they take by at least half at the same quality, using human annotators and GPT-4 as raters. They define exposure "as a proxy for potential economic impact without distinguishing between labor-augmenting or labor-displacing effects."¹ The AIOE works at the level of abilities rather than tasks: it links 10 AI applications to O*NET's 52 human abilities and weights each occupation by its mix of abilities.⁴ The ILO's 2025 index combines worker surveys, expert rounds and an AI model to score tasks under the International Standard Classification of Occupations (ISCO-08).⁶

Observed usage. Microsoft's applicability score and Anthropic's Economic Index start from conversations with each company's own chatbot, mapped to O*NET work activities or tasks.⁷⁸ They show what people ask one product to do, not what employers have deployed, and Anthropic's paper notes that its data paints "a picture of AI usage on a single platform."⁸ Anthropic's 2026 "observed exposure" combines that usage with the Eloundou et al. ratings, and Anthropic describes it as a measure of "AI displacement risk." That is Anthropic's own label for its measure.⁹

Robot capability. Anthropic's robot exposure index asks whether present-day robots can do a task, and in how structured a setting: from a purpose-built robotic work environment such as a factory assembly line (E1) to a structured facility such as a warehouse (E2) to an unstructured place such as a city road (E3). It counts demonstrations as well as deployments and sales, and Anthropic reports that results are similar without the ratings that rely on demonstrations.¹⁰ On a MetaRoles scorecard, a demonstration alone is a prototype and doesn't make a task "Does it".

The scorecard differs from all of these in what it counts: confirmed deployments at named organisations, graded one by one. It is narrower and slower to move than an exposure score, and it doesn't produce a single number for the economy.

6. What none of these measures show

None of the measures in the table, including ours, is a count of jobs lost or a measure of wages. The publishers say so in their own words. Eloundou et al. write: "We do not make predictions about the development or adoption timeline of such LLMs."¹ Microsoft's authors write that reading high applicability as future "job or wage loss" "would be a mistake," because "AI's effects on employment and wages will depend on hard-to-predict business decisions."⁷ The ILO concludes that "transformation of jobs is the most likely impact of GenAI," since most occupations consist of tasks that require human input.⁶

Where a publisher does link its measure to employment, the finding is its own, with its own method. Anthropic reported in March 2026 that it found "no systematic increase in unemployment for highly exposed workers since late 2022," along with "suggestive evidence that hiring of younger workers has slowed in exposed occupations."⁹ In its robot study, Anthropic reports that over the past 50 years "jobs more exposed to robots experienced greater declines in wages and employment than others."¹⁰ MetaRoles reports findings like these attributed to their authors, with dates, and never presents an exposure or usage score as job loss.