WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data

WearableQA is a benchmark of 4,084 ten-option multiple-choice questions built from the wearable time series, blood biomarkers, and demographics of 200 real users, each with up to about 500 days of daily measurements. Unlike benchmarks built on synthetic or idealized signals, it preserves authentic wearable distributions — device noise, missing days, and inter-individual variability included.

Each question asks a model to reason over one user's longitudinal record: to compute over the raw measurements, to interpret them physiologically, or both.

The 16 question types are organized along two complementary axes:

• Data vs. health reasoning — computing over longitudinal measurements (correlations, excursion counts, recovery times, trend shapes) versus interpreting them physiologically (risk assessment, differential diagnosis, prognostic prediction).
• Single- vs. cross-signal reasoning — reasoning within one metric versus integrating several.

Ground-truth answers are balanced uniformly across options A–J within each reasoning group, so the random baseline is 10%.

What's in this repository

Three files, in two formats — a ready-to-use rendered dataset, and the structured source it is rendered from.

1. WearableQA.jsonl — the pre-rendered benchmark

4,084 lines, one JSON object per question. This is the file to use if you just want to run a model against the benchmark: every prompt is already a single string, so no assembly is needed.

{
  "id": "Lit_user_25_2024-08-06",
  "question": "You are given a user's demographics, wearable health sensor history, ...",
  "choices": {"A": "an elevated resting heart rate with low daily activity ...", "...": "..."},
  "answer": "A",
  "category": "fitness_prediction",
  "reasoning_group": "health",
  "signal": "cross",
  "grounding": "literature",
  "representation": "row"
}

The question string is assembled from these sections:

(instruction)
=== USER PROFILE ===                                age, sex, BMI, ethnicity
=== SENSOR DATA (row: one line per day) ===         up to ~500 days of daily metrics
=== BLOOD BIOMARKER PANEL ===                       17 biomarkers, where available
=== COHORT REFERENCE (population percentiles) ===   p10/p25/p50/p75/p90 for 5 metrics
=== QUESTION ===                                    the question stem
=== OPTIONS ===                                     A-J

Prompts are large: median ~86k characters, max ~124k. Budget context accordingly.

2. WearableQA_raw.json — the structured source

The same benchmark before rendering, so you can serialize the time series differently, feed the numbers to a tool-using agent, or build your own prompt template.

3. render_raw.py — the renderer

Rebuilds a .jsonl from the structured source. With no arguments it reproduces the released WearableQA.jsonl byte-for-byte.

Getting the data (Git LFS)

The dataset files are stored with Git LFS. Install LFS before cloning, or you will get small text pointer files instead of the data.

Quick start

The dataset is also on the Hugging Face Hub, which is the quickest way to get it — no clone, no LFS setup:

from datasets import load_dataset
ds = load_dataset("facebook/WearableQA", split="test")

Citation

@misc{lee2026wearableqa,
      title={{WearableQA}: A Benchmark for Health Reasoning over Real-World Wearable Data},
      author={Ji Soo Lee and Xilun Chen and Pierce Chuang and Ashish Shenoy and Jason Wei and Dohwan Ko and Hyunwoo J. Kim and Benoit Corda},
      year={2026},
      eprint={2609.05405},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2609.05405},
}

License

The data is licensed under Creative Commons Attribution-Non Commercial 4.0 International (CC BY-NC 4.0), and subject to the following additional terms: (i) No re-identification or attempted re-identification; (ii) No use in connection with clinical, diagnostic, or treatment decisions; (iii) No use in a manner that is discriminatory, harmful, or misleading with respect to health-related outcomes; (iv) The Dataset is provided "as is", without warranties of any kind, whether express or implied, including without limitation accuracy, completeness, or fitness for a particular purpose, and is provided for research and benchmarking purposes only.