AutoTTS

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

Tong Zheng, Haolin Liu, Chengsong Huang, Huiwen Bao, Sheng Zhang, Rui Liu, Runpeng Dai, Ruibo Chen, Chenxi Liu, Tianyi Xiong, Xidong Wu, Hongming Zhang, Heng Huang UMD · UVA · WUSTL · UNC · Google · Meta

Project page

AutoTTS reframes TTS strategy design from hand-crafting heuristics to environment-driven automatic search: humans only construct an offline replay environment (states, actions, feedback, objectives), and a coding agent iteratively proposes and refines code-defined controllers within it — code edits, no gradient updates. Cheap: 0 LLM calls, fully replay.

Quick links: Install · Reproduction · Citation

Highlighted results

• ~69.5% tokens saved vs SC@64 at β ≈ 0.5; held-out average accuracy matches SC@64 across four backbone scales.
• $39.9 estimated monetary cost for one full discovery run.
• 160 minutes wall-clock for the same run.
• 0 LLM calls during discovery evaluation (replays cached segments only).

The discovered controller is the Confidence Momentum Controller (CMC), characterized by trend-based stopping, coupled width–depth control, alignment-aware depth allocation, and conservative branch abandonment.

Problem setup

We treat adaptive test-time inference as allocating a finite budget over branches in fixed-length intervals.

State at step t:

s_t = (q, m_t, I_t, ℓ_t, Ω_t)

q: question; m_t: number of instantiated branches; I_t: active branch set; ℓ_t: depth vector; Ω_t: revealed probe triples.

Admissible actions A(s_t):

• BRANCH — open a new branch through the first interval.
• CONTINUE(i) — advance branch i by one interval.
• PROBE(i) — reveal ω_{i,ℓ} without advancing depth.
• PRUNE(i) — deactivate branch i; depths and past probes stay recorded.
• ANSWER — terminate and apply the controller's terminal aggregator.

Cost in interval units:

Cost(s_t) = Σ_i ℓ_{t,i} + κ_probe · |Ω_t|        (often κ_probe = 0)

Objective. A code-defined policy π(· | s, β) is parameterized by a scalar meta-parameter β that deterministically schedules every internal hyper-parameter. Over tasks (q, y) ~ 𝒟:

max_{π, β}  E_{q,y}[ 1{ŷ_{π,β}(q) = y}  −  γ · C_{π,β}(q) ]

The outer loop searches over implementations of π. Each candidate is replay-evaluated on offline caches; traces and scaling curves enter the next round's history.

Environment construction (run once per (model, benchmark))

The MDP above is instantiated as a concrete replay environment before the discovery loop starts:

1. Specify the interface. Fix s_t, A(s_t), Cost(s_t), and the accuracy–cost objective.
2. Offline trajectory collection. For each query, draw N parallel independent reasoning traces from the backbone (full strings first), then partition each trace into fixed-length segments of Δ tokens and enumerate branch prefixes z_{i,k} with probe responses ω_{i,k}.
3. Materialize the replay store. Every environment transition consults the archived table; e.g. PROBE(i) retrieves the cached ω_{i,k} without any new decoding.
4. Hand off to discovery. Candidate controllers are simulated exclusively through observe/step. Asymptotic evaluation cost is dominated by table replay.

Steps 1–3 run once. Iterative coding-agent discovery starts only after the replay store is frozen.

In this repository:

• efficient_reasoning_controller/workspace/code_base/environment/ — search-set replay store.
• efficient_reasoning_controller/test_environment/ — held-out replay store; never exposed to the proposer.

Discovery: β parameterization & trace feedback

• β parameterization. Each candidate controller exports a single scalar β plus a deterministic, monotonic map from β to every internal knob. Outer search collapses to sweeping β, eliminating brittle thresholds tuned only to the search set.
• History augmentation with execution traces. Alongside each round's β-sweep we archive both empirical scaling curves and the full action-by-action trajectories reconstructed during replay. Traces give the explorer fine-grained behavioral evidence to localize defects before rewriting code.

Main results

AutoTTS is optimized on AIME24 replay constructions and evaluated on held-out AIME25 / HMMT25 benchmarks across four Qwen3 backbone scales. The project page reports the following trends:

• Better accuracy–token trade-offs. Discovered controllers typically shift the empirical Pareto frontier beyond handcrafted baselines such as SC@64, ASC, ESC, and Parallel-Probe.
• Held-out generalization. Policies discovered on AIME24 transfer to held-out benchmarks, outperforming every handcrafted baseline on average accuracy for three of four backbone scales and remaining competitive on Qwen3-8B.
• β = 0.5 operating point. Cuts aggregate token usage by roughly 69.5% compared with SC@64 while matching mean held-out accuracy across models.
• β = 1.0 operating point. Pushes peak accuracy beyond all handcrafted baselines in five of the eight tabulated comparison cells on the project page.

Sweeping β traces accuracy–token scaling curves: larger β generally moves toward higher-budget, accuracy-first behavior, while smaller β favors cheaper inference.

Evolution of the discovery process

The round-level trajectory (e.g., t1 -> t5) shows a consistent move toward better objective values over the search process:

• On the search benchmark, later rounds improve accuracy while keeping token growth controlled, indicating progressively better policy structure rather than random fluctuation.
• On held-out benchmarks, the same trajectory remains competitive and often improves, suggesting that the discovered control logic transfers beyond the optimization split.
• The trajectory reflects objective-seeking code evolution without gradient updates: the agent edits explicit controller programs, receives replay-based accuracy/cost feedback, and iteratively shifts behavior toward better empirical trade-offs.

This is a key point of AutoTTS: optimization is achieved through iterative program search in a fixed replay environment, not through backpropagation or parameter fine-tuning of the backbone model.

Discovered controller: CMC

The discovered controller is named the Confidence Momentum Controller (CMC). Its main mechanisms are:

• Trend-based stopping. CMC maintains an exponential moving average of pool confidence and stops only when the confidence level is high and the trend is non-negative. This avoids stopping on transient confidence spikes.
• Coupled width–depth control. Widening and deepening are linked through the EMA delta: strong confidence gains suppress new branch spawning, while stagnation or regression triggers widening.
• Alignment-aware depth allocation. Branches whose latest answer matches the pool winner receive extra probe steps, concentrating compute on the emerging consensus while still advancing active branches.
• Conservative branch abandonment. A branch is abandoned only after persistently deviating for multiple rounds, and at least two active branches are preserved.

These mechanisms are implemented as code-defined controller logic and evaluated through the same replay environment as the handcrafted baselines.

Show full OptimalController source (CMC)

class OptimalController(LLMDesignedMethod):
    """Confidence Momentum Controller (CMC)."""
    NAME = "optimal_controller"
    _MAX_BRANCH   = 64
    _MAX_OUTER    = 500
    def _schedule(self, beta: float) -> dict:
        b = max(0.0, min(1.0, float(beta)))
        n_init           = max(2, round(2  + 6  * b))
        max_branch_use   = min(self._MAX_BRANCH, round(4 + 60 * b))
        warm_up          = max(2, round(2  + 8  * b))
        abandon_patience = max(3, round(3  + 9  * b))
        T_ema            = max(2, round(2  + 6  * b))
        ema_alpha        = 0.70 - 0.40 * b
        conf_thresh      = 0.85 + 0.12 * b
        delta_slack      = 0.04 - 0.03 * b
        burst_aligned    = max(1, round(1 + 2 * b))
        widen_burst      = max(1, round(1 + 3 * b))
        trend_thresh     = 0.04 - 0.03 * b
        min_complete     = max(2, round(2 + 3 * b))
        return {
            "n_init":           n_init,
            "max_branch_use":   max_branch_use,
            "warm_up":          warm_up,
            "abandon_patience": abandon_patience,
            "T_ema":            T_ema,
            "ema_alpha":        round(ema_alpha, 4),
            "conf_thresh":      round(conf_thresh, 4),
            "delta_slack":      round(delta_slack, 4),
            "burst_aligned":    burst_aligned,
            "widen_burst":      widen_burst,
            "trend_thresh":     round(trend_thresh, 4),
            "min_complete":     min_complete,
        }

The same source also lives in efficient_reasoning_controller/workspace/code_base/method.py.

Repository structure

AutoTTS/
└── efficient_reasoning_controller/
    ├── eval/
    ├── logs/search_history/
    ├── workspace/
    │   ├── code_base/
    │   │   ├── data_loader.py
    │   │   ├── method.py
    │   │   ├── method.template.py
    │   │   ├── eval.py
    │   │   ├── evaluator.py
    │   │   ├── controller_api.py
    │   │   ├── trace_schema.py
    │   │   ├── environment/
    │   │   └── history/
    │   └── controller_search/
    │       ├── run_workflow.sh
    │       ├── workflow_propose_critic.py
    │       ├── claude_proposer.py
    │       ├── codex_proposer.py
    │       └── prompts/
    └── test_environment/

Install

Depending on how you reproduce results:

• Evaluate our controllers only — create the Conda environment and install numpy, pandas, tqdm. No Node.js, Claude CLI, or API keys are required for replay evaluation.
• Run discovery yourself — complete all subsections: Conda, Claude environment setup, and API environment setup.

Conda environment

conda create -n autotts python=3.12 -y
conda activate autotts

Claude environment setup

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash

source ~/.bashrc

nvm install 21

npm install -g @anthropic-ai/claude-code

pip install claude-agent-sdk==0.1.58

pip install numpy pandas tqdm

API environment setup

cat >> ~/.bashrc <<'EOF'
export OPENROUTER_API_KEY="your_openrouter_api_key"

export ANTHROPIC_BASE_URL="https://openrouter.ai/api"
export ANTHROPIC_AUTH_TOKEN="$OPENROUTER_API_KEY"
export ANTHROPIC_API_KEY=""

export ANTHROPIC_DEFAULT_SONNET_MODEL="anthropic/claude-sonnet-4.6"
export ANTHROPIC_DEFAULT_OPUS_MODEL="anthropic/claude-opus-4.6"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="anthropic/claude-haiku-4.5"
export CLAUDE_CODE_SUBAGENT_MODEL="anthropic/claude-opus-4.6"
export CLAUDE_CODE_SKIP_FAST_MODE_ORG_CHECK=1
EOF

source ~/.bashrc

Reproduction

There are two supported workflows:

Complete Install before Way B. Way A only requires the Conda setup and numpy / pandas / tqdm.

Way A — Evaluate our programs (eval/)

Use this when you want tables and traces on the bundled replay data without launching search.

1. Controller code. The repo ships a working efficient_reasoning_controller/eval/method.py. To evaluate a specific snapshot from our search logs, copy it over that file, e.g. from logs/search_history/<run>/code_base/method.py.
2. Configure sweeps. Edit models, datasets, and method lists at the top of eval/eval.py.
3. Run evaluation from the repository root:

cd efficient_reasoning_controller
python eval/eval.py

1. Outputs land under eval/test_results/, e.g. eval/test_results/matrix_results_<MODEL>/ with <DATASET>_raw_new_api.csv and <DATASET>_trace_new_api.jsonl.

Discovery evaluation inside the research codebase uses the same logic under workspace/code_base/eval.py; it writes to code_base/training_results/ instead. Use eval/ for the standalone "evaluate what we ship" layout.

Way B — Run discovery yourself (workspace/)

Use this to reproduce or extend the automated search loop (costs LLM calls; evaluation steps remain replay-only).

1. Environment. Finish Install (Conda + nvm/Node + claude-agent-sdk + API exports). Authenticate the Claude Code CLI (claude login) as needed.
2. Set up History: Download History from huggingface:

huggingface-cli download AutoTTS/history --local-dir ./history
cp -r ./history efficient_reasoning_controller/workspace/code_base/

1. Launch the workflow:

cd efficient_reasoning_controller/workspace
bash controller_search/run_workflow.sh

1. Optional tuning via environment variables:

export WORKFLOW_PROPOSER_BACKEND=claude
export WORKFLOW_ROUNDS=5
export WORKFLOW_EVAL_CMD="python code_base/eval.py"
export WORKFLOW_RESUME=1

Each round writes a snapshot under:

code_base/history/rNNNN_<timestamp>_<uid>/
├── method.py
└── proposal_results/

code_base/method.py is reset from code_base/method.template.py at the start of every round; each candidate must be self-contained in method.py.

Evaluation during search. WORKFLOW_EVAL_CMD defaults to python code_base/eval.py; matrices appear under code_base/training_results/:

code_base/training_results/
└── matrix_results_<MODEL>/
    ├── <DATASET>_raw_new_api.csv
    └── <DATASET>_trace_new_api.jsonl

After discovery. Copy any round's method.py into eval/method.py and follow Way A for a standalone rerun under eval/test_results/.

Built-in baselines

code_base/method.py ships:

• ASCMethod — adaptive self-consistency with Beta-confidence early stopping.
• ESCMethod — early stopping by sliding-window answer consistency.
• Parallel_Probe — parallel chains with warm-up, off-track pruning, and stable-majority termination.
• OptimalController — the target class rewritten by the search workflow (e.g. CMC).

Pre-computed seed baseline results are stored under:

efficient_reasoning_controller/workspace/code_base/history/seed_algorithms/

Citation

@article{zheng2026autotts,
  title  = {LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling},
  author = {Zheng, Tong and Liu, Haolin and Huang, Chengsong and Bao, Huiwen and
            Zhang, Sheng and Liu, Rui and Dai, Runpeng and Chen, Ruibo and
            Liu, Chenxi and Xiong, Tianyi and Wu, Xidong and Zhang, Hongming and
            Huang, Heng},
  journal={arXiv preprint arXiv:2605.08083},
  year    = {2026}
}

@article{zheng2026parallel,
  title={Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probing},
  author={Zheng, Tong and Huang, Chengsong and Dai, Runpeng and He, Yun and Liu, Rui and Ni, Xin and Bao, Huiwen and Wang, Kaishen and Zhu, Hongtu and Huang, Jiaxin and others},
  journal={arXiv preprint arXiv:2602.03845},
  year={2026}
}