A panel-format adapter produces shorter answers with lower accuracy

[ 2026-04-24 · updated 2026-10-24 ] // Qwen3-30B-A3B // 10 MIN READ

An exploratory study. I trained a small adapter that makes Qwen3-30B-A3B-Base reason as a panel of debating experts. On MATH500 level 5 and a 20-problem AIME subset, it wrote completions several times shorter than those of Qwen's production thinking model and was less accurate. The two models differ in training history as well as format.

The question and the comparison

DeepSeek-R1-Zero (DeepSeek-AI, 2025) showed that reinforcement learning (RL) applied directly to a base model, rewarding only a correct answer in the expected format, can produce long step-by-step reasoning. Its prompt, the recipe's only instruction on how to reason, casts the model as a single assistant, so the prompt is where that framing can be changed. Following that RL-only approach and reward, I trained a Qwen base model with a prompt that asks for a panel-of-experts debate instead. The question was what a short RL run learns under that framing, and how it compares with Qwen's thinking model on accuracy and completion length.

The panel model is Qwen3-30B-A3B-Base, a mixture-of-experts model with 30 billion parameters (about 3 billion active per token), plus a rank-32 LoRA adapter, a small set of low-rank weight updates on the frozen model. I trained it with RL only (no example answers or distillation) on Tinker, Thinking Machines' hosted training service. The prompt is abridged below (full text in envs/multipersona_gsm8k.py); the reward is answer correctness, minus 0.2 for missing or malformed tags. Training ran 128 steps on GSM8K grade-school word problems, then 128 on the training split of MATH, competition problems rated level 1 to 5 (hardest), with 16 problems × 8 samples per step at learning rate 5×10⁻⁶. The released adapter is the post-MATH checkpoint.

# Panel prompt template, abridged (used at both train and eval time)
A conversation between User and Multi-Persona Panel of Experts.
The user asks a question, and the Panel solves it. The Panel first
deliberates and debates the reasoning process with each other and
then provides the user with the answer.

User: {problem}
Assistant: <mutipersonaDebate> … </mutipersonaDebate> <answer> … </answer>

The comparison model is Qwen/Qwen3-30B-A3B, Qwen's production hybrid model, run with apply_chat_template(enable_thinking=True) so that it writes a <think> block before answering. It shares the architecture but went through Qwen's full post-training instead of my two RL stages.

MATH500 is a 500-problem subset of the MATH test set chosen by Lightman et al. (2023); AIME is the American Invitational Mathematics Examination, a high-school competition with whole-number answers from 0 to 999. Both models were sampled at temperature 1.0 and graded by the same answer checker, with different token limits and sample counts:

Both subsets were taken in dataset order: the first 10 problems at each MATH500 level, and the first 10 of each AIME year, listed with answers in the AIME summary file.

The panel framing and its prompt wording are mine. Claude Opus 4.7 was my coding agent; the repository's commit history credits it as co-author of the token-efficiency analysis, README updates and public-release clean-up. The evaluation scripts graded the answers, and every result here comes from summary files the evaluation and analysis scripts saved; I rechecked the post against those files with Claude Opus 5.5 in September and October 2026.

Accuracy and completion length

Token counts are completion tokens as the sampler recorded them. Table 02 pairs each thinking sample with the panel sample of the same index on the same problem: 536 pairs on MATH500 level 5 (the panel's first 4 of 8 samples per problem) and 320 on AIME.

On MATH500 level 5 the matched estimate (318 against 487 correct of 536, a 31.5-point gap) differs from the all-sample estimate, which adds the panel's other 4 samples per problem: 626 of 1,072 (58.4%) against 487 of 536 (90.9%), a 32.5-point gap and the k = 1 value in Table 04. On AIME every sample is matched (72 and 234 correct of 320). Those are 16 draws on each of 20 problems, and the MATH500 pairs come from 134 problems; repeated samples of a problem are not independent tests.

Tokens per correct sampled answer divides the tokens a model spent by its number of correct samples. It is a retrospective evaluation measure, not a deployment cost: in use nobody knows which sample is right, and the panel's samples are wrong more often. A cost comparison needs a fixed answer-selection procedure, which I did not run.

The length difference holds pair by pair, not only in the means (on AIME, panel 1,408 against thinking 11,328 tokens):

Pass@k

Pass@k, the chance that at least one of k samples is correct, uses all samples. The panel trails at every k I measured, and the gap narrows as k grows.

A narrowing gap does not show that the panel explores more widely. Independent sampling alone narrows it as the stronger model nears its ceiling and the weaker one still has room to rise: if the panel solves a problem half the time and the thinking model 90% of the time, the gap is 0.40 at k = 1 and about 0.06 at k = 4 (0.9375 against 0.9999). On the 50-problem MATH500 slice the thinking model's pass@3 is already 1.0, so from k = 3 the gap can only shrink. On AIME the panel solved 11 of the 20 problems at least once in 16 samples, the thinking model 18, and none was solved only by the panel.

Interpretation and limits

The study describes how two models differ on these problems, not why. My adapter after 256 RL steps and Qwen's fully post-trained model differ in training history as well as format, and either could produce the accuracy and length gaps. This confound is the study's most consequential limitation; the matched-format control below would separate the two.

Three things affect the estimates themselves:

• Token limits and sample counts differ (Table 01), and I did not count how many panel samples hit their lower limits.
• The sets are small: a 20-problem AIME subset, 134 MATH500 level-5 problems and a 50-problem MATH500 slice.
• Each training stage is a single run with one seed.

Next experiment

• A matched-format control: a thinking-format arm trained from the same base with the same recipe, steps and LoRA rank, both arms evaluated at the same token limit and sample count. The repository has this arm's environments and launch scripts, but the arm was not completed.
• Count how many panel samples stop at their token limit, and extend pass@k beyond 16 samples.
• A cost comparison: pick one answer per problem by a fixed procedure, such as a majority vote over a set number of samples, and compare both models' accuracy and total tokens at matched limits.
• Measure variety between samples on the full trace with persona names stripped, and count distinct solution approaches per problem by blinded annotation.

Follow-up: training on partially solved problems

The follow-up note, Exploratory follow-up on training with partially solved math problems, looked at where each model gets a learning signal under group-relative RL, whose correctness reward carries signal only on problems a model solves on some but not all attempts. Of 877 competition problems sampled 8 times per model, the panel partially solved 382 and the thinking model 209. One 100-step run trained the panel on its 354 partially solved problems outside a 100-problem held-out set, where its score rose from 14% to 29% by the last evaluation, at step 90. The held-out set was evaluated every 10 steps, the thinking-model run was not completed, and one run does not show that this selection beats another curriculum. Both the sampling and the run used the post-GSM8K adapter, not the released post-MATH one.

Reproducibility

Appendix: a lesson in measuring variety between samples

I also tried to measure how different a model's samples on one problem are, and the attempt is more useful as a lesson about measurement than as a result. I averaged the cosine distance between all-mpnet-base-v2 sentence embeddings of the reasoning text (inside the debate or think tag) over all pairs of a model's samples on each problem, and compared models with a paired t-test over problems. The panel's samples came out farther apart, but the measure cannot say whether they differ in strategy.

Three confounds limit what this number can mean:

• Truncation: it sees only the opening. The embedding model drops everything after about 384 word pieces, and most thinking traces and many panel traces are longer; runs on the first 2,000 and 8,000 characters give the same numbers, because the model cuts both off at the same point.
• Naming and formatting: the persona names are included. Panel samples name their speakers differently ("Expert 1", "Persona 2", invented names), so names, phrasing and formatting can add distance without any difference in strategy.
• Scope: it compares samples within one model. It does not show whether the panel reaches ideas the thinking model does not.