Question Bank¶
Organize stable task sets for evaluation and informed dataset improvements.
A question bank defines tasks, inputs, and scoring criteria. It establishes what success means and gives model versions a consistent basis for comparison.
Use task sets with Mint Recursive¶
The current console selects benchmarks on Evaluation (评测) and exposes runs in My evaluations (我的评测). Authoring tasks, generating solutions, and preparing training examples happen in external benchmark or data projects. There is currently no standalone console editor or automatic question-bank-to-SFT workflow.
For an available benchmark, follow Evaluation. A custom question bank needs its evaluation project integrated first; uploading a training JSONL file does not register a new benchmark.
Design a reusable evaluation set¶
| Element | Purpose |
|---|---|
| Stable task identifier | Locate the same task across models and runs |
| Input and task description | Define the work expected from the model |
| Reference answer or scoring rule | Define a successful outcome |
| Dataset version | Keep dataset changes separate from model improvements |
Cover common cases, boundaries, and known failures. Reserve an independent test set that does not enter training.
Turn findings into better training data¶
Passing answers or trajectories are candidate examples, not automatically high-quality training data. Verify correctness, deduplicate, and convert them to the required format in your external workflow.
Upload the resulting SFT JSONL through My data (我的数据) as a new dataset version. Evaluate the next trained model against the fixed held-out set.
- Evaluation tasks
- Result analysis
- External preparation
- New dataset version
Next: Data format and validation · Evaluation