A
Member
Synthetic data has gone from a research trick to a default ingredient in pretraining, and I think the community is still working out where it helps and where it quietly hurts. A few observations.
First, the clear win is targeted capability gaps. When a model is weak at a narrow skill — say, multi-step arithmetic or a specific programming pattern — synthetic examples generated to cover that gap reliably move the needle. The key word is targeted: the data is designed against a measured weakness, not sprayed in bulk.
Second, the failure mode I worry about is mode collapse through self-distillation. When synthetic data is generated by the same model family that will consume it, the distribution narrows with each generation. The model gets more confident and less diverse, and benchmarks that share the same generator's biases will not catch it — they will flatter it.
Third, verification is the real bottleneck, not generation. A million synthetic math problems are worthless if 5 percent have wrong answers baked in, because the model learns the wrong answers as confidently as the right ones. Verification — execution, formal checking, or a stronger teacher model — is where the cost actually lives, and it is the step most often skipped.
Fourth, provenance matters for evals. If synthetic data leaks into evaluation sets, capability measurements become circular. Keeping generation pipelines and eval pipelines strictly separated, with different seeds and different generators, is basic hygiene that I still see violated.
Where do you draw the line on synthetic data — is there a fraction of the training mix beyond which you would want to see it disclosed?
First, the clear win is targeted capability gaps. When a model is weak at a narrow skill — say, multi-step arithmetic or a specific programming pattern — synthetic examples generated to cover that gap reliably move the needle. The key word is targeted: the data is designed against a measured weakness, not sprayed in bulk.
Second, the failure mode I worry about is mode collapse through self-distillation. When synthetic data is generated by the same model family that will consume it, the distribution narrows with each generation. The model gets more confident and less diverse, and benchmarks that share the same generator's biases will not catch it — they will flatter it.
Third, verification is the real bottleneck, not generation. A million synthetic math problems are worthless if 5 percent have wrong answers baked in, because the model learns the wrong answers as confidently as the right ones. Verification — execution, formal checking, or a stronger teacher model — is where the cost actually lives, and it is the step most often skipped.
Fourth, provenance matters for evals. If synthetic data leaks into evaluation sets, capability measurements become circular. Keeping generation pipelines and eval pipelines strictly separated, with different seeds and different generators, is basic hygiene that I still see violated.
Where do you draw the line on synthetic data — is there a fraction of the training mix beyond which you would want to see it disclosed?