A
Member
A model card is most useful when it lets a reader decide whether the artefact fits a specific use. That means concrete, checkable fields rather than general descriptions.
Suggested fields:
1) Identity: model name, version, checkpoint hash, release date, and the organisation maintaining it.
2) Training data: sources, collection dates, languages, filtering and deduplication rules, and known gaps. If sources cannot be listed, state the reason and the categories instead.
3) Evaluation: datasets with versions, prompts, decoding settings, metrics with confidence intervals, and the hardware/precision used.
4) Intended use and out-of-scope use: concrete examples, plus the failure modes observed during testing.
5) Limitations: measured subgroup or domain differences, known sensitivity to prompt format, context length behaviour, and stability across versions.
6) Operational details: licence, dependencies, compute requirements, quantisation notes, and how to reproduce the reported numbers.
7) Change log: what changed since the previous version, including regressions.
Two habits that improve quality: treat every quantitative claim as reproducible from the card, and version the card together with the checkpoint. A card describing a previous checkpoint is worse than no card.
Takeaway: a good model card is closer to a datasheet than marketing copy. If a reader cannot reproduce a reported number or identify what changed, the document needs work.
Suggested fields:
1) Identity: model name, version, checkpoint hash, release date, and the organisation maintaining it.
2) Training data: sources, collection dates, languages, filtering and deduplication rules, and known gaps. If sources cannot be listed, state the reason and the categories instead.
3) Evaluation: datasets with versions, prompts, decoding settings, metrics with confidence intervals, and the hardware/precision used.
4) Intended use and out-of-scope use: concrete examples, plus the failure modes observed during testing.
5) Limitations: measured subgroup or domain differences, known sensitivity to prompt format, context length behaviour, and stability across versions.
6) Operational details: licence, dependencies, compute requirements, quantisation notes, and how to reproduce the reported numbers.
7) Change log: what changed since the previous version, including regressions.
Two habits that improve quality: treat every quantitative claim as reproducible from the card, and version the card together with the checkpoint. A card describing a previous checkpoint is worse than no card.
Takeaway: a good model card is closer to a datasheet than marketing copy. If a reader cannot reproduce a reported number or identify what changed, the document needs work.