Two feedback products
The InstructGPT work described two human-produced inputs: written demonstrations of desired outputs and rankings comparing model outputs (Ouyang et al., arXiv:2203.02155). They have different quality profiles: demonstrations are judged on correctness and style, rankings on consistency and calibration.
Quality checks for preference data
| Metric | What it reveals |
|---|---|
| Inter-rater agreement on shared items | Whether the criteria are learnable |
| Self-consistency on repeated items | Rater attention |
| Agreement with expert adjudication | Whether the crowd tracks expert judgement |
| Tie / 'both bad' rate | Whether the task forces false choices |
When expertise changes the answer
For general helpfulness, generalist raters may be adequate. For medicine, law or code security, non-experts can prefer the confident wrong answer. That is the case for specialist feedback supply — see expert annotation services.
For owners of feedback archives
- Keep the rater guideline version attached to every judgement.
- Retain the prompts and both outputs, not just the preference.
- Confirm the model outputs being judged are licensable to share.
Have an annotated dataset or specialist expertise?
Tell us what you own, how the labels were reviewed and which rights you can license. A short overview of your dataset or annotation business is a useful place to start.
Email us about your dataset
Prepare for a licensing conversation Prefer to write directly? hello@callwavemedia.com