Verbatim · Inter-coder reliability
Percent (observed) agreement
Raw observed agreement; NOT chance-corrected. Report alongside a chance-corrected coefficient, never instead of one — it overstates reliability when categories are skewed.
Po = (# units both coders coded identically) / (# jointly coded units)
- Measurement level
- nominal
- Coders
- Exactly 2 coders
- Seminal reference
- Scott (1955) (critique of raw agreement); Neuendorf (2017), The Content Analysis Guidebook 2nd ed.
Benchmark bands
Benchmarks are reporting guidance, not strict cut-offs — the engine presents them with their source.
| Agreement | Reading | Source |
|---|---|---|
| 0.0 – 1.0 | report only | Not chance-corrected — no benchmark band applies; present as descriptive context only (Neuendorf 2017). |
Verbatim computes Percent (observed) agreement with a deterministic, golden-verified math kernel — never a language model — with a bootstrap confidence interval and full provenance (coder + unit + kernel + fixture + run-id). The number is measured; the interpretation stays yours.
Measure this in the workshop →