Cohen's Kappa Calculator (Inter-Rater Agreement)
How well do two coders agree beyond chance?
Spec sheet
cohen_1960 · v1.0.0Cohen's kappa (1960)
- 01
p_o = share of items both raters coded the same - 02
p_e = Σ_c p_A(c) · p_B(c) - 03
κ = (p_o − p_e) / (1 − p_e) - 04
SE ≈ √(p_o(1 − p_o) / (n(1 − p_e)²))
- Two raters only — use Krippendorff's alpha for more coders or missing data.
- Kappa depends on category prevalence; very unbalanced codes can give low κ despite high agreement (the kappa paradox).
- The standard error is the large-sample approximation.
- Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46standard
- Landis & Koch (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174standard
Worked example
computed by the same formula10 items, two coders
- Rater A codes (one per item, in order)
- yes, yes, no, no, yes, no, yes, yes, no, yes
- Rater B codes (same items, same order)
- yes, no, no, no, yes, no, yes, yes, yes, yes
- Cohen's κ
- 0.583
- Landis & Koch
- Moderate
- Observed agreement
- 80 %
- Chance agreement
- 52 %
- Approx. 95 % CI
- 0.067 to 1.1
- Items
- 10
- 1. Categories(no, yes) = 2
Questions
How is the cohen's kappa calculator calculated?
›
Cohen's kappa (1960). p_o = share of items both raters coded the same · p_e = Σ_c p_A(c) · p_B(c) · κ = (p_o − p_e) / (1 − p_e) · SE ≈ √(p_o(1 − p_o) / (n(1 − p_e)²))
Example: 10 items, two coders?
›
Cohen's κ: 0.583; landis & koch: Moderate.
What does this calculator not cover?
›
Two raters only — use Krippendorff's alpha for more coders or missing data. Kappa depends on category prevalence; very unbalanced codes can give low κ despite high agreement (the kappa paradox). The standard error is the large-sample approximation.
Where do the numbers come from?
›
Sources: Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46; Landis & Koch (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174. The formula is versioned (cohen_1960 v1.0.0) and covered by automated tests.
Is my data stored?
›
No. The calculation runs in your browser and nothing is sent unless you choose to save a shareable link.
Also searched ascohen's kappa calculator · inter rater reliability · intercoder agreement · kappa statistic · inter-coder reliability qualitative coding