The results were evaluated by using kappa (K) analysis in terms of interrater agreement
Data collection for the two students began as soon as interrater agreement
levels of at least 100% were achieved for three consecutive trials with the first author.
In the second phase (assessment of variability), recordings of paired automated and manual assessments were made to analyze variability and interrater agreement
for pupil size, detection of anisocoria, and PLR between 2 independent observers.
Prior to implementation of data collection, the TBOM and the SSIBOM were reviewed for clarity and face validity by experts in the field and were piloted to examine interrater agreement
Cohen's Kappa test was also used to evaluate interrater agreement
for qualitative (categorical) items.
Other authors have suggested a significantly higher number of repeated scans before an ultrasound-naive nurse can achieve a good interrater agreement
with that of an expert.
Nine of these 19 had an interrater agreement
greater than 0.6 using the Cohen [kappa] (Figure 7, a) versus 12 using the Gwet [AC.sub.1] (Figure 7, b).
An unbiased estimate of global interrater agreement
. Educational and Psychological Measurement, 13164416654740.
The interrater agreement
on the classification of potential eligible extrasystoles was "moderate" to "strong" by common standards [17, 18].
To determine interrater agreement
and accuracy of the triage, 12 nurses working at the ED independently triaged 10 different written case scenarios.
The authors report adequate levels of interrater agreement
for the CPRSR (r = .71, p = .001) and acceptable levels of concurrent criterion-related validity, in that scores have been found to be significantly correlated (r = .64) with scores on the Beck Scale for Suicidal Ideation (Beck & Steer, 1991).