|
Inter-rater Reliability of Alignment between Science Items and Indices |
|---|---|
| รหัสดีโอไอ | |
| Creator | Budsayarat Janprasert |
| Title | Inter-rater Reliability of Alignment between Science Items and Indices |
| Contributor | Nattaporn Lawthong, Sungworn Ngudgratoke |
| Publisher | Centre for Education Innovation, Print and Online Media |
| Publication Year | 2563 |
| Journal Title | Journal of Education Studies, Chulalongkorn University |
| Journal Vol. | 48 |
| Journal No. | 3 |
| Page no. | 144-163 |
| Keyword | alignment, inter-rater reliability, Fleiss’ kappa statistic, intra-class |
| URL Website | https://so02.tci-thaijo.org/index.php/EDUCU |
| Website title | Journal of Education Studies, Chulalongkorn University |
| ISSN | 2651-2017 (Online) |
| Abstract | This research aimed to 1) examine the inter-rater reliability of alignment between science items and indices of science subjects in junior secondary school education, and 2) evaluate the alignment between science items and indices of science subjects in junior secondary school education. Research subjects were 1,089 science test items used in junior secondary schools, chosen by using a multi-stage random sampling procedure. Analysis relied on 20 expert panelists to evaluate the alignment. The data were analyzed for inter-rater reliability by Fleiss’ kappa statistic and the intra-class correlation (ICC), and mean scores of alignment between science items and indices were calculated. The findings reveal that 1) in the cognitive complexity evaluation part, there was good inter-rater reliability, as demonstrated by the Fleiss’ kappa statistic (Kf = 0.510), 2) in the evaluation of alignment between science test items, there was excellent inter-rater reliability, demonstrated by the intra-class correlation (ICC = 0.954, Sig. = .000), and 3) 92.93 percent of items aligned with the specified indices, with mean scores of 3.20-4.00. |
