{"id":474,"date":"2017-04-24T13:02:46","date_gmt":"2017-04-24T20:02:46","guid":{"rendered":"http:\/\/test-inside.ewu.edu\/successfultransitions\/?page_id=474"},"modified":"2024-03-28T16:57:08","modified_gmt":"2024-03-28T23:57:08","slug":"final-consensus-of-reliability","status":"publish","type":"page","link":"https:\/\/test-inside.ewu.edu\/successfultransitions\/mathematics-inquiries\/algebra-a\/final-consensus-of-reliability\/","title":{"rendered":"Final Consensus of Reliability"},"content":{"rendered":"<p>Inter-rater reliability for the fourth design cycle was generally high, although there still appear to be areas where definitions need to be refined. Overall, the observers had an average agreement of 81.39%. Areas of strong agreement where in the \u201cTeacher is Resource\u201d and \u201cStudent uses Student\u201d codes. These codes were the most common in all videos and were the most heavily negotiated (Figures 11, 12).<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-585\" src=\"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-content\/uploads\/sites\/111\/2017\/04\/Figure11-300x14.png\" alt=\"\" width=\"300\" height=\"14\" \/><\/p>\n<p>Figure 11. Sample comparison of \u201cTeacher is Resource\u201d for two observers, showing 95.6% agreement.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-578\" src=\"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-content\/uploads\/sites\/111\/2017\/04\/Figure12-300x14.png\" alt=\"\" width=\"300\" height=\"14\" \/><\/p>\n<p>Figure 12. Sample comparison of \u201cStudent uses Student\u201d for two observers, showing 94.6% agreement<\/p>\n<p>&nbsp;<\/p>\n<p>There were some areas of disagreement in \u201cStudent uses Teacher\u201d but the disagreements appear to be durational. Overall, observers agreed on the beginnings of \u201cStudent uses Teacher\u201d events.\u00a0 The difference appears to be whether or not the observer coded the teacher\u2019s response as \u201cStudent uses Teacher\u201d or only the incidence of the student asking the question as \u201cStudent uses Teacher\u201d (Figure 13).<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-579\" src=\"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-content\/uploads\/sites\/111\/2017\/04\/Figure13-300x14.png\" alt=\"\" width=\"300\" height=\"14\" \/><\/p>\n<p>Figure 13. Sample comparison of \u201cStudent uses Teacher\u201d showing the longer codes due to including the teacher\u2019s response. This particular example has 71.1% agreement.<\/p>\n<p>&nbsp;<\/p>\n<p>The greatest disagreement between observers was for the rarely occurring codes, \u201cTeacher points to Resource\u201d and \u201cStudent uses Object.\u201d A sample of the disagreement on \u201cTeacher points to Resource\u201d can be found in Figure 14. Only one observer coded \u201cStudent uses Object\u201d in the final run. It appears that these rare codes still need to be better negotiated and defined.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-580\" src=\"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-content\/uploads\/sites\/111\/2017\/04\/Figure14-300x14.png\" alt=\"\" width=\"300\" height=\"14\" \/><\/p>\n<p>Figure 14. Sample comparison of \u201cTeacher points to Resource\u201d showing low agreement. This particular example shows 57.7% agreement. Counting only the durations coded, there is 5% agreement given the existence of a code.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Inter-rater reliability for the fourth design cycle was generally high, although there still appear to be areas where definitions need to be refined. Overall, the observers had an average agreement of 81.39%. Areas of strong agreement where in the \u201cTeacher is Resource\u201d and \u201cStudent uses Student\u201d codes. These codes were the most common in all &#8230; <a title=\"Final Consensus of Reliability\" class=\"read-more\" href=\"https:\/\/test-inside.ewu.edu\/successfultransitions\/mathematics-inquiries\/algebra-a\/final-consensus-of-reliability\/\" aria-label=\"Read more about Final Consensus of Reliability\">Read more<\/a><\/p>\n","protected":false},"author":8580,"featured_media":0,"parent":35494,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"wpo365_audiences":[],"wpo365_private":false,"footnotes":""},"class_list":["post-474","page","type-page","status-publish"],"_links":{"self":[{"href":"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-json\/wp\/v2\/pages\/474"}],"collection":[{"href":"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-json\/wp\/v2\/users\/8580"}],"replies":[{"embeddable":true,"href":"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-json\/wp\/v2\/comments?post=474"}],"version-history":[{"count":1,"href":"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-json\/wp\/v2\/pages\/474\/revisions"}],"predecessor-version":[{"id":35613,"href":"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-json\/wp\/v2\/pages\/474\/revisions\/35613"}],"up":[{"embeddable":true,"href":"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-json\/wp\/v2\/pages\/35494"}],"wp:attachment":[{"href":"https:\/\/test-inside.ewu.edu\/successfultransitions\/wp-json\/wp\/v2\/media?parent=474"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}