Article Instance

HTTP 200 OK
Allow: GET
Content-Type: application/json
Vary: Accept

{
    "pk": 10152,
    "title": "Inter-rater Agreement of End-of-shift Evaluations Based on a Single Encounter",
    "subtitle": null,
    "abstract": "Introduction\n End-of-shift evaluation (ESE) forms, also known as daily encounter cards, represent a subset of encounter-based assessment forms.  Encounter cards have become prevalent for formative evaluation, with some suggesting a potential for summative evaluation.  Our objective was to evaluate the inter-rater agreement of ESE forms using a single scripted encounter at a conference of emergency medicine (EM) educators.\n \n \n \nMethods\n Following institutional review board exemption a scripted video simulating an encounter between an intern and a patient with an ankle injury was created.  That video was shown during a lecture at the Council of EM Residency Director’s Academic Assembly with attendees asked to evaluate the “resident” using 1 of 8 possible ESE forms randomly distributed.  Descriptive statistics were used to analyze the results with Fleiss’ Kappa to evaluate inter-rater agreement.\n \n \n \nResults\n Most of the 324 respondents were leadership in residency programs (66%).  Despite that, few individuals (5%) felt they were experts in assessing residents based on EM milestones.  Fleiss’ Kappa ranged from 0.157 - 0.308 and did not perform much better in two post-hoc subgroup analyses.\n \n \n \nConclusion \n \nThe kappa ranges found show only slight to fair inter-rater agreement and raise concerns about the use of ESE forms in assessment of EM residents.  Despite limitations present in this study, these results and a lack of other studies on inter-rater agreement of encounter cards should prompt further studies of such methods of assessment.  Additionally, EM educators should focus research on methods to improve inter-rater agreement of ESE forms or other evaluating other methods of assessment of EM residents.",
    "language": "en",
    "license": {
        "name": "Creative Commons Attribution 4.0",
        "short_name": "CC BY 4.0",
        "text": "Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.\r\n\r\nNo additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.",
        "url": "https://creativecommons.org/licenses/by/4.0"
    },
    "keywords": [
        {
            "word": "Encounter Cards"
        },
        {
            "word": "Interrater agreement"
        },
        {
            "word": "Emergency Medicine Milestones"
        },
        {
            "word": "Assessment"
        }
    ],
    "section": "Education",
    "is_remote": true,
    "remote_url": "https://escholarship.org/uc/item/64v6d72b",
    "frozenauthors": [
        {
            "first_name": "Steven",
            "middle_name": "Jay",
            "last_name": "Warrington",
            "name_suffix": "",
            "institution": "Kaweah Delta Medical Center",
            "department": "None"
        },
        {
            "first_name": "Michael",
            "middle_name": "S",
            "last_name": "Beeson",
            "name_suffix": "",
            "institution": "Akron General Medical Center; Northeast Ohio Medical University",
            "department": "None"
        },
        {
            "first_name": "Amber",
            "middle_name": "",
            "last_name": "Bradford-Saffles",
            "name_suffix": "",
            "institution": "Akron General Medical Center; Northeast Ohio Medical University",
            "department": "None"
        }
    ],
    "date_submitted": "2016-08-09T02:29:55Z",
    "date_accepted": "2016-08-09T02:29:55Z",
    "date_published": "2017-03-08T01:08:38Z",
    "render_galley": null,
    "galleys": [
        {
            "label": "",
            "type": "pdf",
            "path": "https://journalpub.escholarship.org/westjem/article/10152/galley/5556/download/"
        }
    ]
}