How Tired Docs Write: A Tech Take

Doctors jotting down notes after seeing patients might not just be recording medical details. These notes could also hint at something else: how tired the doctor is.

The Study

A recent study dug into this idea using a whopping 129,228 emergency room visits. The goal? To train a computer model to spot notes written by doctors who are likely running on empty.

Methodology

• Target Group: Doctors who had worked at least five out of the last seven shifts in the emergency department.
• Model Training: Recognize patterns in these notes.
• Additional Factors: Signs of fatigue in other high-pressure situations, like overnight shifts or when there were too many patients to handle.

Findings

• Decision Quality: When the model flagged a note as potentially written by a tired doctor, the decisions made for that patient weren't as good.
• Example: The chances of correctly identifying a heart attack dropped by 19% for every step up in predicted fatigue.
• Predictability: Notes from tired doctors had a certain predictability. The next word was easier to guess based on what came before.
• LLMs Comparison: Notes generated by LLMs were 74% more likely to show signs of fatigue compared to those written by actual doctors.

Implications

This raises a big question: Are LLMs introducing biases or errors we don't even know about yet?