Abstention-Aware Message Routing for Patient Portal Safety Review in Pediatric Primary Care

Abstract

Patient portal messaging has become a routine part of pediatric primary care, but clinics often struggle to route messages quickly to the right staff member. A message about fever, medication access, school forms, vaccine records, or worsening breathing may arrive through the same digital channel, even though each requires a different workflow. Automated message routing can help reduce delay, but unsafe routing may occur when a model assigns confident labels to vague, emotionally charged, or clinically incomplete messages. This paper reports an empirical study of abstention-aware message routing for pediatric patient portals. We assembled 94,360 de-identified portal messages from four pediatric primary care networks and manually adjudicated 12,480 messages into seven routing categories: routine administrative, scheduling, medication refill, vaccine or form request, nurse clinical review, same-day clinician review, and urgent safety escalation. A transformer classifier was compared with a calibrated abstention-aware routing model that separates clear routing decisions from messages requiring human intake. The proposed model improved macro F1 from 0.742 to 0.817, reduced unsafe under-routing from 5.8% to 2.4%, and lowered urgent-message miss rate from 9.6% to 3.9%. It abstained on 13.7% of messages, most often when symptoms were underspecified or when administrative and clinical requests were mixed. Prospective-style temporal testing showed smaller degradation than standard classification. The results suggest that portal triage models should be judged not only by routing accuracy, but also by their ability to recognize when a message is not safe to route automatically.