Utilizing User-Centered EHR Design for Systematic Deep Brain Stimulation Data Collection (2024)

Abstract

This project aims to assess usability and acceptance of a customized Epic-based flowsheet designed to streamline the complex workflows associated with care of patients with implanted Deep Brain Stimulators (DBS). DBS patient care workflows are markedly fragmented, requiring providers to switch between multiple disparate systems. This is the first attempt to systematically evaluate usability of a unified solution built as a flowsheet in Epic. Iterative development processes were applied, collecting formal feedback throughout. Evaluation consisted of cognitive walkthroughs, heuristic analysis, and ‘think-aloud’ technique. Participants completed 3 tasks and multiple questionnaires with Likert-like questions and long-form written feedback. Results demonstrate that the strengths of the flowsheet are its consistency, mapping, and affordance. System Usability Scale scores place this first version of the flowsheet above the 70th percentile with an ‘above average’ usability rating. Most importantly, a copious amount of actionable feedback was captured to inform the next iteration of this build.

Introduction

Deep Brain Stimulation (DBS) is now an established treatment for Parkinson’s disease, essential tremor and dystonia, and it holds potential to treat numerous other neurological and neuropsychiatric disorders. This modality has seen rapid adoption accompanied by a quickly growing industry; there are multiple manufacturers of DBS- devices, and each manufacturer provides their own propriety data entry platforms, interfaces, and suggested workflow. Although DBS products may use similar data elements and track similar metrics, the totality of this data is not available in a single source. Patient visits are therefore complicated, requiring the capture of diverse heterogeneous data about the person as well as device specifications, performance variables and stimulation parameters. DBS providers examine data from these devices and from ongoing non-DBS treatments, so they must switch between multiple platforms and different applications in addition to the Electronic Medical Record (EMR) to find critical information about patient care.

These complexities are multiplied by frequent patient visits, as there are numerous ongoing adjustments to optimize DBS therapy. Furthermore, since DBS is already the most frequently performed surgical procedure for the treatment of advanced Parkinson’s Disease¹, the volume of patients is expected to increase. Specialized multidisciplinary care centers that focus on DBS are already being formed; in New York, The Mount Sinai Health System now hosts the Nash Family Center for Advanced Circuit Therapeutics (C-ACT), to facilitate coordinated and comprehensive care. This comprehensive treatment model is involved in all stages of DBS therapy, including patient screening, imaging- guided surgical planning, implantation surgery, intraoperative testing, therapeutic parameter selection, ongoing testing and optimization, and long-term management with ongoing medication adjustments and rehabilitation throughout the treatment trajectory for each patient.

Unfortunately, excitement about DBS as a modality is somewhat tempered, owing to its complex and fragmented workflows. Providers - including physicians, PAs, nurses, and neurosurgeons – must endure multiple points of data entry across disparate systems and even on paper. Retrieving information is equally complex, as there is no one ‘source of truth’. This presents an increased risk for errors in patient care, and it is detrimental to any underlying research. Moreover, this contributes to significant documentation-related stress, adding to frustration, decreasing job satisfaction, and ultimately leading to burnout. Usability analysis is crucial to resolve these issues in this nascent workflow.

An effective solution to streamline the process may be to create a single ‘source of truth’. As the most comprehensive patient data already exists in the EMR, we chose this to be the single point of entry and reference for our staff. There are no pre-existing tools in the EMR that fit the needs of this specialty service. This project represents the first attempt to integrate all components of the DBS workflow into a single source by following user- centered design principles throughout all iterations of development.

Methods – System Design

User-centered design principles guided development at all phases.

Initial specifications were obtained through meetings with stakeholders and one-on-one sit downs with experts; these formative activities defined users, user needs, and basic workflows². This feedback shaped the first functional version of the flowsheet (Figure 1).

The flowsheet was built in Epic 2019 with the standard interface of this build. A ‘table of contents’ that displays multiple groups arranged in a predicted workflow is visible on the left side of the screen. Each group corresponds with a different component of the patient encounter; many of these groups represent the disparate systems we seek to unite into a single interface. For example, the ‘Device Settings’ groups are intended to record DBS device- specific facts such as voltage, pulsewidth, and impedance, which are traditionally stored on a manufacturer-supplied device. Subsequent groups represent surveys such as the UPDRS or Y-BOCS, which capture information from later in the patient encounter.

Explanations for the fields (called ‘Row Information’) are visible on the right side of the screen. Field contents can be restricted to certain values relevant to that row of the flowsheet. Some fields require the entry of free text, whereas others require users to pick an item from a predefined list.

Methods - Study Design

The research assistant prepared a workstation already logged in to an Epic 2019 training environment with a mock patient encounter open and the flowsheet present on the screen. Upon sitting down at the workstation, the participant was presented with a standardized packet of surveys and instructions. Participants first filled out a demographic survey before reading the instruction packet and then performing 3 tasks.

Task 1 prompted the user to enter the current date into the flowsheet. This task was relatively simple and could be completed with few actions. Task 2 was progressively more complex: the participant would enter mock data into multiple fields, and could refer to the row information on screen for guidance. Task 3 expanded on this same format, requiring users to find the ‘Settings’ section of the flowsheet, which would cascade open upon entering data.

Participants were instructed to ‘think aloud’ during each task, and participants’ comments were recorded by the research assistant. Participants were prompted to keep talking if they stopped.

At the completion of each task, subjects were asked to grade that task on a scale of 1 to 5 using a 3-item Task Self- Assessment survey (Figure 2) which included the following questions: 1) How difficult or easy was it to complete this task? 2) How satisfied are you with using this application/system to complete this task? 3) How would you rate the amount of time it took to complete this task?

Upon completion of all tasks, participants filled out a heuristic evaluation (Figure 3) presented as a short 6- item survey based on Design Principles for Usability³, with Likert-like responses.

Finally, participants were given an exit survey, and the System Usability Scale (SUS).

Results

12 subjects participated in this phase of the formal usability evaluation.

All tasks were successfully completed by all 12 participants, and there were no requests for help. On average, Task 1 required the least amount of time to complete, whereas Task 3 required the most time (Table 1).

Task Self-Assessment ratings were highest for Task 1, and the lowest for Task 3 (Table 2).

Heuristics scores were highest for Consistency, Mapping, and Affordance. Scores for Constraints were lowest (Table 3).

The System Usability Scale (SUS) mean of 76.7 indicates an ‘above average’ usability, placing this flowsheet above the 70th percentile.

Discussion

DBS patient encounters are complex, and a single visit involves capturing both device data and other neuropsychiatric data such as movement disorder scales and depression scales, acquired by multiple caregivers from various specialties. Providers’ original workflow involved switching between multiple platforms within the same visit, repeatedly entering secure log in credentials,

identifying the correct patient, opening the correct encounter, then finding the relevant information and documentation. This complexity creates opportunity for errors of commission and errors of omission², but it also contributes to EMR frustration. As clinician dissatisfaction and burnout has now reached epidemic proportions⁴, it is incumbent on us to perform formal usability evaluations on our tools, to reduce the frustrations contributing to this crisis.

Flowsheet design began with interviews of the experts and stakeholders during one-on-one sit downs. These encounters outlined the extent of the data that needed to be captured in a single source. Since DBS patient care involves an interdisciplinary team across multiple settings, the flowsheet would need to capture extraordinarily diverse data. Multiple mock-ups and static images were brought to the sit-down sessions, and these were useful to shape the overall design and feature set. These versions became the model to build the first functional flowsheet.

To best capture actionable feedback about this first version, a structured evaluation with both quantitative and qualitative elements was applied. Testing took place in a Playground environment that functionally matched the Production version. Study participants were chosen from the pool of clinicians and support staff who regularly need to access, record, and retrieve information in patients’ health records.

Cognitive walkthroughs were chosen since they evaluate how well the interface supports “exploratory learning,” or how well the first-time user can perform a task without formal training; they may uncover errors in design that interfere with task completion, and they explain mismatches between the users’ and the designers’ conception of a task⁵. When coupled with Think-aloud Technique, this combination provided copious feedback.

Tasks were selected for scenario-based testing and arranged in order of increasing complexity, appropriate for the end-user role of Clinician interviewing a patient. The first task was relatively simple, requiring few actions and taking little time, whereas subsequent tasks require the user to enter multiple fields and to explore distant parts of the flowsheet. Task #3 required users to enter information in a particular sequence, to trigger a cascade; certain rows that were hidden to reduce visual clutter would appear when users enter data in a select field. More complicated tasks were expected to require more time to complete, and this matches the Task Performance Summary results. It also offered more time for Think-aloud feedback during the more complex tasks.

Think-aloud feedback captured by the participants during their tasks yielded multiple action points (Table 4).

Although the general workflow was outlined prior to development during formative sit-downs, clinical staff quickly discovered avenues for further optimization.

Users were particularly sensitive to row information (Figure 4) and row restrictions, when applicable.

Heuristic evaluation also offered valuable insight to the users’ experience with the flowsheet interface. This inspection method was chosen since it may prospectively uncover both major and minor problems with the user interface, indicate severity, and - importantly - suggest solutions. Additionally, it may capture issues that are missed by other methods⁶. For this study, a quick 6-item survey based on Norman’s Design Principles of Usability was provided to participants. Lowest scores for ‘Constraints’ and ‘Visibility’ emphasize the importance of building row restrictions (Figure 5) and an orderly table of contents, respectively. The high scores for Consistency, Mapping, and Affordance reflect the strengths of this flowsheet interface.

System Usability Scale scores were normalized and calculated in the usual way (Figure 6). The SUS average of 76.7 places this first iteration of the flowsheet above the 70th percentile. While this confers an ‘above average’ usability rating, it may simply reflect Epic 2019’s general flowsheet design. Users are already familiar with basic flowsheets and how to navigate them. However, this serves as a useful baseline for future builds, which will need to accommodate more functionality and capture more complex data.

Overall, the results from this first version of the flowsheet highlight the importance of following formal usability practices during design, such as beginning with a formative evaluation and then engaging the target audience for structured feedback. Baseline metrics and actionable data are the crucial results from this stage of development. Quantitative and qualitative measures (the heuristic evaluation and the Think-aloud, in particular) provided valuable insights that will lead to a redesign of the flowsheet.

Conclusion

This is the first attempt to systematically evaluate the usability of an integrated DBS flowsheet. Now that

References