William A. Hyman
Professor Emeritus, Biomedical Engineering
Texas A&M University, w-hyman@tamu.edu
Read other articles by this author
One form of Clinical Decision Support (CDS) software is that which suggests one or more diagnoses or other important assessments based on some set of patient specific data. In some cases such software is considered by the FDA to be a regulated medical device because of the simple reason that it meets the official definition of a medical device, even though the software is not otherwise part of a hardware system and does not by itself do anything to the patient.
A case in point is the recent FDA classification of “Computerized cognitive assessment aid for concussion” software as a Class II medical device. Such software provides an indication of the current level of cognitive function in response to concussion but does not identify the presence or absence of concussion. The FDA identified the primary safety issue with such software as the risk of providing either a positive or negative false result. This risk exists whether or not the clinician can and will make their own assessment. Class II devices are (unless exempt) subject to the FDA Premarket Notification route to market, commonly known as requiring a 510(k). The FDA specifically noted that it was not exempting such products from the 510(k) requirements. Devices that successfully traverse the 510(k) route are said to be “cleared” and specifically are not “approved”.
According to the FDA the development of the software must include a detailed description of the software requirements and design specifications along with verification, validation, and hazard analysis. Verification here means that the software does what the specifications say it should do while validation means that the end product meets the needs of the clinical user. Or, as has long been said for hardware (and requiring careful reading), verification means did you build the thing right, while validation means did you build the right thing. In this regard, I was intrigued to read that the cause of a recent surgical robot recall was a software “anomaly” which in this case was failure to prevent unexpected movement. Anomaly is another great software word which means the software didn’t do what it was supposed to do. Of course, this anomaly was going to be fixed with an “upgrade”.
The requirements for clinical evaluation of cognitive assessment software should be in general applicable to all CDS. There must be performance data that demonstrates how the device functions in the interpretation of the current level of cognitive function in an individual that has recently received an injury that causes concern about a possible concussion. Associated pre-market testing must include the evaluation of the device output and clinical interpretation, the device test-retest reliability, and the validity of the cognitive assessments. In addition, there must be a description of how the normative database was constructed including how the clinical workup established a “normal” population and the establishment of inclusion and exclusion criteria. Also noteworthy is whether or not the normative database was adjusted due to differences in age and gender. This is a particular case of the more general CDS question of to whom does the software apply and to whom does it not.
Because the cognitive assessment software is judged to be not safe for use except under the supervision of a practitioner licensed by law, it is a prescription device and must satisfy associated requirements. Who is a licensed practitioner in this regard is a function of state law and may include people other than physicians.
This type of software is an example of regulated stand-alone software which is a medical device. To me this has clear parallels to other patient data driven assessment or diagnostic software including some types of CDS. Even though such software cannot directly do anything to the patient, and requires a healthcare intermediary, it clearly has the potential to be cause harm, even if indirectly. That’s is why questions of requirements, design, testing and proper definition of applicable populations are important, and why regulation is necessary. While some might agonize about regulatory restraints on innovation, I agonize over people who don’t want to do the work necessary to create products that actually do what they proport to do.