The evaluation and enhancement of case driven diagnostic advice systems : a study in three domains
Relevant literature has been reviewed regarding the performance, implementation and evaluation of computer based medical decision support systems. The diagnostic performance of five simple case driven acute chest pain advice systems, have been compared using a standardized set of clinical records. A Bayesian inference model demonstrated superiority over two derived by logistic regression. Small data set flow charts performed well but both relied upon the use of expert opinion. A Bayesian acute abdominal pain diagnostic advice system has been evaluated in a clinical trial. Standardized data collection improved the diagnostic performance of doctors. In practice, the computer system offered little additional user benefit. From further tests in primary care, it was concluded that, whereas general practitioners might enhance their performance by using data collection sheets, paramedics might benefit through direct use of the computer. DERMIS is a new dermatology primary care diagnostic advice system. Components include a database derived from 5203 prospectively collected clinical records, a user interface, and an enhanced Bayesian inference model incorporating combined frequency estimates, expert beliefs and rationalized end-point groups. On laboratory testing, the diagnostic accuracy of DERMIS was 83%. The correct diagnosis appeared in the top three, of a possible 42 disease list on 97% of occasions. In a semi-field trial of DERMIS involving 49 general practitioners, doctors did not always collect the same information as a dermatologist but were able to significantly increase their chance of making a correct diagnosis through use of the computer system. It has been concluded that although implementation of DERMIS might well increase general practitioner diagnostic accuracy and lead to improvements in the management of skin disease in primary care, rates of referral for specialist opinion might not be affected unless standard management plans are adopted.