Decision Trees For Differential Diagnosis Pdf Writer
Request PDF on ResearchGate Differential Diagnosis of Erythemato-Squamous Diseases Using Ensemble of Decision Trees The differential diagnosis of erythemato-squamous diseases (ESD) in. Algorithms & charts Over 200 diagnosis and treatment algorithms, including online-only exclusives help you to diagnose clinical signs and symptoms, and treatment of a variety of clinical symptoms. All Algorithms & Charts. The aim of this study was to develop and explore the diagnostic accuracy of a decision tree derived from a large real-life primary care population. Data from 9297 primary care patients (45% male, mean age 53±17 years) with suspicion of an obstructive pulmonary disease was derived from an asthma/chronic obstructive pulmonary disease (COPD) service where patients were assessed using spirometry.
Associated Data
(0.03 MB DOC)
(0.06 MB DOC)
Abstract
Background
Dengue is re-emerging throughout the tropical world, causing frequent recurrent epidemics. The initial clinical manifestation of dengue often is confused with other febrile states confounding both clinical management and disease surveillance. Evidence-based triage strategies that identify individuals likely to be in the early stages of dengue illness can direct patient stratification for clinical investigations, management, and virological surveillance. Here we report the identification of algorithms that differentiate dengue from other febrile illnesses in the primary care setting and predict severe disease in adults.
Methods and Findings
A total of 1,200 patients presenting in the first 72 hours of acute febrile illness were recruited and followed up for up to a 4-week period prospectively; 1,012 of these were recruited from Singapore and 188 from Vietnam. Of these, 364 were dengue RT-PCR positive; 173 had dengue fever, 171 had dengue hemorrhagic fever, and 20 had dengue shock syndrome as final diagnosis. Using a C4.5 decision tree classifier for analysis of all clinical, haematological, and virological data, we obtained a diagnostic algorithm that differentiates dengue from non-dengue febrile illness with an accuracy of 84.7%. The algorithm can be used differently in different disease prevalence to yield clinically useful positive and negative predictive values. Furthermore, an algorithm using platelet count, crossover threshold value of a real-time RT-PCR for dengue viral RNA, and presence of pre-existing anti-dengue IgG antibodies in sequential order identified cases with sensitivity and specificity of 78.2% and 80.2%, respectively, that eventually developed thrombocytopenia of 50,000 platelet/mm3 or less, a level previously shown to be associated with haemorrhage and shock in adults with dengue fever.
Conclusion
This study shows a proof-of-concept that decision algorithms using simple clinical and haematological parameters can predict diagnosis and prognosis of dengue disease, a finding that could prove useful in disease management and surveillance.
Author Summary
Dengue illness appears similar to other febrile illness, particularly in the early stages of disease. Consequently, diagnosis is often delayed or confused with other illnesses, reducing the effectiveness of using clinical diagnosis for patient care and disease surveillance. To address this shortcoming, we have studied 1,200 patients who presented within 72 hours from onset of fever; 30.3% of these had dengue infection, while the remaining 69.7% had other causes of fever. Using body temperature and the results of simple laboratory tests on blood samples of these patients, we have constructed a decision algorithm that is able to distinguish patients with dengue illness from those with other causes of fever with an accuracy of 84.7%. Another decision algorithm is able to predict which of the dengue patients would go on to develop severe disease, as indicated by an eventual drop in the platelet count to 50,000/mm3 blood or below. Our study shows a proof-of-concept that simple decision algorithms can predict dengue diagnosis and the likelihood of developing severe disease, a finding that could prove useful in the management of dengue patients and to public health efforts in preventing virus transmission.
Introduction
Dengue fever/dengue haemorrhagic fever (DF/DHF) is a re-emerging disease throughout the tropical world. The disease is caused by four closely related dengue viruses, which are transmitted by the Aedes mosquitoes, principally Aedes aegypti. DHF and dengue shock syndrome (DSS) represent the severe end of the disease spectrum, which if not properly managed, would result in significant mortality. The pathophysiology of severe DHF and DSS is characterized by plasma leakage as a result of alteration in microvascular permeability . There is as yet no vaccine or specific antiviral therapy for DF/DHF and management of cases remains largely supportive [3].
Dengue illness is often confused with other viral febrile states, confounding both clinical management [4]– and disease surveillance for viral transmission prevention [7]. This difficulty is especially striking during the early phase of illness, where non-specific clinical symptoms and signs accompany the febrile illness [4]. More definitive symptoms, such as retro-orbital pain, and clinical signs, such as petechiae, do not appear until the later stages of illness, if at all. Definitive early dengue diagnosis thus requires laboratory tests and those suitable for use at this stage of illness are either costly, such as RT-PCR for dengue; not sufficiently rapid, such as virus isolation; or undergoing field trials, such as ELISA for NS1 protein of dengue virus ,. Furthermore, many dengue endemic places lack the necessary laboratory infrastructure or support [7] and thus a scheme for reliable clinical diagnosis, using data that can be obtained routinely, would be useful for early recognition of dengue fever, not only for case management but also for dengue surveillance. The current World Health Organization (WHO) scheme for classifying dengue infection (Table S1) makes use of symptoms and signs that are often not present in the first few days of illness, and thus not a guide for early diagnosis, but are instead designed for monitoring disease progression for clinical management of the severe DHF/DSS. Other attempts at identifying clinical features for the diagnosis of dengue disease have made use of univariate or multivariate analysis of clinical symptoms and signs, haematological or biochemical parameters ,. Although such studies provide a list of symptoms and signs that could be associated with dengue disease, how these should be applied for clinical diagnosis is not apparent. Evidence-based triage strategies that identify individuals likely to have dengue infection in the early stages of illness are needed to direct patient stratification in clinical investigations, management and healthcare resource planning.
To address this goal, we show here that a decision tree approach can be useful to develop an intuitive diagnostic algorithm, using clinical and haematological parameters, that is able to distinguish dengue from non-dengue disease in the first 72 hours of illness. We also demonstrate a proof-of-concept that such an approach can be useful for early dengue disease prognostication.
Materials and Methods
Patients and clinical methods
Ethical considerations
The study protocol was approved by each organization's institutional review board. Patient enrolment, clinical and epidemiological data collection within the National Healthcare Group, Singapore was approved by the NHG IRB (DSRB B/05/013). Patient enrolment, clinical and epidemiological data collection in Dong Thap Hospital was approved by the hospital scientific and ethical committee as well as the Oxfordshire Tropical Research Ethical Committee, UK. Enrolment of study participants was conditional on appropriate informed consent administered by a study research nurse. All biological materials collected were anonymized after completion of demographic and clinical data collection.
Screening and recruitment
The protocol for patient recruitment in Singapore (the early dengue infection and outcome (EDEN) study) was described previously . Adult patients (age >18 years) presenting at selected primary care polyclinics within 72 hours of onset of acute febrile illness and without rhinitis or clinically obvious alternative diagnoses for fever were eligible for study inclusion. Upon consent, anonymized demographic, clinical and epidemiological information were collected on a standardized data entry form on 3 occasions: 1–3 days post-onset of fever (1st visit), 4–7 days post-onset of fever (2nd visit) and 3–4 weeks post-onset of fever (3rd visit). Venous blood was also collected for haematological, virological and serological analyses at every visit. Remaining serum and blood were anonymized and stored at −80°C until use. The list of parameters monitored in this study is shown in the supplementary Table S2.
Children or adults in whom there was a clinical suspicion of dengue were recruited within 72 hours of illness onset in Dong Thap Hospital, Vietnam. Blood samples were collected for diagnostic investigations at study enrolment and again at hospital discharge. Clinical data were collected daily on standard case record forms.
Laboratory Methods
Haematology
A full blood count was performed on anticoagulated whole blood collected at all time points. A bench-top, FDA-approved haematocytometer was used for this application (iPoch-100, Sysmex, Japan). Calibration by internal and external QC controls was also performed on a regular basis.
Serology and antigen detection
IgM and IgG antibodies against dengue virus were detected using commercially available ELISAs (PanBio, Brisbane, Australia) according to manufacturer's instructions.
Reverse-transcription polymerase chain reaction (RT-PCR)
RNAs were extracted from the first serum portion or virus culture supernatant using QIAamp Viral RNA mini kit (Qiagen, Hilden, Germany) according to the manufacturer's protocol. RT-PCR to detect dengue viral RNA was carried out using a set of generic pan-dengue primers that targeted the 3′ non-coding region of dengue viruses as previously described . Results were analysed with LightCycler software version 3.5 (Roche Diagnostics, Mannheim, Germany). Reactions with high crossover threshold (Ct) value or ambiguous melting curve results were analysed by electrophoresis on a 2% agarose gel, to confirm presence of product of the correct size. RNA extracted from previously obtained clinical isolates, namely dengue-1 (S144), dengue-2 (ST), dengue-3 (SGH) and dengue-4 (S006), propagated in C6/36 cell cultures were included as external control in every RT-PCR run.
Decision tree analyses for disease modelling
Classifier modelling
The C4.5 decision tree classifier [14] software Inforsense (InforSense Ltd., London, UK) was used. A standard pruning confidence of 25% was used to remove branches where the algorithm was 25% or more confident so as to avoid having specific branches that would not be representative for generalisation. This prevents over-fitting of the data.
The parameter ‘minimal cases’ represents a stopping criterion for further partition of the data at specific decision nodes. Tree growing at a specific decision node was stopped when at least one class had equal or less cases than the ‘minimal cases’. This prevents the tree from sub-dividing into overly specific nodes which have little supporting data. Choosing an appropriate value for ‘missing cases’ was done using k-fold cross validation (see below). Briefly, various decision trees with different ‘minimal cases’ were calculated and the value resulting in the tree with the best performance was chosen.
The calculated algorithms were validated using the k-fold cross-validation approach. This is considered to be a powerful methodology to overcome data over-fitting [15]. Briefly, the original sample was divided into k sub-samples. Each sub-sample was put aside as evaluation data for testing a model, and the remaining k-1 sub-samples were used for training the model. The cross-validation process was repeated k times (folds) and each of the k sub-samples was used once as the validation data. The k results obtained from the k-folds could then be averaged to produce a single estimation of model performance [15]. The fold value was set to k = 10.
To analyse the sensitivity and specificity of the decision algorithm, an averaged receiver-operating characteristic (ROC) curve was constructed. The area under the curve (AUC) serves as an indicator of the overall performance of the algorithm. The algorithms with the highest sensitivity along with a high AUC were selected.
Statistical analysis
All results have been summarized in terms of means and standard deviation for continuous variables using independent sample T-test. Shapiro-Wilk normality test was used to check for non-normally distributed parameters whereby a p value <0.05 indicated that the parameter was unlikely to originate from a normal distribution. Non-normally distributed parameters were log-transformed and rechecked for normality. If the log-transformation still resulted in non-normal distribution, non-parametric Kruskal-Willis test was used for continuous variables whereas Student's t test was exploited for normally distributed continuous variables. For dichotomous variables, Chi-square test was used in case of expected frequencies that were higher than 5, whereas Fisher's exact test was performed when the expected table values were smaller than 5. Cases with missing values were excluded from the analysis and thus, the number of cases used for calculations varied between different covariates. All calculations were performed using Systat for Windows (SYSTAT Software Inc. San Jose, CA). A two-tailed p value <0.05 was considered as statistically significant.
Results
We constructed a decision tree for dengue diagnosis with 1,200 patients with acute febrile illness. Of these, 1,012 were recruited from the EDEN study and 188 from Vietnam. The EDEN cohort consisted of 173 DF, 3 DHF and 836 non-dengue cases while the Vietnam cohort consisted of 168 DHF and 20 DSS cases, resulting in a total of 364 dengue and 836 non-dengue cases used for our diagnostic tree construction.
The resulting diagnostic algorithm is shown in Figure 1. The first splitting parameter is a platelet count of 196,000/mm3 blood or less followed by the total white cell or lymphocyte counts, body temperature, haematocrit or neutrophil count and another platelet count at presentation. The predicted diagnosis is shown in colours, with red indicating probable dengue, brown indicating likely dengue, green indicating likely non-dengue and blue indicating probable non-dengue (Figure 1A). Each of the nodes showed statistical significance in the proportion of dengue and non-dengue cases, with the odds ratio calculated as shown in Figure 1B. The performance of this algorithm is shown in Figure 2. The overall error rate estimated after k-fold cross validation was 15.7%, with a sensitivity and specificity of 71.2% and 90.1%, respectively (Figure 2B).
Decision algorithm for dengue diagnosis.A. Decision algorithm for predicting dengue diagnosis calculated on 1200 patients with data obtained in the first 72 hours of illness. PLT = platelet count; WBC = white blood cell count; T = body temperature; HCT = hematocrit; Lymphocyte = absolute number of lymphocytes; Neutrophil = absolute number of neutrophils. The prediction of the algorithm is shown in colours: Red indicates probable dengue; brown indicate likely dengue; green indicates likely non-dengue and blue indicates probably non-dengue. B. Statistical (chi-square) analysis of splitting criteria performed on each subgroup at the decision nodes. OR = odds ratio; CI = 95% confidence interval.
Performance of the decision algorithm for dengue diagnosis.A. Receiver operating characteristics (ROC) curve for the diagnostic algorithm in predicting dengue positive and dengue negative cases. B. Summary of K-fold (k = 10) cross-validation analysis for the dengue diagnostic algorithm with 2×2 analysis for the algorithm's sensitivity and specificity in dengue diagnosis.
Prediction of disease severity
We also examined if the use of a decision tree would be useful for prognostication.
For the EDEN cohort, we used a platelet count of less than 50,000/mm3 on days 5 to 7 of illness as a marker of severe disease. This level of thrombocytopenia has been shown to be associated with the development of complications such as bleeding and shock in adults –.
Fifteen cases were excluded from this analysis as they were either admitted to private hospitals where access to the clinical information was not available to us, or were foreigners who returned to their country of origin to seek medical treatment. Thus, 161 Singaporean dengue cases were analysed and the pruning confidence was set to 25% with minimal cases defined as 16.
The best performing decision algorithm made used of platelet count, the crossover value (Ct) of the real-time RT-PCR for dengue viral RNA (a marker for viremia levels) and the presence of anti-dengue IgG antibodies, as the first, second and third splitting parameters, respectively (Figure 3). All 3 parameters were obtained from the first visit. All three DHF cases were correctly classified using this algorithm, one into the group with a platelet count of 108,000 mm3 or less, the other two into the group with pre-existing anti-dengue IgG antibodies. The predicted outcome of disease is shown in colours, with red indicating probable severe dengue, brown indicating likely severe dengue, green indicating likely non-severe dengue and blue indicating probable non-severe dengue (Figure 3A). The statistical significance of each node of the algorithm and their odds ratio with severe dengue are shown in Figure 3B. The performance of this algorithm is shown in Figure 4. The overall error rate using k-fold crossover validation analysis was 20.5%, with a sensitivity of 78.2% and specificity of 80.2% (Figure 4B).
Decision algorithm for predicting severe dengue disease.A. Decision algorithm for severity prediction calculated on 169 patients with clinical data obtained at the first visit. PLT = platelet count; Ct = viral load whereby a high Ct-value indicates a low viral load; DV IgG = indicator for primary/secondary infection whereby a positive result indicates a secondary infection. Low = platelet nadir of 50,000/mm3 or less; high = platelet nadir greater than 50,000/mm3. The prediction of the algorithm is shown in colours: Red indicates probably severe dengue; brown indicates likely severe dengue; green indicates likely non-severe dengue and blue indicates probable non-severe dengue B. Statistical (chi square) analysis of splitting criteria performed on each subgroup at the decision nodes. OR = odds ratio; CI = 95% confidence interval.. PLT = platelet count; Ct = crossover threshold value of real-time RT-PCR and indicative of level of viremia; DV IgG = indicator for primary/secondary infection whereby a positive result indicates a secondary infection; OR = odds ratio; CI = confidence interval.
Performance of the decision algorithm for predicting severe dengue disease.A. Receiver operating characteristics (ROC) curve for the algorithm in predicting the development of severe disease among dengue cases. B. Summary of K-fold (k = 10) cross-validation for severity prediction algorithm with 2×2 analysis for the algorithm's sensitivity and specificity in predicting severe dengue disease.
The use of data obtained from the 89 hospitalised cases alone resulted in a very similar decision algorithm, although the AUCs were substantially lower than the above analysis due largely to the smaller dataset. Taken together, these indicate that the prediction algorithm as defined in Figure 3A is stable.
We next examined the clinical outcomes of the patients grouped according to the decision algorithm in Figure 3A. The results are summarised in Table 1. Each of the four groups of patients showed different rates of hospitalisation, duration of hospitalisation and the proportion of clinically severe cases. The latter was defined as patients who met the criteria for DHF; had a systolic blood pressure less than 90 mmHg; a serum transaminase of greater than 1000 which suggests severe liver involvement and who received blood transfusion. The results indicate that statistically significant differences were observed between the groupings as indicated in Table 1.
Table 1
| Prognostic tree grouping | No. of cases hospitalised (%) | Mean number of days hospitalised | SD (days) | No. of clinically severe cases (%)† |
| Probable severe dengue (n = 26) | 25 (96.2) | 5.2 | 1.4 | 10 (38.5) |
| Likely severe dengue (n = 38) | 26 (63.4) | 4.9 | 1.6 | 9 (23.7) |
| Likely non-severe dengue (n = 49) | 27# (55.1) | 3.9* | 1.2 | 4 (8.2)# |
| Probable non-severe dengue (n = 48) | 10** (20.8) | 3.5** | 1.2 | 0# |
†: indicates cases with DHF/SBP<90mmHg/serum transaminase>1000/transfusion.
*: p<0.05 and ** p<0.001 when compared to either probable severe dengue or likely severe dengue. # indicates p<0.05 when compared to the probable severe dengue only.
Discussion
The lack of evidence-based diagnostic algorithm for early dengue diagnosis as well as prognostic triage strategies limits effective patient management, use of healthcare resources and disease surveillance efforts. For instance, over 80% of the total dengue cases in Singapore are admitted for hospitalised care, mostly to monitor for signs of clinical deterioration. Prognostication in the early stages of dengue illness could significantly influence clinical management and the use of healthcare resources, particularly during dengue outbreaks, such as occurred in Singapore in 2005 where up to 8% of all acute hospitals beds available were occupied by dengue cases [19].
To identify the decision algorithms, we have used a C4.5 decision tree classifier, which has several advantages over other statistical tools [14]. Briefly, decision algorithms are in principle simple to understand and are able to handle both nominal and categorical data. Importantly, they are also able to handle missing values, which are commonly encountered in clinical studies. In contrast, logistic regression and discriminant analyses require much more data preparation and appropriate handling of missing values for reliable calculations [15]. Decision algorithms are also easy to interpret, use and validate using common statistical techniques. Importantly, it provides a means to identify parameters that would be significantly associated with disease when analysed in sub-groups but not in the total study population. To our understanding, this is the first time decision tree modelling has been used to identify prognostic markers for dengue disease.
While dengue is predominantly a paediatric disease, dengue in adults has become an increasingly recognised problem, both in dengue-endemic regions , as well as in adult travellers returning from the tropics . The case recruitment in Singapore has thus focused on adult cases. Since the course of disease in all but three of the Singaporean adult cases were consistent with DF instead of DHF, we have included 188 DHF/DSS paediatric and adult cases from Vietnam in order to ensure that the diagnostic algorithm developed here is robust across a spectrum of dengue presentations.
The decision algorithm for the diagnosis of dengue within the first three days of illness made use of a combination of platelet count, total white cell count, body temperature, absolute lymphocyte and neutrophil counts, in sequential order (Figure 1A). Each node of the decision tree has statistically significant odds ratio ranging from 5.9 to 13.8 (Figure 1B).
Although the tree has an optimal combined sensitivity and specificity of 71.2% and 90.3%, respectively, its usage can be adjusted according to the objective in which it is used for. In an outbreak where the aim is to identify all dengue cases for laboratory investigation and clinical follow-up, the tree could be used to exclude dengue cases whereupon all cases except those predicted as probable non-dengue (shown in blue in Figure 1A) are tested for dengue virus. When applied hypothetically to an outbreak similar to that observed in Singapore in 2005 where 29% of the acute febrile cases recruited into our study was dengue, the positive and negative predictive values of the tree are 57.7% and 94.4%, respectively. Conversely, when the dengue prevalence is low as was encountered in our EDEN study between 2006 and August 2007 where only 43 out of 555 (7.7%) cases presenting with acute febrile illness were dengue, increasing the specificity of clinical diagnosis by selecting patients with probable dengue (shown in red in Figure 1A) would result in a positive and negative predictive values of 51.1% and 97.7%, respectively. Such a level of positive predictive value would be useful to guide the selection of patients for virological surveillance, a critical part of any dengue prevention program [7], –.
Upon diagnosis, current dengue management strategies require daily observation for signs of clinical deterioration, particularly for clinical or laboratory evidence of hemorrhage or plasma leakage. In situations of high prevalence of dengue illness, such an approach can quickly overwhelm limited healthcare resources. It would be advantageous to be able to stratify dengue cases for clinical follow-up and management based on the likely outcome of disease. We thus searched for an algorithm that could be used for prognostication. Since the incidence of DHF is low in Singapore and hospitalisation of the dengue cases is subject to variation arising from physician-to-physician differences in decision-making, we have used platelet count nadir of 50,000/mm3 or less at 5 to 7 days after onset of illness as an objective end-point for our analysis. This level of thrombocytopenia has been found to be associated with increased risks of haemorrhage and shock in adults with DF –. We were unable to include the DHF and DSS cases recruited in Vietnam for the tree construction as daily laboratory parameters comparable to those collected for the Singapore cohort were not available.
The decision algorithm for prognostication (Figure 3A) uses the platelet count as the first splitting criteria, followed by the dengue virus genome copy number estimated by real-time RT-PCR as the second splitting criteria for those with platelet count greater than 108,000/mm3 blood. The second splitting criterion is a marker of viral load. Although we have used the Ct value of our real-time RT-PCR in this analysis, it is likely that other parameters that provide estimates for the viral load could be substituted for the viral genome copy numbers. The development of NS1 antigen ELISA that is currently being evaluated in several places could be one such alternative. The third splitting criterion uses the presence of anti-dengue IgG antibody, indicating secondary infection.
Although thrombocytopaenia –, high viremia and presence of pre-existing anti-dengue antibodies – have previously been reported to be associated with severe disease, how these factors should be used clinically for prognostication has never been described. Furthermore, using these parameters singly also presents difficulties since these parameters are dynamic and the window period in which these parameters offer peak predictive values is extremely short . The use of a decision tree approach could thus provide clinicians with an algorithm to guide the evaluation of a panel of critical laboratory parameters and the sequential order these should be considered within the first 72 hours of illness.
Each of the four groups of patients under the decision algorithm for prognosis (Figure 3A) also showed significant differences in clinical outcome (Table 1). While only three cases met all the criteria for DHF according to the WHO dengue classification, the clinical records of another 20 cases showed that they either had a period of hypotension (systolic blood pressure of less than 90mmHg) or severe liver inflammation (liver transaminases>1000), both without documented pleural effusion, ascites or rise in serial hematocrit, or received platelet/blood transfusion. These clinical parameters have been previously observed in severe dengue [15], and we have taken these cases collectively as clinically severe outcomes. Of these 23 cases, 19 (82.6%) were predicted by our tree as either probable severe dengue or likely severe dengue with data obtained in the first three days of illness. Conversely, 91.8% and 100% of the patients in the groups predicted by our tree as either likely non-severe dengue or probable non-severe dengue, respectively, did not show severe clinical outcomes (Table 1).
The use of such a prognostic algorithm could prove useful in segregating patients according to likely clinical outcomes to guide clinical management and follow-up visits. Although our EDEN cohort in Singapore has focused on dengue in the adult population, our findings demonstrate a proof-of-concept that the use of simple haematological and virological parameters is predictive of disease outcome, and can be built upon to develop prognosis-based protocols for dengue case management that begins at the primary healthcare setting.
Our study represents the first to demonstrate that decision algorithms for dengue diagnosis and prognosis can be developed for clinical use. While a large multi-centre prospective study will be needed for these algorithms to be applied globally, our analysis indicates that a decision tree approach can differentiate dengue from non-dengue febrile illness and predict outcome of disease.
Supporting Information
Table S1
Criteria for the classification of DF/DHF and the recommended approach to diagnosis, according to the WHO Guidelines.
(0.03 MB DOC)
Table S2
Parameters and the respective units of measure used in the EDEN study to monitor the recruited cases in all three visits.
(0.06 MB DOC)
Acknowledgments
We are grateful to all the patients and primary care physicians who participated in the study. We thank Edison Liu for his invaluable and constructive comments on the manuscript; Diana Tan and Lay Pheng Lim, our research nurses for their contribution toward patient recruitment and data collection; Nguyen Thi Hong Tham, Tran Thi Thao Uyen and Duong Thi Hue Kien for patient recruitment in Dong Thap Hospital; Hwee Cheng Tan for the dengue virus isolation and typing work; the entire staff at the Singapore Tissue Network for their contribution in sample processing, storage, haematology analysis and data entry.
During this study, Lukas Tanner was enrolled in the Joint MSc programme in Infectious Diseases organised in conjunction with the National University of Singapore, the Novartis Institute of Tropical Diseases, the Swiss Tropical Institute and the University of Basel.
Footnotes
The authors have declared that no competing interests exist.
This study was funded by the Biomedical Research Council (BMRC) of the Agency for Science, Technology and Research, Singapore. The BMRC had no role in study design, data collection and analysis, decision to publish or preparation of manuscript.
References
Differential Diagnosis Website
Abstract
The aim of this study was to develop and explore the diagnostic accuracy of a decision tree derived from a large real-life primary care population.
Data from 9297 primary care patients (45% male, mean age 53±17 years) with suspicion of an obstructive pulmonary disease was derived from an asthma/chronic obstructive pulmonary disease (COPD) service where patients were assessed using spirometry, the Asthma Control Questionnaire, the Clinical COPD Questionnaire, history data and medication use. All patients were diagnosed through the Internet by a pulmonologist. The Chi-squared Automatic Interaction Detection method was used to build the decision tree. The tree was externally validated in another real-life primary care population (n=3215).
Our tree correctly diagnosed 79% of the asthma patients, 85% of the COPD patients and 32% of the asthma–COPD overlap syndrome (ACOS) patients. External validation showed a comparable pattern (correct: asthma 78%, COPD 83%, ACOS 24%).
Our decision tree is considered to be promising because it was based on real-life primary care patients with a specialist's diagnosis. In most patients the diagnosis could be correctly predicted. Predicting ACOS, however, remained a challenge. The total decision tree can be implemented in computer-assisted diagnostic systems for individual patients. A simplified version of this tree can be used in daily clinical practice as a desk tool.
Short abstract
A real-life diagnostic decision tree that can be implemented in digital decision-making programmeshttp://ow.ly/VnHut
Introduction
Diagnostic reasoning and clinical decision making is essential in daily clinical practice and depends on the physician's ability to synthesise and interpret clinical information. Different attempts have been made to support physicians in this process by developing decision support tools. These tools have the potential to improve care and decrease variation in care delivery [], and can provide useful diagnostic suggestions leading to a decrease in diagnostic errors []. Probably the most promising approach to improve diagnostic accuracy is to incorporate decision aids directly into daily clinical practice using computer-assisted diagnostic support systems []. These decision support tools based on expert opinion can provide expert consultation to physicians [].
Many clinicians who have to deal with individual patients have a negative attitude towards these systems, as most are not developed in real-life situations, thus reducing generalisability []. Another shortcoming of currently available tools is that they are mostly based on regression and, hence, are too complex and time-consuming for use in daily clinical practice []. A new way to develop decision support tools is using data from real-life clinical decisions to develop decision trees.
Decision trees based on real-life data are promising because they can detect previously unknown interactions between the various items of clinical information and reveal relationships between assessment outcomes and patient characteristics. Additionally, decision trees are visually easy to interpret and transparent so that clinicians see the thresholds leading to the outcome. Moreover, they can trace back the model [] and they can see what can be expected if the patient's status changes [].
We set out to develop a decision tree to predict asthma, chronic obstructive pulmonary disease (COPD) and asthma–COPD overlap syndrome (ACOS) diagnosis based on careful analysis of 9297 real-life individual patient assessments in a primary care-based diagnostic support system []. All patients were suspected to have an obstructive pulmonary disease (OPD) and were assessed identically according to a structured protocol. Each patient was diagnosed by an experienced pulmonologist (n=10). The aim of this study was to enhance diagnostic accuracy and decrease diagnostic variation. We present a decision tree that should be able to be implemented as a decision aid in computer-assisted diagnostic support systems and a simplified and compact version of the decision tree should be able to be used on paper in daily clinical practice as desk tool.
Method
Study design
We retrospectively analysed data obtained from 2007 until 2012 from the Groningen asthma/COPD service for primary care (the Netherlands) []. The Standards for Reporting Diagnostic Accuracy (STARD) guidelines were used as a basis for this study. According to Dutch regulations, a separate ethical committee approval was not required because data were used anonymously and encrypted.
Patient cohort for dataset derivation
We only included patient data from experienced pulmonologists (n=10), who had each assessed ≥300 patients in the asthma/COPD service, in order to avoid the influence of learning effects in our results. Patients (aged >15 years) referred to the asthma/COPD service by their general practitioner for diagnostic assessment were included in the study (table 1). This was an unselected primary care population of patients with respiratory complaints. The proportion of no-show in the asthma/COPD service is on average 12%. The initial dataset consisted of 10 058 patients. Data from 761 patients were excluded because they could not perform an assessable spirometry (n=626) or had missing data at random (n=135). The analysis was therefore based on the remaining 9297 patients.
TABLE 1
Overview of the patient characteristics from the derivation and validation databases
| Derivation database# | Validation database¶ | |
| Patients | 9297 | 3142+ |
| Diagnosis | ||
| COPD | 1716 (18.5) | 555 (17.7) |
| Asthma | 4125 (44.4) | 685 (21.8) |
| Probable asthma | 836 (26.6) | |
| ACOS | 711 (7.6) | 247 (7.9) |
| Other | 2745 (29.5) | 818 (26.0) |
| Patient characteristics | ||
| Male | 4146 (44.6) | 1347 (42.9) |
| Smoked | ||
| Never smoked | 2833 (30.5) | 1182 (37.7) |
| Ever smoked | 6464 (69.5) | 1895 (62.3) |
| Family history | ||
| No or unknown family history | 4525 (48.7) | 2146 (68.3) |
| Positive family history | 4772 (51.3) | 996 (31.7) |
| Allergy | ||
| No allergy | 5542 (58.9) | 932 (29.7) |
| ≥1 allergy | 3755 (39.9) | 1651 (52.5) |
| Missing data | 105 (1.1) | 559 (17.8) |
| Hyperreactivity | ||
| No hyperreactivity | 3105 (33.4) | 2347 (74.7) |
| Hyperreactivity present | 6192 (66.6) | 795 (25.3) |
| Occupational risk | ||
| No occupational risk | 8742 (94.0) | Unknown |
| Occupational risk present | 555 (6.0) | |
| Age years | 53.3±17.1 | 49.4±16.8 |
| Age of onset years | 35.4±23.3 | 36.1±21.6 |
| Total ACQ score | 1.2±0.9 | 1.3±0.9 |
| Total CCQ score | 1.4±0.9 | 1.5±0.9 |
| Lung function post bronchodilator | ||
| FEV1 L | 2.9±1.0 | 2.9±1.0 |
| FEV1 % predicted | 89.4±19.3 | 92.1±20.0 |
| FVC L | 3.9±1.1 | 4.0±1.1 |
| FVC % predicted | 101.6±16.5 | 106.7±35.9 |
| FEV1/FVC | 73.0±12.9 | 72.1±13.6 |
| Reversibility % | 6.1±7.5 | 6.9±9.0 |
Data are presented as n, n (%) or mean±sd. COPD: chronic obstructive pulmonary disease; ACOS: asthma–COPD overlap syndrome; ACQ: Asthma Control Questionnaire; CCQ: Clinical COPD Questionnaire; FEV1: forced expiratory volume in 1 s; FVC: forced vital capacity. #: database from the asthma/COPD service used for development of the decision tree (Groningen, the Netherlands); ¶: database from the asthma/COPD service used in the external validation (Rotterdam, the Netherlands); +: total diagnosed was 3141 because one patient could not perform a proper lung function test.
Predictors
Predictors could be divided into 1) patient characteristics, 2) patient-reported outcomes (PROs) and 3) spirometry results. All 22 predictors were collected during one regular baseline assessment procedure in the asthma/COPD service. No adverse effects were to be expected from the assessments.
Patient characteristics
A medical history questionnaire with questions about sex, age, age of onset of respiratory symptoms, family history, current and past symptoms, exacerbations, allergy and other stimuli provoking symptoms, current medication, occupation and smoking was collected.
PROs: the Asthma Control Questionnaire and the Clinical COPD Questionnaire
The Asthma Control Questionnaire (ACQ) [] was used to measure asthma control and contains six questions. The Clinical COPD Questionnaire (CCQ) [] was used to measure COPD health status and contains 10 questions. In the decision tree analysis we included all individual questions from the ACQ and CCQ and the total score on each questionnaire, to examine whether disease severity and specific symptoms could be used to distinguish between the different diagnoses.
Spirometry results
Spirometry was performed according to current guidelines [, 11]. We analysed post-bronchodilator (post-BD) forced expiratory volume in 1 s (FEV1), post-BD forced vital capacity (FVC) and post-BD FEV1/FVC ratio. Also, reversibility of FEV1 (in litres) after 400 μg salbutamol was examined.
Statistical analyses
SPSS package 22.0 (IBM Corp., Armonk, NY, USA) was used for the statistical analyses. Initially, continuous variables were divided into categorical counterparts using optimal binning [] to enhance the performance and accuracy of the decision tree. The number of predefined categorical counterparts was two, except for the body mass index and FEV1 post-BD, where we chose to accept a maximum of four counterparts (table 2) [, 13].
TABLE 2
Transformation of continuous predictors to ordinal predictors
| Predictor | Established categories |
| Patient characteristics | |
| Age years | <55 |
| ≥55 | |
| Age of onset years | <38 |
| ≥38 | |
| BMI kg·m−2 | <22 |
| ≥22 and <36 | |
| ≥36 | |
| Allergy total | No allergy |
| ≥1 allergy | |
| Hyperreactivity | No hyperreactivity |
| ≥1 hyperreactivity | |
| ACQ and CCQ | |
| ACQ1 | 0 or 1 |
| ≥2 | |
| ACQ total, ACQ2, ACQ4, ACQ5, ACQ6 | 0 |
| ≥1 | |
| CCQ subscale mental, CCQ1, CCQ2, CCQ4 | 0 |
| ≥1 | |
| CCQ subscale symptoms, CCQ6 | 0 or 1 |
| ≥2 | |
| CCQ7 | <6 |
| ≥6 | |
| Spirometry results | |
| FEV1 % predicted | <78 |
| ≥78 and <92 | |
| ≥92 and <102 | |
| ≥102 | |
| FVC % predicted | <81 |
| ≥81 | |
| Reversibility % | <7 |
| ≥7 |
Continuous predictors were transformed to ordinal predictors using minimum descriptive length discretisation. It was not possible to create bins for Asthma Control Questionnaire (ACQ) question 3, Clinical COPD Questionnaire (CCQ) questions 3, 5, 8, 9 or 10, CCQ total or CCQ subscale functional, because of low association with the dependent variable. BMI: body mass index; FEV1: forced expiratory volume in 1 s; FVC: forced vital capacity.
Development of the decision tree
We used the exhaustive Chi-squared Automatic Interaction Detection (CHAID) method [13] to develop our decision tree. For an overview of relevant decision tree concepts see figure 1. In the decision tree we combined “indication of restriction”, “diagnosis unclear” or “no disease” with “other”. The maximum tree depth was five levels and the significance level for merging nodes was 0.01. Bonferroni correction was applied to correct for overstating of the significance level caused by multiple comparisons. The minimum number of patients in a child leaf was 94 (>1% of the total number of patients).
The most important decision tree concepts. In our analyses we included 9297 patients. The minimum accepted number of patients in an end leaf was set at 94, which is >1% of the patient total.
A simplified compact version of the decision tree [13] was developed by reducing the initial decision tree with a technique called pruning. Branches were pruned if the difference in main category between the parent leaf and the child leaf was <10%. For example, if the proportion of asthmatics in the parent leaf is 43% and the proportion of asthmatics in the child leaf is 40%, the branch will be pruned because the difference is <10%. To enhance usability we determined the maximum tree depth to be four levels and discussed this tool with experienced clinicians.
Internal validation
We validated our decision tree with the “10-fold cross validation” method (figure 2). The dataset was randomly divided into ten mutually exclusive subsets and each subset was held out in turn to function as validation sample. The decision tree was then developed on the combined nine remaining subsets. This procedure was repeated 10 times so that each subset was used once as validation set, according to Wittenet al. [14], so that the final decision tree was based on 100 tree analyses.
Overview of a single “10-fold cross validation”. This was repeated 10 times and each time an error rate was produced. In this study we used the results of the decision tree with the lowest error rate, which was 0.314.
External validation
We validated our decision tree in an external database of another Dutch asthma/COPD service for primary care that operates in Rotterdam and has a similar structure to the service in Groningen. Patients were assessed by two pulmonologists and two specialised general practitioners. This database is called the validation database.
Results
Patient characteristics
We included 9297 patients (mean age 53±17 years, 44.6% male, diagnosis by pulmonologist: 44.4% asthma, 18.5% COPD, 7.6% ACOS and 29.5% other). Patients from the validation dataset (n=3142) were comparable with patients from the derivation dataset (mean age 49±17 years, 42.9% male, 21.8% asthma, 26.6% “probable asthma”, 17.7% COPD, 7.9% ACOS and 26.0% other). However, the proportion of asthma diagnoses given by the pulmonologists differed (derivation: 44.4% asthma; validation: 21.8% asthma) (table 1).
Exhaustive CHAID analysis
The final decision tree consisted of the following predictors (in order of importance): FEV1/FVC, age of onset, smoking, allergy, reversibility, ACQ question 5 (“In general, during the past week, how much of the time did you wheeze?”), age, FEV1 and bronchial hyperreactivity. Comparisons between the predicted diagnoses and actual pulmonologists’ diagnoses are given in tables 3–5. The average predictive value of the decision tree before pruning was 69.0% (proportion correct: asthma 78.9%, COPD 84.7%, ACOS 31.6% and other 53.9%) (table 3). The most important pathways leading to diagnoses were: 1) no obstruction, onset age <38 years, allergy and reversibility ≥7%, leading to asthma (89% correct); and 2) obstruction, smoked, onset age ≥38 years and FEV1 <78% predicted, leading to COPD (81% correct). ACOS was only predicted by one pathway (obstruction, smoked, onset age <38 years). The pathway “no obstruction, no allergy, reversibility <7% and onset age ≥38 years” did not predict diagnosis and led to the category “other” in 1961 patients, which is 21.2% of the total patient population and is the largest branch. For an overview of all pathways, see table 6.
TABLE 3
Comparison of individual patient diagnoses given by the pulmonologists and diagnoses predicted with the decision tree
| Diagnosis by pulmonologist | Diagnosis predicted by decision tree | Total | Correct | |||
| ACOS | COPD | Asthma | Other# | |||
| ACOS | 225 | 355 | 98 | 33 | 711 (7.6) | 225 (31.6) |
| COPD | 135 | 1454 | 68 | 59 | 1716 (18.5) | 1454 (84.7) |
| Asthma | 162 | 101 | 3253 | 609 | 4125 (44.4) | 3253 (78.9) |
| Other# | 28 | 128 | 1109 | 1480 | 2745 (29.5) | 1480 (53.9) |
| Total | 550 (5.9) | 2038 (21.9) | 4528 (48.7) | 2181 (23.5) | 9297 (100) | 6412 (69.0) |
Data are presented as n or n (%). ACOS: asthma–COPD overlap syndrome; COPD: chronic obstructive pulmonary disease. #: “diagnosis unclear”, “indication of restriction” or “no disease”. Bold indicates diagnoses that were correctly predicted.
TABLE 5
Comparison of individual patient diagnoses given by the pulmonologists from the validation asthma/COPD service and diagnoses predicted with the decision tree
| Diagnosis by validation pulmonologist | Diagnosis predicted by decision tree | Total | Correct | |||
| ACOS | COPD | Asthma | Other# | |||
| ACOS | 59 | 151 | 35 | 2 | 247 (7.9) | 59 (23.9) |
| COPD | 53 | 459 | 37 | 6 | 555 (17.7) | 459 (82.7) |
| Asthma | 42 | 32 | 533 | 78 | 685 (21.8) | 533 (77.8) |
| Probable asthma | 11 | 7 | 580 | 238 | 836 (26.6) | 580 (69.4) |
| Other# | 10 | 59 | 336 | 413 | 818 (26.0) | 413 (50.5) |
| Total | 175 | 708 | 1521 | 737 | 3141 (100.0) | 2044(65.1) |
Data are presented as n or n (%). ACOS: asthma–COPD overlap syndrome; COPD: chronic obstructive pulmonary disease. #: “diagnosis unclear”, “indication of restriction” or “no disease”. Bold indicates diagnoses that were correctly predicted.
TABLE 6
Branches in the decision tree and an overview of the predicted diagnoses
| Rule branch | Main outcome | Total leaf n (% total leaf) | ACOS n (% total) | COPD n (% total) | Asthma n (% total) | Other# n (% total) |
| FEV1/FVC ≥70% predicted | Asthma | 1415 (15.2) | 11 (0.8) | 3 (0.2) | 1108 (78.3) | 293 (20.7) |
| Onset age <38 years | ||||||
| ≥1 allergy | ||||||
| Reversibility <7% | ||||||
| FEV1/FVC ≥70% predicted | Asthma | 724 (7.8) | 11 (1.5) | 1 (0.1) | 647 (89.4) | 65 (9.0) |
| Onset age <38 years | ||||||
| ≥1 allergy | ||||||
| Reversibility ≥7% | ||||||
| FEV1/FVC ≥70% predicted | Asthma | 829 (8.9) | 16 (1.9) | 5 (0.6) | 593 (71.5) | 215 (25.9) |
| Onset age <38 years | ||||||
| No allergy | ||||||
| Wheezing | ||||||
| FEV1/FVC ≥70% predicted | Asthma | 548 (5.9) | 11 (2.0) | 7 (1.3) | 276 (50.4) | 254 (46.4) |
| Onset age ≥38 years | ||||||
| ≥1 allergy | ||||||
| Reversibility <7% | ||||||
| FEV1/FVC ≥70% predicted | Asthma | 181 (1.9) | 8 (4.4) | 2 (1.1) | 133 (73.5) | 38 (21.0) |
| Onset age ≥38 years | ||||||
| ≥1 allergy | ||||||
| Reversibility ≥7% | ||||||
| FEV1/FVC <70% predicted | Asthma | 356 (3.8) | 35 (9.8) | 43 (12.2) | 219 (61.5) | 59 (16.6) |
| Never smoked | ||||||
| FEV1/FVC <70% predicted | ACOS | 783 (8.4) | 302 (38.6) | 252 (32.2) | 183 (23.4) | 46 (5.9) |
| Onset age <38 years | ||||||
| Smoked | ||||||
| FEV1/FVC <70% predicted | COPD | 1142 (12.3) | 164 (14.4) | 928 (81.3) | 19 (1.7) | 31 (2.7) |
| Onset age ≥38 years | ||||||
| Smoked | ||||||
| FEV1<78% predicted | ||||||
| FEV1/FVC <70% predicted | COPD | 257 (2.8) | 26 (10.1) | 203 (79.0) | 11 (4.3) | 17 (6.6) |
| Onset age ≥38 years | ||||||
| Smoked | ||||||
| FEV1≥78% and <92% predicted | ||||||
| Reversibility <7% | ||||||
| FEV1/FVC <70% predicted | COPD | 168 (1.8) | 56 (33.3) | 79 (47.0) | 18 (10.7) | 15 (8.9) |
| Onset age ≥38 years | ||||||
| Smoked | ||||||
| FEV1≥78% and <92% predicted | ||||||
| Reversibility ≥7% | ||||||
| FEV1/FVC <70% predicted | COPD | 238 (2.6) | 32 (13.4) | 127 (53.4) | 32 (13.4) | 47 (19.7) |
| Onset age ≥38 years | ||||||
| Smoked | ||||||
| FEV1≥92% predicted | ||||||
| FEV1/FVC ≥70% predicted | Other# | 1961 (21.2) | 33 (1.7) | 61 (3.1) | 561 (28.6) | 1306 (66.6) |
| Onset age ≥38 years | ||||||
| No allergy | ||||||
| FEV1/FVC ≥70% predicted | Other# | 695 (7.5) | 6 (0.9) | 5 (0.7) | 323 (46.8) | 359 (51.7) |
| Onset age <38 years | ||||||
| No allergy | ||||||
| No wheezing |
ACOS: asthma–COPD overlap syndrome; COPD: chronic obstructive pulmonary disease; FEV1: forced expiratory volume in 1 s; FVC: forced vital capacity. #: “diagnosis unclear”, “indication of restriction” or “no disease”. Bold indicates diagnoses that were correctly predicted. Spirometry results were taken after admission of bronchodilation.
The simplified compact version of the decision tree (figure 3) was slightly more efficient, with 11 termination leaves. The simplified tree is practical in clinical practice. However, the overall precision of this tree was slightly lower than the complete decision tree: overall 67.5% were correctly predicted (proportion correct: asthma 72.1%, COPD 77.9%, ACOS 42.5% and other 60.7%). After discussion with experienced clinicians (n=3), we decided to exclude FEV1 post-BD, to enhance applicability. For a comparison between the predicted diagnoses from this simplified decision tree and the actual pulmonologists' diagnoses, see table 4.
The simplified decision tree derived from the total decision tree gives an overview of the important pathways. COPD: chronic obstructive pulmonary disease; ACOS: asthma–COPD overlap syndrome.
TABLE 4
Comparison of individual patient diagnoses given by the pulmonologists and diagnoses predicted with the simplified decision tree
| Diagnosis by pulmonologist | Diagnosis predicted by simplified tree | Total | Correct | |||
| ACOS | COPD | Asthma | Other# | |||
| ACOS | 302 | 278 | 92 | 39 | 711 (7.6) | 302 (42.5) |
| COPD | 252 | 1337 | 61 | 66 | 1716 (18.5) | 1337 (77.9) |
| Asthma | 183 | 80 | 2976 | 886 | 4125 (44.4) | 2976 (72.1) |
| Other# | 46 | 110 | 924 | 1665 | 2745 (29.5) | 1665 (60.7) |
| Total | 783 (8.4) | 1805 (19.4) | 4053 (43.6) | 2656 (28.6) | 9297 (100) | 6280 (67.5) |
Data are presented as n or n (%). ACOS: asthma–COPD overlap syndrome; COPD: chronic obstructive pulmonary disease. #: “diagnosis unclear”, “indication of restriction” or “no disease”. Bold indicates diagnoses that were correctly predicted.
Internal validation
The error rates of the 10 repeated decision tree analyses ranged from 0.314 to 0.318, with an average error of 0.316. Variation in error rates exist because small differences in random splits used for the “10-fold cross validation” occur. We have selected the decision tree with the lowest error rate (0.314).
External validation
Our decision tree could correctly predict diagnosis in 54.2% of the patients in the validation dataset (proportion correct: asthma 77.8%, COPD 82.7%, ACOS 23.9% and other 39.4%). In 836 (26.6%) patients from the validation database with unclear diagnosis, the assessing pulmonologists added a remark in the database with the notion “probable asthma”. We repeated the validation procedure and included “probable asthma” patients in the asthma group. The accuracy of our decision tree improved substantially: the overall proportion correct became 65.1% (ACOS 23.9%, COPD 82.7%, asthma 77.8% and other 50.5%), which is comparable with the accuracy of the decision tree in the derivation dataset (table 5).
Discussion
Main results
In this study, we have presented a thoroughly developed diagnostic support tool, based on a large database with real-life primary care patients suspected to have OPD who have received a structured assessment and an expert diagnosis. We chose this patient population because OPDs like asthma and COPD are common in primary care, and underdiagnosis of COPD and misdiagnosis between COPD and asthma are an important clinical problem []. Our tool was able to correctly predict diagnosis in 69% of the patients (proportion correct: asthma 79%, COPD 85% and ACOS 32%) and was based on a combination of patient characteristics, symptoms and spirometry results, which are part of guideline recommended assessments. Our decision tree provides a simple, well interpretable and practical overview that generates a diagnostic suggestion for primary care patients suspected to have an OPD. Additionally, we have developed a simplified version of the decision tree to be used as a desk tool in clinical practice. This slightly decreased the accuracy of the original decision tree (proportion correct: overall 68%, asthma 72% and COPD 78%) but increased the proportion of correctly predicted ACOS patients (43%).
Limitations
Although most patients could be correctly diagnosed with our decision tree, still 31% of the patients could not be diagnosed correctly using the diagnosis originally made by the pulmonologist as gold standard. This might have been caused by the diagnostic variation among pulmonologists, which was previously described by Mettinget al. []. Despite this diagnostic variation between the pulmonologists, additional data from 1856 patients showed that most diagnoses were confirmed at follow-up (confirmed in 92% of the asthma patients, in 86% of the COPD patients and in 73% of the ACOS patients). According to Buffelset al. [], in the absence of a gold standard, a pulmonologist's diagnosis is most accurate. Of course, elimination of all uncertainty in a diagnostic support tool is not feasible; this would cost too much in terms of resources []. Response to treatment might determine whether the predicted diagnosis was satisfactory [] and the predicted diagnosis can be considered as a working diagnosis.
Another limitation is that the decision tree does not differentiate between patients with or without disease. The diagnosis “no disease” is combined with “indication of restriction” and “diagnosis unclear” in the umbrella term “other”. However, the proportion of patients without disease was very small (n=709, 7.6%) and would therefore be difficult to predict with a decision tree.
Finally, the decision tree has a low accuracy in diagnosing ACOS. Again, using the diagnosis originally made by the pulmonologist as gold standard, it means that the pulmonologists had little agreement about this diagnosis at the time the data were collected. It is known that ACOS is difficult to diagnose from both asthma and COPD, which was reflected in our decision tree. Differentiating between asthma, COPD and ACOS is important because the treatment and prognosis are different []. ACOS patients have more respiratory symptoms, more functional limitations, and are more frequently hospitalised []. In 2014, the Global Initiative for Chronic Obstructive Lung Disease (GOLD) and the Global Initiative for Asthma (GINA) presented new guidelines for ACOS that might enhance future diagnostic accuracy [19] and will probably lead to more consensus among physicians.
Strengths and weaknesses
Internal validity
CHAID is based on the maximum likelihood ratio and is considered to be at least as good as log regression techniques; however, it is easier to interpret and no calculation of risk scores is needed because the user can simply follow the tree []. The exhaustive CHAID method provides an even more thorough heuristic for finding the optimal way of grouping the categories of each predictor, and provides a better suited approximation for the Bonferroni correction [13]. We performed the “10-fold cross validation” method because this method is considered to be the best validation method [14].
We used specialists' diagnoses, which we considered to be the gold standard. Patients in the asthma/COPD service were diagnosed from spirometry and history data through the Internet. Previously, Lucaset al. [] showed that pulmonologists can reliably diagnose patients from written spirometry and history data. However, all diagnoses in this system were based on the available variables and were not confirmed by, for example, bronchial hyperresponsiveness testing, exhaled nitric oxide fraction or extended radiology, because these are not used in primary care practice. One can therefore argue that these diagnoses are not fully confirmed and are just a step in the diagnostic process.
External validation
The decision tree could correctly predict 54% of the patients in the validation dataset. However, adding “probable asthma” to the asthma group improved the accuracy substantially (from 54% to 65%). The lower overall prediction performance in the validation dataset might be caused by the difference in opinion from the pulmonologists who assessed the patients in the validation dataset to the pulmonologists in the original dataset. We make this assumption because the proportion of patients diagnosed with asthma by the pulmonologists was lower (22% in the validation dataset, compared with 44% in the original dataset). Most patients with “probable asthma” in Rotterdam were referred for a histamine provocation test (n=628, 75%). Apparently, pulmonologists from the derivation asthma/COPD service in Groningen establish the diagnosis of asthma more quickly than the pulmonologists in the validation asthma/COPD service. Additional analyses showed that probable asthma patients had on average lower reversibility compared with asthma patients (mean±sd reversibility: probable asthma patients 3.6±4.9%, asthma patients 12.5±12.1%; p<0.001).
An effectiveness study has shown that patients who were diagnosed and followed-up by the asthma/COPD service in Groningen improved in health status, asthma control and exacerbation rate []. We therefore assume that our decision tree is of added value for primary care respiratory patients and that the external validity of our decision tree is high because we have included a large sample of real-life primary care patients, while our decision tree is developed with common predictors that are part of guideline recommended assessments in patients suspected to have an OPD [, 11]. Therefore, the generalisability of our decision tree is expected to be high.
Comparison with existing literature
In the field of respiratory medicine, several decision trees have been developed to predict severity [], mortality [], hospitalisation [] and clinical outcomes []. In this article, we have presented the first real-life decision tree to predict diagnosis in patients suspected to have an OPD in primary care daily clinical practice. This is important because diagnostic errors are common [, , ] in general practice []. 10–15% of all diagnoses are estimated to be incorrect []. These errors affect patients outcomes [, ], and can lead to inappropriate patient care and increased healthcare costs [, ]. Being a physician can be demanding [] and making decisions under time pressure can negatively influence diagnostic performance [].
In the past 20 years, a consensus has been reached about a dual-system theory that proposes two modes of clinical decision making. The first system consists of one nonverbal intuitive cognition system, which is fast but error prone [] and is based on intuitive reasoning, while the second system is based on the classical analytical reasoning approach []. Experienced physicians use both systems while novices mostly rely on the second hypothesis-testing system []. The decision support tool presented here matches both pathways by providing diagnostic suggestions. It points out possible diagnoses along with an estimation of probability, which can support the nonverbal intuitive cognition system. It also supports the analytic reasoning approach by giving feedback so that the initial diagnosis can be confirmed or dismissed. Our decision tree can be used by novices and experienced physicians, so that novices can function like a more experienced physician [, ] and experts can use the tree as a feedback tool to confirm their initial diagnosis or suggest another.
Spirometry is considered to be essential for proper diagnosis, according to the GOLD and GINA guidelines []. Symptom-based questionnaires in combination with spirometry enhance diagnostic accuracy of OPD even more []. Our decision tree combined both and produced transparent thresholds for continuous variables like age or reversibility that can be used in clinical practice.
In the past years, more emphasis has been given to personalised medicine instead of the “one size fits all” approach. We found that there are different pathways leading to the same diagnosis. We found six pathways leading to asthma and four leading to COPD (table 6). This is consistent with the new insights that asthma and COPD are heterogeneous diseases.
Implementation
We have presented a computer-assisted diagnostic support system for OPDs based on real-life primary care data that can be implemented in digital automated decision-making programmes. The transparency of our decision tree is valuable because the proposed diagnosis is accompanied by a probability that can support the physicians in diagnosing and treating their individual patients. This might enhance diagnostic accuracy. The simplified and compact paper version of the decision tree could be helpful in clinical practice as a desk tool.
Recommendation for future research
The next step is to validate our decision tree in other primary care populations and in clinical practice, to optimise the predictive value and the applicability in individual patients with suspicion of OPD.
Footnotes
Support statement: Funding was received from the Universitair Medisch Centrum Groningen (regular PhD salary) and cofunding was received from Novartis (grant for department). Funding information for this article has been deposited with FundRef.
Conflict of interest: Disclosures can be found alongside this article at openres.ersjournals.com