Let us build the decision tree using the above dataset. We assume ‘Detected’ as positive and ‘Not Detected’ as negative. All logarithms are calculated with the base as 2.
Here, n = 14, p = 8, q = 6
Therefore, Entropy(target) = – (8*log(8/14)/14) – (6*log(6/14)/14) = 0.99
Considering the column “Fever”:
Total items with label ‘Yes’ = 8
Total items with label ‘No’ = 6
Positive correspondences for label ‘Yes’ = 6
Positive correspondences for label ‘No’ = 2
Negative correspondences of label ‘Yes’ = 2
Negative correspondences of label ‘No’ = 4
Entropy(Yes) = – (6*log(6/8)/8) – (2*log(2/8)/8) = 0.81
Entropy(No) = -(2*log(2/6)/6) – (4*log(4/6)/6) = 0.91
Information Gain(Fever) = 0.99 – ((8*0.81)/14 + (6*0.91)/14) = 0.13