Decision Tree

Decision Tree

1. Information Entropy

   • Uncertainty
   • more uncertain, more information, more bits
   • 0.940, around 1, most information, most uncertainty

2. Information Gain

   • Information entropy - conditional entropy
   • the relevance between variable and decision
   • the bigger, the more relevant, the most important to consider

3. Conditional entropy

   • the uncertainty of the random variables based on some condition
   • 0.693, the smallest uncertainty
   • on the condition of "outlook", the uncertainty of "play or not" is the smallest
   • the relevance between" outlook" and "play" is the strongest

4. 信息熵

   • 不确定性
   • 不确定性大 --》 信息量大 --》 比特数多

5. 条件熵

   • 一定条件下，随机变量的不确定性
   • 数值越小，一定条件下不确定性越小，相关性越大

6. 信息增益

   • 变量和结果的相关性
   • 信息增益越大，相关性越大，越适合做根节点进行决策