Information Theoretic Measures

Communication system is vital for message transmission, affected by noise. Researchers contribute to improving communication. Information theory development is significant, enhanced by C. E. Shannon’s work. Shannon model encodes and decodes messages. Researchers generalize information measures for various applications. This book focuses on information theoretic measures, their analysis and applications. Information measures are applied in data mining, including weighted residual information measure. Classification methods and results are compared for improvement. Information theoretic measures in ID3 algorithm find information gain as splitting criteria for data. ID3 algorithm applied to census 2011 data of India to classify attributes into States and Union Territories. Decision tree made using Renyi entropy, rules developed to decide project implementation region. Varma entropy based ID3 algorithm used to analyze health services in India up to 2015. C4.5 algorithm applied on same data with discrete attribute values to develop decision tree. Weighted generalized residual information measure of order α and type β developed as extension of information theoretic measures.