Wavelet and Linear Algebra

Abstract Shannon's entropy plays an important role in information theory, dynamical systems and thermodynamics. In this paper we applying Jensen's inequality in information theory and we obtain some results for the Shannon's entropy of random variables and Shannon's entropy of stochastic process. Also we obtain upper bound and lower bound for Shannon's entropy of information sources.