AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |
Back to Blog
Information entropy8/31/2023 ![]() ![]() On average, how many questions do you expect to ask, to determine a symbol from Machine Two? This can be explained Otherwise, we have to ask a third question to identify which of the Otherwise, we are left with two equal outcomes, D or, B and C We could ask, "is it D?". We could start by asking "is it A?", if it is A we are done, only one question in this case. Here A has a 50% chance of occurring, and all other letters add to 50%. Probability of each symbol is different, so we can ask What about Machine Two? As with Machine One, we could ask two questions to determine the next symbol. We can say the uncertainty of Machine One is two questions per symbol. So we simply pick one, such as "is it A?", and after this second question, we will have correctly Of the possibilities, and we will be left with two After getting the answer, we can eliminate half For example, our first question, we could ask if it is any two symbols, such as "is it A or B?", since there is a 50% chance of A or B and a 50% chance of C or D. Is to pose a question which divides the possibilities in half. ![]() You would expect to ask? Let's look at Machine One. Is the minimum number of yes or no questions If you had to predict the next symbol from each machine, what More information? Claude Shannon cleverly Machine One generatesĮach symbol randomly, they all occur 25% of the time, while Machine Two generates symbols according to the following probabilities. They both output messages from an alphabet of A, B, C, or D. Why should I know What is entropy? Where it will help me? or where it helps the world to know about entropy Is this entropy being used in some real world application, if yes then if you could have quoted a real world example in the video it would have been more clear. Is the second sentence produced by machine 2 is more uncertain than machine 1. In the above i would say machine 2 is more informative as it gives more detailed information about the subject I wanted to know. Machine 2: computer is an electronic device which can perform huge arithmetic operations in fraction of seconds. Machine 1: computer is an electronic device. For example I choose that I need information about "Computers" and then two machines gave me information as below: for that first thing I have to choose is for what I need information. To me information is "what is conveyed or represented by a particular arrangement or sequence of things" are we talking about this information here.Īlso I would like to tell what more information to me means. What exactly do we mean by information here. How uncertainty is related to information. Can't get how machine 1 gives more information than machine 2 just because data coming out of it is more uncertain. ![]()
0 Comments
Read More
Leave a Reply. |