Karteikarten
Lecture 1 - Information theory & Statistical Mechanics
Karten in diesem Stapel (9)
- 1Boltzmann entropy corresponds to the statistical weight \(\Sigma\) of a macrostate by: \[S=k_Bln\Sigma\] WhereTo derive distributions (Boltzmann, FD, BE): place system in a reservoir (energy or particles) and \emph{demand that entropy is maximised}.
- 2What is the a priori definition of probability?Known as the degree of belief:
\[p_i\] is a quantitative measurement of the degree of rational belief that a procedure will yield outcome i.
e.g. If there are \(q\) possible outcomes of a trial, and no weighting on each outcome, then each outcome ha a probability of \[\frac{1}{q}\] assigned to it. - 3Conditions to obtain probability as degree of beliefe.g. r mutually exclusive and exhaustive outcomes
\[(i)\;0\le p_i \le 1\]
\[(ii)\;p_{i_1} \text{ or } i_2 = p_{i_1}+p_{i_2}\]
\[(iii)\;\sum_{i=1}^rp_i=1\]
\[(iv)\;y-bar = \sum_{i=1}^rp_iy_i\] - 4Clearly the probability distribution quantifies the uncertainty of the trial's outcome.e.g. If \[p_j=1\] there is 100% certainty that the outcome of the trial will be j, so there is no point carrying it out. In this case there is no missing information. On the other hand if \[p_j=\frac{1}{r}\] then all outcomes are equally likely and there is a maximum uncertainty or max. missing information in the outcome.
- 5Deduce a function \(S({p}_r\) measuring the missing information, where \({p}_r\) is the set of probabilities for the r possible outcomes.(i)For \(p_1=p_2=...=\frac{1}{r}\), S should be an increasing function of r. i.e. for equally likely outcomes, the more outcomes, the more uncertainty.
\[\](ii)S should be a continuous function of its arguments so that changing the probabilities a little only changes S a little.
\[\](iii) S should be symmetric of it's arguments as relabelling the arguments should not change the information content of the probabilities. - 6And...(iv) If we divide each outcome into n groups labeled \(j=1,...,n\) each group contains \(r_j\) outcomes and the probability that the outcomes is in one of those in group j is \(\omega_j\).
\[S({p}_r)-S({\omega}_n)\]\[=\sum_{j=1}^n \omega_j S \left(\frac{p_{i1}}{\omega_j},\frac{p_{i2}}{\omega_j},...,\frac{p_{irj}}{\omega_j}\right) \]
The LHS is the missing information about the outcome minus the missing info on the group i.e. the missing info about the outcome given that we know the group it is in. \[\]
The RHS is the missing information about which outcome given that we know that it is in group j averaged to the probability of each group j. - 7For example, consider three probabilities \(p_1,p_2,p_3\), then \(S(p_1,p_2,p_3)\) be the missing information. Suppose someone measures the system and finds no difference between alternatives 2 and 3, so they ascribe it a single probability \(p_1+p_2\).Then comparing point (iV) \[\omega_1=p_1,\omega_2=p_2+P_3\].
The missing information in this case would then be:
\[S(p_1,p_2,p_3)=S(\omega_1,\omega_2)+\omega_2S\left(\frac{p_2}{\omega_2},\frac{p_3}{\omega_2}\right)\] As, recall, that if we get result 1, we must have p1, so there is 0 missing information so \(S(1)=0\). - 8Through some nice manipulation (see equations): \[S({p}_r)=-k\sum_{i=1}^rp_{i}ln\: p_i\]For Shannon (mis-)information, \(k=\frac{1}{ln(2)}\). However if \(k=k_B\) is Boltzmann's constant then we have defined the Gibbs entropy: missing information associated with Gibbs free energy.
Here we make the CRITICAL link that missing information corresponds to disorder. - 9Advantages of Gibbs entropy over Boltzmann entropy?- It defines entropy from the distribution of the microstates thus avoiding misidentification of a macrostate.
\[\]
-It defines entropy for smaller systems o we can divide an conquer problems.
\[\]
-It defines entropy for systems which are not in thermal equilibrium but are perturbed.
\[\]
We can RECOVER Boltzmann's entropy by \(p=\frac{1}{\Sigma}\).
Informationen
- Beschreibung
- Degree Statistical Physics (Assemblies (1-4)) Karteikarten am Lecture 1 - Information theory & Statistical Mechanics, erstellt von rmdc am 17/04/2014.
- Erstellt von
- rmdc
- Aufrufe
- 26
- Bewertung