Application of Statistics in Physics
Claimed by Edwin Solis (April 16th, Spring 2022)
With the development of Quantum Mechanics and Statistical Mechanics, the subject of Statistics has become quintessential for understanding the foundation of these physical theories.
Basics
Probability
Probability is the numerical description of the likelihood of an event occurring from a sample space written as a value between 0 and 1. This event is just the outcome of executing an experiment, and the sample space is just the whole set of outcomes possible from this experiment. Usually, the probability for equally likely outcomes is written mathematically as
[math]\displaystyle{ P(A)=\frac{\text{# Number of times A occurs}}{\text{# Total number of outcomes}} }[/math]
For example, for a six face fair dice, we have a sample space [math]\displaystyle{ S=\{1, 2, 3, 4, 5, 6\} }[/math], where the probability of obtaining a 3 from a dice throw is given by [math]\displaystyle{ P(X=3)=\frac{1}{6} }[/math]. Here the dice throw is the experiment while the event is the outcome of the number obtained from the dice.
Note that we then define the probability of being in the sample space [math]\displaystyle{ P(S) = 1 }[/math] as an axiom which is congruent with the definition of probability.
More generally, however, we can define a probability space for an experiment with discrete outcomes as a mathematical function called probability mass function (pmf) [math]\displaystyle{ p(x) }[/math]. In the dice example, the pmf would be written as
[math]\displaystyle{ \begin{align} P(X=x) = p(x)=\begin{cases} \frac{1}{6} &\;|\;x=1,2,3,4,5,6\\ 0 &\;|\;otherwise \end{cases} \end{align} }[/math]
Now, consider an experiment as throwing a dart into a dartboard or measuring the length of a piece of rope. In both cases, we expect that there be infinitely points that are infinitely close to each other where the dart can land and values of length that the rope can have. Therefore, in this case, we find ourselves with a set of continuous values inside the sample space. As there are uncountable possible non-zero probabilities for these points, we would end up with a diverging value for [math]\displaystyle{ P(S) }[/math] which goes against the definition.
Therefore, for these continuous cases, we define the mathematical function, the probability density function (pdf) [math]\displaystyle{ f(x) }[/math]. Instead of assigning a probability for an outcome to take a specific value, the pdf assigns a probability to an interval of values that the outcome could come to have.
For example, if a there is a uniform probability of a bus arriving between [math]\displaystyle{ 10:00\text{ am} \text{ and } 11:00 \text{ am} }[/math], then the pdf of arriving [math]\displaystyle{ t }[/math] minutes after [math]\displaystyle{ 10:00\text{ am} }[/math] would be written as: [math]\displaystyle{ f(x) = \begin{cases} \frac{1}{60} &\;|\;0\leq x \leq 60\\ 0 &\;|\; otherwise \end{cases} }[/math] To obtain the actual probability we integrate over the interval we desire to evaluate. So, for the probability of the bus arriving during the first 10 minutes we would have:
[math]\displaystyle{ P(0\leq X \leq 10) = \int^{10}_0{f(x)\text{d}x} = \frac{1}{6} }[/math]
Therefore, we have the following properties:
Discrete Sample Space
For the pmf [math]\displaystyle{ p(x) }[/math] with [math]\displaystyle{ n }[/math] possible events,
- [math]\displaystyle{ p(x)\geq 0 }[/math] if [math]\displaystyle{ x\in S }[/math], else [math]\displaystyle{ p(x) = 0 \text{ for } x\notin S }[/math]
- [math]\displaystyle{ P(X=x) = p(x) }[/math]
- [math]\displaystyle{ \sum_{x_i\in S}P(X=x_i) = 1 }[/math] with [math]\displaystyle{ S=\{x_1, x_2,...\} }[/math]
Continuous Sample Space
For the pdf [math]\displaystyle{ f(x) }[/math],
- [math]\displaystyle{ f(x)\geq0 }[/math]
- [math]\displaystyle{ P(a\lt X\lt b)=\int^b_a{f(x)\,\text{d}x} }[/math] from which follows that the probability of a specific value is [math]\displaystyle{ P(X=c)=\int^c_c{f(x)\,\text{d}x}=0 }[/math] if [math]\displaystyle{ f(c) }[/math] is finite.
- [math]\displaystyle{ P(-\infty\lt X\lt \infty)=\int^{\infty}_{-\infty}{f(x)\,\text{d}x}=1 }[/math]
Random Variables and Distributions
From the base definition of probability, we can go a step further and deal with outcomes of experiments as their own variable. The outcome of a random experiment is called a Random Variable (r.v.). This variable does not have a definite value per se, rather, it possesses certain properties linked to the underlying sample space of the experiment. This means that all the possible events in the sample space are specific values a random variable can attain.
Using the dice example, the experiment of throwing the dice results in the outcome [math]\displaystyle{ X }[/math] which is the random variable of the result of the dice. [math]\displaystyle{ X }[/math] can take the values [math]\displaystyle{ 1,2,3,4,5,6 }[/math]. We can also define a random variable [math]\displaystyle{ Y }[/math] to be the sum of two consecutive dice throws, so [math]\displaystyle{ Y }[/math] can take the values [math]\displaystyle{ 1 \text{ to } 12 }[/math]. The properties of the random variable depend on the probability function we use for the sample space. This means there are two types of a random variable: discrete which is described by the probability mass function, and continuous which is described by the probability density function (sometimes, also described by the cumulative distribution function cdf).
The set of mathematical descriptions for the sample space and probability space of a random variable is called a distribution. From the respective type of random variables, we have discrete and continuous distributions.
Discrete
Using the probability mass function we have direct probabilities for specific values that the discrete random variable can take. This is useful for countable sets of events or measurements that can occur. Discrete distributions can be represented using a line graph by mapping every value [math]\displaystyle{ x\in S }[/math] to its corresponding probability [math]\displaystyle{ P(X=x)=p(x) }[/math].
Common discrete distributions are the Discrete Uniform Distribution, Bernoulli Distribution, Binomial Distribution, Poisson Distribution, and Hypergeometric Distribution
Continuous
For the continuous case, we have to evaluate the integral of the interval we require to find the probability using the probability density function. These distributions are mainly used for quantities that are known to have an infinite number of values; however, they also find utility in approximating discrete distributions composed of a large population of values. Continous distribution can be represented by graphing the pdf where the area an interval represents the probability, as well as they can be graphed using the cdf to better see the change in probability from two extreme points of an interval.
The most common used continuous distribution is the Normal Distribution; other common ones are the cousins of the normal distribution: [math]\displaystyle{ \chi^2 }[/math] Distribution, t-Distribution, as well as Continuous Uniform Distribution, Exponential Distribution, and Boltzmann Distribution.
Multivariable
In addition to one given an occurrence a probability, it is possible to give a probability for two joint events. This is called a joint pmf.
Expectation
The expected value of a random variable (also known as mean, average, or expectation) is the weighted average of the possible values the random variable can take according to its probability. The common knowledge is that the average of is defined as [math]\displaystyle{ \bar{x}=\frac{\sum^n_{i=1}x_i}{n} }[/math], yet this is only the special case that all the possible values are equiprobable. In general, we define the expectation of a random variable [math]\displaystyle{ X, E[X], }[/math] as
[math]\displaystyle{ E[X] = \sum_{x\in S}xp(x) }[/math] for the discrete case, and
[math]\displaystyle{ E[X] = \int^\infty_{-\infty}{xf(x)\;\text{d}x} }[/math] for the continuous case.
In physics, specially in Quantum Mechanics, it is more common to see the expectation of a quantity [math]\displaystyle{ X }[/math] written as [math]\displaystyle{ \langle X \rangle }[/math].
Note that we can generalize, this concept of expectation to the expectation of a function [math]\displaystyle{ g(X) }[/math] depending on [math]\displaystyle{ X }[/math] as [math]\displaystyle{ E[g(X)] = \sum_{x\in S}g(x)p(x) }[/math], and [math]\displaystyle{ E[g(X)] = \int^\infty_{-\infty}{g(x)f(x)\;\text{d}x} }[/math]
Similarly for the multivariate case, the dependence on the the many variables leads to the expressions: [math]\displaystyle{ E[g(X_1, ..., X_n)] = \sum_{x_1 \in S_1}...\sum_{x_n\in S_n}g(x_1,...,x_n)p(x_1,...x_n) }[/math], and [math]\displaystyle{ E[g(X_1, X_2,...)] = \int...\int^\infty_{-\infty}{g(x_1,x_2,...)f(x_1,x_2,...)\;\text{d}x_1\text{d}x_2...} }[/math]