Least Squares Data Fitting with Applications

Appendix D: Statistics

Appendix D Statistics D.1 Definitions We list some basic definitions and techniques taken from statistics that are needed to understand some sections of this book. • A sample space is the set of possible outcomes of an experiment or a random trial. For some kind of experiments (trials), there may be several plausible sample spaces available. The complete set of outcomes can be constructed as a Cartesian product of the individual sample spaces. An event is a subset of a sample space. • We denote by Pr{event} the probability of an event. We often consider finite sample spaces, such that each outcome has the same probability (equiprobable). • X is a random variable defined on a sample space if it assigns a unique numerical value to every outcome, i.e., it is a real-valued function defined on a sample space. • X is a continuous random variable if it can assume every value in an interval, bounded or unbounded (continuous distribution). It can be characterized by a probability density function (pdf) p(x) defined by Pr {a ≤ X ≤ b} =  b a p(x) dx. • X is a discrete random variable if it assumes at most a countable set of different values (discrete distribution). It can be characterized by its probability function (pf), which specifies the probability 275 276 APPENDIX D that the random variable takes each of the possible values from say {x1, x2, . . . , xi, . . .}: p(xi) = Pr{X = xi}. • All distributions, discrete or continuous, can be characterized through their (cumulative) distribution function, the total probability up to, and including, a point x. For a discrete random variable, P(xi) = Pr{X ≤ xi}. • The expectation or expected value E [X] for a random variable X or equivalently, the mean μ of its distribution, contains a summary of the probabilistic information about X. For a continuous variable X, the expectation is defined by E [X] =  ∞ −∞ x p(x) dx. For a discrete random variable with probability function p(x) the expectation is defined as E[X] =  ∀i xi p(xi). For a discrete random variable X with values x1, x2, . . . , xN and with all p(xi) equal (all xi equiprobable, p(xi) = 1 N ), the expected value coincides with the arithmetic mean x̄ = 1 N N  i=1 xi. • The expectation is linear, i.e., for a, b, c constants and X, Y two random variables, E[X + c] = E[X] + c, E[aX + bY ] = E[aX] + E[bY ]. • A convenient measure of the dispersion (or spread about the average) is the variance of the random variable, var [X] or ς2 : var [X] = ς2 = E[(X − E(X))2 ]. In the equiprobable case this reduces to var [X] = ς2 = 1 N N  i=1 (xi − x̄)2 . STATISTICS 277 • For a random sample of observations x1, x2, . . . , xN , the following formula is an estimate of the variance ς2 : s2 = 1 N − 1 N  i=1 (xi − x̄)2 . In the case of a sample from a normal distribution this is a particularly good estimate. • The square root of the variance is the standard deviation, ς. It is known by physicists as RMS or root mean square in the equiprobable case: ς = 1 2 2 3 1 N N  i=1 (xi − x̄)2. • Given two random variables X and Y with expected values E [X] and E [Y ] , respectively, the covariance is a measure of how the values of X and Y are related: Cov{X, Y } = E(XY ) − E(X)E(Y ). For random vectors X ∈ Rm and Y ∈ Rn the covariance is the m × n matrix Cov{X, Y } = E(XY T ) − E(X)E(X)T . The (i, j)th element of this matrix is the covariance between the ith component of X and the jth component of Y . • In order to estimate the degree of interrelation between variables in a manner not influenced by measurement units, the (Pearson) correlation coefficient is used: cXY = Cov{X, Y } (var [X] var [Y ]) 1 2 . Correlation is a measure of the strength of the linear relationship between the random variables; nonlinear ones are not measured satisfactorily . • If Cov{X, Y } = 0, the correlation coefficient is zero and the variables X, Y are uncorrelated. • The random variables X and Y , with distribution functions PX(x), PY (y) and densities pX(x), pY (y), respectively, are statistically independent if and only if...