-- AlexanderFedotov - 2015-03-03

## 1. Wikipedia links

## 2. Uncorrelated measurements

where the *weight matrix* of dimension
is diagonal and defined as the *inverse of the diagonal covariance matrix for* :
i.e. .

In matrix notation (considering and as columns and respectively), one has
or
In a general case of linear transformation ,
the covariance matrice for is transformed into that for
via . Hence,
With by the definition of , that simplifies to
and
## 3. Correlated measurements

where .
and
## 4. An iterative solution

### 4.1 The procedure

The subset of the parameters corresponding to the group
of indices, can be considered as an column
Let denote the set of indices
that are complementary to indices in .
### 4.2 The step

the (4.2.1) can be written as
which is analogous to eq.(2.2).

Therefore, the solution is given by formulae (2.4):
or
Let us write the last expression via matrices of dimensions
and
such that all the arythmetics is done in rows / columns (or )
while complementary rows / columns contain zeros.
It is noteworthy that
Then (4.2.5) can be written as
This is a representation of the first line of (4.1.3).

The second line of (4.1.3) can be represented with
Summing up (4.2.10) and (4.2.11) gives
where we denoted
### 4.3 steps

#### 4.3.1 as a function of

and
Then, from (4.2.12),
Indeed, by induction:
#### 4.3.2 The covariance matrix of

Comparing this with the covariance matrix of the exact solution, eq.(2.6), yields
### 4.4 steps for

one can build the expression for the case *in one step* as
Substituting (4.3.6) into (4.4.2) gives
Transforming similarly the formula to the one, gives
Then for the case one has
and so on...

## Linear Least Squares |

Sections:

- Least squares http://en.wikipedia.org/wiki/Least_squares
- Linear least squares http://en.wikipedia.org/wiki/Least_squares#Linear_least_squares
- Weighted least squares http://en.wikipedia.org/wiki/Least_squares#Weighted_least_squares

- Linear least squares (mathematics) http://en.wikipedia.org/wiki/Linear_least_squares_%28mathematics%29
- Weighted linear least squares http://en.wikipedia.org/wiki/Linear_least_squares_%28mathematics%29#Weighted_linear_least_squares

- Propagation of uncertainty http://en.wikipedia.org/wiki/Propagation_of_uncertainty

Let with variances be measurements of functions with the known matrix and unknown parameters .

In linear least square method, one estimates the parameter vector by minimizing over the expression

In matrix notation (considering and as columns and respectively), one has

The estimate is the solution of the system of equations

Note, that

Let be uncorrelated measurements as those in the previous section, and ( is an invertible matrix). Then are generally correlated and have the covariance matrix .

With and , one gets

Similarly,

Thus, all the formulae for the correlated measurements are similar to those for the uncorrelated , with the only complication being the replacement of a diagonal weight matrix with a non-diagonal one:

Let indices of parameters be distributed among groups
with sizes respectively.

Consider the following iterative procedure.

- Start with an vector as an initial approximation to .
- Make N steps or, equivalently, iterations for , thus finding vectors by minimizing over at the step :
where is the "point" where as a function of parameters , takes minimum (while the rest parameters are fixed at the values obtained in the previous iteration: ) . - Repeat (4.1.3) infinitly, defining for (i.e. ) .

The iteration consists in finding , a set of parameters, that minimizes

Let
be an submatrix of
built from the columns of with indices belonging to
(in other words, it is what remains
after *removing* columns which have indices ** not** in ).

Similarly, let be an submatrix consisting of the columns of with indices belonging to .

Introducing

Therefore, the solution is given by formulae (2.4):

We define matrices and as follows

We also define the matrix
(subscript stands for *extended*) via the
matrix :

The second line of (4.1.3) can be represented with

Let us define matrices and as

- eq.(4.3.3) holds for :
- and assuming it is true for , leads to

In particular, the eq.(4.3.3) is valid for :

According to the general rule of covariance matrix transformation ( ), eq.(4.3.3) gives rise to

With the expression (4.3.6) of the form

This generalizes to

This topic: Main > TWikiUsers > AlexanderFedotov > AVFedotovLogA > AVFLogA012LeastSquares

Topic revision: r14 - 2017-01-15 - AlexanderFedotov

Copyright &© 2008-2020 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.

Ideas, requests, problems regarding TWiki? Send feedback

Ideas, requests, problems regarding TWiki? Send feedback