-- AlexanderFedotov - 2015-03-03

Linear Least Squares

1. Wikipedia links

2. Uncorrelated measurements

Let with variances be measurements of functions with the known matrix and unknown parameters .

In linear least square method, one estimates the parameter vector by minimizing over the expression

where the weight matrix of dimension is diagonal and defined as the inverse of the diagonal covariance matrix for : i.e. .
In matrix notation (considering and as columns and respectively), one has

The estimate is the solution of the system of equations

In a general case of linear transformation , the covariance matrice for is transformed into that for via . Hence,
With by the definition of , that simplifies to

Note, that


3. Correlated measurements

Let be uncorrelated measurements as those in the previous section, and ( is an invertible matrix). Then are generally correlated and have the covariance matrix .

With and , one gets

where .



Thus, all the formulae for the correlated measurements are similar to those for the uncorrelated , with the only complication being the replacement of a diagonal weight matrix with a non-diagonal one:

4. An iterative solution

Let indices of parameters be distributed among groups with sizes respectively.

The subset of the parameters corresponding to the group of indices, can be considered as an column
Let denote the set of indices that are complementary to indices in .

Consider the following iterative procedure.

  1. Start with an vector as an initial approximation to
  2. Make N steps or, equivalently, iterations for , thus finding vectors by minimizing over at step :
    where is the "point" where as a function of parameters , takes minimum (while the rest parameters are fixed at the values obtained in the previous iteration: )
  3. Repeat (4.3) infinitly, defining for    (i.e. ) .
One can expect that
Edit | Attach | Watch | Print version | History: r14 | r8 < r7 < r6 < r5 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r6 - 2015-03-18 - AlexanderFedotov
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright & 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback