Least Squares Approximation: Example Solved Problems

LEAST SQUARES APPROXIMATION

WORKED EXAMPLE PROBLEMS

Example 1

Find the least squares line and error for the following data (1,2), (2,3), (3,5), (4,7)

Solution:

The straight line y = ct + d

Thus y = 1.7 t is the least squares line.

The error E may be computed directly as E = || Ax₀ ‒ y ||².

Let E = || y−Ax₀||²

 || y – Ax₀ || = √< y – Ax₀ ; y – Ax₀ >

= √[ (0.3)² + (−0.4)² + (− 0.1)² + (0.2)² ]

= √[0.09 +0.16+0.01 +0.04] = √0.3

E = || y – Ax₀ ||² = 0.3

Note: If a polynomial y = ct² + dt +e of degree at most 2.

Example 2

Use the least square approximation to find the best fit with both (i) a linear function, and (ii) a quadratic function, compute the error for both cases (−3, 9) (−2, 6) (0, 2) (1, 1).

Solution:

(i) The straight line y = ct + d (Linear function)

y = ct + d = ‒2t + 5/2 is the least squares line.

The Error is denoted by E = || y−Ax₀||²

 || y – Ax₀ || = √< y – Ax₀ ; y – Ax₀ >

= √[ (0.5)² + (− 0.5)² + (−0.5)² + (0.5)² ]

= √ [ 0.25 +0.25 +0.25 +0.25] = 1

 || y – Ax₀ ||² = 1.

 The error is computed as E = 1.0.

 (ii) Let y = ct² + dt +e be a quadratic equation. (Quadratic function)

 c = 1/3 ; d= ‒4/3; e = 2.

 The quadratic function is obtained as y = (1/3)t² ‒ (4/3)t + 2

The error is E = || y – Ax₀||²

 ⇒ E.= 0.

Example 3

Find the least squares line and error for the following data (1,0), (2,1), (3,3).

Solution:

The straight line is y = ct + d

 Here c=3/2 and d=−5/3.

⇒ y = ct+d

⇒ y = (3/2)t ‒ 5/3, is the least square line.

The error E may be computed directly be using the relation

E = || Ax₀ − y ||²

 Let E = || y−Ax₀ ||²

 || y – Ax₀ || = √ <y−Ax₀; y−Ax₀ >

= √ [ (‒1/6)² + (‒1/3)² + (1/6)² ]

= √ [ 0.027 + 0.1 + 0.027 ]

= √0.154

 || y − Ax₀ ||² = 0.154

The error E is computed as E=0.154

Example 4

Find the minimal solution to the following system of linear equations: x+2y−z=1; 2x+3y+z=2 and 4x + 7y−z=4.

Solution:

Let us consider

To find the minimal solution for the given problem, let us find the solution u to AA* X=b. The minimal solution of the system AX=b is obtained by A* u where u is the solution of AA* X = b.

Let us now find the value of AA*.

Now let us consider the system of equations

6x+7y+19z = 1

7x+14y+28z = 2

19x+28y +66z = 4

                      ...(1)

The above system of linear equations can be solved by any one of the numerical methods.

(Such as Gauss elimination method (or) Gauss Jordan method).

By using Gauss−elimination method one such solution is

  The minimal solution is X= A* u.

The minimal solution to the given system of equations is

x=0.287, y=0.431 and z = 0.142