Linear Algebra: UNIT III: Inner Product Spaces
Least Squares Approximation
Definition and Theorems - Least Squares Approximation
LEAST SQUARES
APPROXIMATION
Let us consider the
following problem: Collect data by taking measurements y1, y2, ... ym
at times t1, t2, …..tm respectively. Suppose
that data (t1, y1),
(t2, y2), ... (tm, ym) are plotted
as points in the plane.
From
this plot there exist an essentially a linear relationship between y and t say
y = ct + d and would like to find the constants c and d so that the line y =
ct+d represents the best possible fit to the collected data.
One
such estimate of fit is to calculate the error E that represents the sum of the
squares of the vertical distance from the points to the line.
(i.e) E = mΣi=1 (yi
‒ cti ‒ d)2
the
line y = ct+d is called the least square line.
it follows that E = ||y−Ax ||2.
Theorem 7
Let
A ∈ Mm×n(F) and
y ∈ Fm. Then
there exist x0 ∈
Fn such that (A*A) x0 = A*y and ||Ax0 ‒y|| ≤
||Ax ‒y|| for all x ∈
Fn. Furthermore, if rank (A) = n, then x0 = (A*A)‒1A*y.
Theorem 8
Let A ∈ Mm×n(F), x ∈ Fn, and y ∈ Fm. Then <Ax,y>μ
= <x, A*y>ν.
Proof:
By
a generalization we know that
<x,
T (y)> = <T*(x), y> = < x, T** (y) >.
Then
<Ax, y>μ = y*(Ax) = (y*A)x = (A*y)*x = ( x, A* y ) n.
Theorem 9
Let A ∈ Mm×n (F). Then rank (A*A)
= rank (A).
Proof:
By
the dimension theorem, we need to show that, for x ∈ Fn, we have
A*Ax=0 if and only if Ax=0. Cleanly, Ax=0 ⇒
A* Ax=0. `So assume that A*Ax=0. Then
0 = <A*Ax, x>ν = <Ax, A** x>μ
= <Ax, Ax>μ ⇒
Ax=0.
Theorem 10
If A is an m×n matrix
such that rank (A) = n, then A*A is invertible.
Proof:
Let
A be an m×n matrix and y ∈
Fm. Define W= { Ax: x ∈
Fn };
(i.e) W=R(LA). Then there exists a
unique vector in W that is closest to y. Call this vector Ax0 where
x0 ∈
Fn. Then || Ax0 − y || ≤ || Ax−y || for all x ∈ Fn. So x0
has the property that E = || Ax0−y || is minimal.
Linear Algebra: UNIT III: Inner Product Spaces : Tag: maths, mathematics : - Least Squares Approximation
Linear Algebra: UNIT III: Inner Product Spaces
Under Subject
Linear Algebra
MA25C02 2nd Semester | 2025 Regulation
Related Subjects
English Essentials II
EN25C02 2nd Semester | 2025 Regulation | 2nd Semester 2025 Regulation
Tamils and Technology தமிழர்களும் தொழில்நுட்பமும்
UC25H02 2nd Semester | 2025 Regulation | 2nd Semester 2025 Regulation
Linear Algebra
MA25C02 2nd Semester | 2025 Regulation
Transforms and its Applications
MA25C03 2nd Semester EEE Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Applied Physics (CE) II
PH25C02 2nd Semester Civil, Agri Depts | 2025 Regulation | 2nd Semester 2025 Regulation
Applied Physics (CSIE) II
PH25C03 2nd Semester AIDS, CSE, IT, CSE(CY) Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Applied Physics (EE) II
PH25C04 2nd Semester EEE Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Applied Physics (ME) II
PH25C05 2nd Semester Mechanical Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Applied Chemistry (CE) II
CY25C02 2nd Semester Civil Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Applied Chemistry (ME) II
CY25C03 2nd Semester Mechanical Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Electron Devices
EC25C01 2nd Semester ECE Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Digital Principles and Computer Organization
CS25C06 2nd Semester AIDS, CSE, IT, CSE(CY) Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Basic Electrical and Electronics Engineering
EE25C01 2nd Semester | 2025 Regulation | 2nd Semester 2025 Regulation
Basic Civil and Mechanical Engineering
GE25C01 2nd Semester EEE Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Data Structures using CPlusPlus
CS25C05 2nd Semester ECE Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Engineering Drawing
ME25C01 EEE, Mech, Agri, EEE Depts | 2025 Regulation | 2nd Semester 2025 Regulation
Data Structures and Algorithms
CS25C04 2nd Semester EEE Dept | 2025 Regulation
Circuits and Network Analysis
EC25C02 2nd Semester ECE Dept | 2025 Regulation | 2nd Semester 2025 Regulation
Engineering Mechanics
ME25C02 2nd Semester Mech, Civil, Agri Depts | 2025 Regulation | 2nd Semester 2025 Regulation
Object Oriented Programming (OOPs)
CS25C07 2nd Semester CSE, CSE(CY) Depts | 2025 Regulation | 2nd Semester 2025 Regulation