Commit 30640e82 authored by Jim Hefferon's avatar Jim Hefferon

lstsqs

parent 16e057b4
No preview for this file type
......@@ -2106,23 +2106,23 @@ beginfig(42) % least squares coin flip
% the axes
pickup pensquare scaled line_width_light
draw z0--z1; % flips axis (x-axis)
label.lrt(btex {\small \textit{flips}} etex,z1);
label.lrt(btex {\scriptsize \textit{flips}} etex,z1);
updown_ticks(9,(10w,0v),(10w,0v));
label.bot(btex {\scriptsize 30} etex,(30w,0v));
label.bot(btex {\scriptsize 60} etex,(60w,0v));
label.bot(btex {\scriptsize 90} etex,(90w,0v));
label.bot(btex {\scriptscriptsize 30} etex,(30w,0v));
label.bot(btex {\scriptscriptsize 60} etex,(60w,0v));
label.bot(btex {\scriptscriptsize 90} etex,(90w,0v));
draw z0--z2; % heads axis (y-axis)
label.ulft(btex {\small \textit{heads}} etex,z2);
label.ulft(btex {\scriptsize \textit{heads}} etex,z2);
sidetoside_ticks(6,(0w,10v),(0w,10v));
label.lft(btex {\scriptsize 30} etex,(0w,30v));
label.lft(btex {\scriptsize 60} etex,(0w,60v));
label.lft(btex {\scriptscriptsize 30} etex,(0w,30v));
label.lft(btex {\scriptscriptsize 60} etex,(0w,60v));
% the line
pickup pencircle scaled line_width_light;
z3=(100w,0.56428571*100v); % 7110/12600
draw z0--z3 dashed evenly scaled .707;
pickup pencircle scaled line_width_dark;
pickup pencircle scaled 1.2pt;
% the points
drawpoint((30w,16v)); drawpoint((60w,34v)); drawpoint((90w,51v));
endfig;
......@@ -2174,6 +2174,7 @@ beginfig(43) % least squares coin flip
x31=x30; y31=y30-exag*(y30-51v);
x33=x32=x31-carryover; y32=y31; y33=y30;
draw z30--z33; draw z31--z32;
pickup pencircle scaled 1.2pt;
drawpoint(z11); drawpoint(z21); drawpoint(z31);
endfig;
......@@ -2193,18 +2194,18 @@ beginfig(44) % least squares money denom vs life
% the axes
pickup pensquare scaled line_width_light
draw z0--z1; % denom axis (x-axis)
label.lrt(btex {\small \textit{denom}} etex,z1);
label.lrt(btex {\scriptsize \textit{denom}} etex,z1);
updown_ticks(10,(10w,0v),(10w,0v));
label.bot(btex {\scriptsize 10} etex,(10w,0v));
label.bot(btex {\scriptsize 30} etex,(30w,0v));
label.bot(btex {\scriptsize 50} etex,(50w,0v));
label.bot(btex {\scriptsize 70} etex,(70w,0v));
label.bot(btex {\scriptsize 90} etex,(90w,0v));
label.bot(btex {\scriptscriptsize 10} etex,(10w,0v));
label.bot(btex {\scriptscriptsize 30} etex,(30w,0v));
label.bot(btex {\scriptscriptsize 50} etex,(50w,0v));
label.bot(btex {\scriptscriptsize 70} etex,(70w,0v));
label.bot(btex {\scriptscriptsize 90} etex,(90w,0v));
draw z0--z2; % heads axis (y-axis)
label.ulft(btex {\small \textit{life (yrs)}} etex,z2);
label.ulft(btex {\scriptsize \textit{life (yrs)}} etex,z2);
sidetoside_ticks(8,(0w,1v),(0w,1v));
label.lft(btex {\scriptsize 4} etex,(0w,4v));
label.lft(btex {\scriptsize 8} etex,(0w,8v));
label.lft(btex {\scriptscriptsize 4} etex,(0w,4v));
label.lft(btex {\scriptscriptsize 8} etex,(0w,8v));
% the line
pickup pencircle scaled line_width_light;
......@@ -2212,7 +2213,7 @@ beginfig(44) % least squares money denom vs life
z4=(0w,(14.1582/12)*v);
draw z4--z3 dashed evenly scaled .707; % .7[white,black];
pickup pencircle scaled line_width_dark;
pickup pencircle scaled 1.2pt;
% the points
z10=(1w,(22/12)*v);
z11=(5w,(15.9/12)*v);
......
This diff is collapsed.
......@@ -2238,12 +2238,14 @@ members of the plane $M$ that are not orthogonal to the dotted line.
But the projection along $\hat{N}$ is orthogonal.
\end{example}
A natural question is:~what is the relationship between the projection
operation defined in this subsection and the operation of orthogonal projection
into a line?
The second picture above suggests the answer\Dash orthogonal projection into
We have seen two projection operations,
orthogonal projection into a line
as well as this subsections's projection into an~$M$ and along an~$N$, and
we naturally ask whether they are related.
The right-hand picture above suggests the
answer\Dash orthogonal projection into
a line is a special case of this subsection's projection;
it is just projection along a subspace perpendicular to the line.
it is projection along a subspace perpendicular to the line.
\begin{center} \small
\includegraphics{ch3.40}
\end{center}
......@@ -2287,7 +2289,7 @@ these two conditions.
\qquad
\colvec[r]{0 \\ 1 \\ 2}\dotprod\colvec{v_1 \\ v_2 \\ v_3}=0
\end{equation*}
We can express those conditions more compactly as a linear system.
Those conditions give a linear system.
\begin{equation*}
P^\perp
=\set{\colvec{v_1 \\ v_2 \\ v_3}\suchthat \begin{mat}[r]
......@@ -2299,7 +2301,7 @@ We can express those conditions more compactly as a linear system.
\end{equation*}
We are thus left with finding the null space of the map represented
by the matrix, that is, with
calculating the solution set of a homogeneous linear system.
calculating the solution set of the homogeneous linear system.
\begin{equation*}
\begin{linsys}{3}
v_1 & & &+ &3v_3 &= &0 \\
......@@ -2316,7 +2318,7 @@ calculating the solution set of a homogeneous linear system.
\begin{example} \label{ex:OrthoCompTwo}
Where \( M \) is the \( xy \)-plane subspace of \( \Re^3 \),
what is \( M^\perp \)?
A common first reaction is that \( M^\perp \) is the \( yz \)-plane, but
A common first reaction is that \( M^\perp \) is the \( yz \)-plane but
that's not right because
some vectors from the \( yz \)-plane are not perpendicular to every
vector in the \( xy \)-plane.
......@@ -2354,7 +2356,7 @@ prior example and taking the natural basis for the $xy$-plane gives this.
If $M$ is a subspace of $\Re^n$ then its
orthogonal complement $M^\perp$ is also a subspace.
The space is the direct sum of the two \( \Re^n=M\directsum M^\perp \).
And, for any $\vec{v}\in\Re^n$,
And, for any $\vec{v}\in\Re^n$
the vector $\vec{v}-\proj{\vec{v}\,}{M}$ is perpendicular to every
vector in $M$.
\end{lemma}
......@@ -2417,7 +2419,7 @@ Therefore equation~($*$) gives that $\vec{w}$ is a linear combination of
Thus this is a basis for $M^\perp$ and
$\Re^n$ is the direct sum of the two.
The final sentence of this result
The final sentence of the lemma
is proved in much the same way.
Write $\vec{v}=\proj{\vec{v}\,}{\spanof{\vec{\kappa}_1}}
+\dots+\proj{\vec{v}\,}{\spanof{\vec{\kappa}_n}}$.
......@@ -2429,17 +2431,19 @@ Therefore $\vec{v}-\proj{\vec{v}\,}{M}$ consists of a linear combination
of elements of $M^\perp$ and so is perpendicular to every vector in $M$.
\end{proof}
We can find the orthogonal projection into a subspace by
following the steps of the proof but the next result gives a
convenient formula.
Given a subspace, we could compute the orthogonal projection into
that subspace by following the steps of that proof: finding a basis,
expanding it to a basis for the entire space, applying Gram-Schmidt
to get an orthogonal basis, and projecting into each linear subspace.
However we will instead use a convenient formula.
\begin{theorem} \label{th:OrthProjMat}
Let $\vec{v}$ be a vector in $\Re^n$ and
let \( M \) be a subspace of \( \Re^n \) with basis
\( \sequence{\vec{\beta}_1,\dots,\vec{\beta}_k} \).
If \( A \) is the matrix whose columns are the \( \vec{\beta} \)'s
then $\proj{\vec{v}\,}{M}=c_1\vec{\beta}_1+\dots+c_k\vec{\beta}_k$
where the coefficients $c_i$ are entries of the vector
Let \( M \) be a subspace of \( \Re^n \) with basis
\( \sequence{\vec{\beta}_1,\dots,\vec{\beta}_k} \) and
let \( A \) be the matrix whose columns are the \( \vec{\beta} \)'s.
Then for any $\vec{v}\in\Re^n$ the orthogonal projection is
$\proj{\vec{v}\,}{M}=c_1\vec{\beta}_1+\dots+c_k\vec{\beta}_k$,
where the coefficients $c_i$ are the entries of the vector
$(\trans{A}A)^{-1}\trans{A}\cdot\vec{v}$.
That is,
$\proj{\vec{v}\,}{M}=A(\trans{A}A)^{-1}\trans{A}\cdot\vec{v}$.
......@@ -2449,27 +2453,30 @@ $\proj{\vec{v}\,}{M}=A(\trans{A}A)^{-1}\trans{A}\cdot\vec{v}$.
The vector \( \proj{\vec{v}}{M} \) is a member of \( M \) and so is a
linear combination of basis vectors
\( c_1\cdot\vec{\beta}_1+\dots+c_k\cdot\vec{\beta}_k \).
Since $A$'s columns are the $\vec{\beta}$'s that can be expressed
as:~there is a \( \vec{c}\in\Re^k \) such that
Since $A$'s columns are the $\vec{\beta}$'s,
there is a \( \vec{c}\in\Re^k \) such that
\( \proj{\vec{v}\,}{M}=A\vec{c} \).
% (this is expressed compactly with matrix
% multiplication as in
% \nearbyexample{ex:OrthoCompOne} and~\ref{ex:OrthoCompTwo}).
The vector \( \vec{v}-\proj{\vec{v}\,}{M} \) is perpendicular to each member
of the basis so we have this.
To find $\vec{c}$ note that
the vector \( \vec{v}-\proj{\vec{v}\,}{M} \) is perpendicular to each member
of the basis so
% (again, expressed compactly).
\begin{equation*}
\zero
=\trans{A}\bigl( \vec{v}-A\vec{c} \bigr)
=\trans{A}\vec{v}-\trans{A}A\vec{c}
\end{equation*}
Solving for \( \vec{c} \) (showing that \( \trans{A}A \) is invertible is an
exercise)
and solving gives this
(showing that \( \trans{A}A \) is invertible is an
exercise).
\begin{equation*}
\vec{c}=\bigl( \trans{A}A\bigr)^{-1}\trans{A}\cdot\vec{v}
\end{equation*}
gives the formula for the projection matrix as
$\proj{\vec{v}\,}{M}=A\cdot\vec{c}$.
Therefore
$\proj{\vec{v}\,}{M}=A\cdot\vec{c}=A(\trans{A}A)^{-1}\trans{A}\cdot\vec{v}$,
as required.
\end{proof}
\begin{example}
......
......@@ -13,4 +13,5 @@ data=[[1870,268.8],
var('slope,intercept')
model(x) = slope*x+intercept
g=points(data)+plot(model(intercept=find_fit(data,model)[0].rhs(),slope=find_fit(data,model)[1].rhs()),(x,1860,1960),color='red',figsize=3,fontsize=7)
g.save("four_minute_mile.png")
\ No newline at end of file
g.save("four_minute_mile.pdf")
File added
# from: http://www.mathgoespop.com/2011/09/moneyball.html
# Math Goes Pop!
#
# team wins losses payroll cost per win
#
# Oakland Athletics 103 59 $40,004,167 $0.388
# Minnesota Twins 94 67 $40,225,000 $0.428
# Montreal Expos 83 79 $38,670,500 $0.466
# Florida Marlins 79 83 $41,979,917 $0.531
# Cincinnati Reds 78 84 $45,050,390 $0.578
# Pittsburgh Pirates 72 89 $42,323,599 $0.588
# Los Angeles Angels 99 63 $61,721,667 $0.624
# Tampa Bay Rays 55 106 $34,380,000 $0.625
# San Diego Padres 66 96 $41,425,000 $0.628
# Chicago White Sox 81 81 $57,052,833 $0.704
# Philadelphia Phillies 80 81 $57,954,999 $0.724
# Houston Astros 84 78 $63,448,417 $0.755
# Kansas City Royals 62 100 $47,257,000 $0.762
# St. Louis Cardinals 97 65 $74,660,875 $0.770
# Colorado Rockies 73 89 $56,851,043 $0.779
# San Francisco Giants 95 66 $78,299,835 $0.824
# Seattle Mariners 93 69 $80,282,668 $0.863
# Milwaukee Brewers 56 106 $50,287,833 $0.898
# Baltimore Orioles 67 95 $60,493,487 $0.903
# Atlanta Braves 101 59 $93,470,367 $0.925
# Toronto Blue Jays 78 84 $76,864,333 $0.985
# Detroit Tigers 55 106 $55,048,000 $1.001
# Los Angeles Dodgers 92 70 $94,850,953 $1.031
# Arizona Diamondbacks 98 64 $102,819,999 $1.049
# Cleveland Indians 74 88 $78,909,449 $1.066
# Chicago Cubs 67 95 $75,690,833 $1.130
# Boston Red Sox 93 69 $108,366,060 $1.165
# New York Yankees 103 58 $125,928,583 $1.223
# New York Mets 75 86 $94,633,593 $1.262
# Texas Rangers 72 90 $105,726,122 $1.468
sal = [40, 40, 39, 42, 45, 42, 62, 34, 41, 57, 58, 63, 47, 75, 57, 78, 80, 50, 60, 93, 77, 55, 95, 103, 79, 76, 108, 126, 95, 106]
wins = [103, 94, 83, 79, 78, 72, 99, 55, 66, 81, 80, 84, 62, 97, 73, 95, 93, 56, 67, 101, 78, 55, 92, 98, 74, 67, 93, 103, 75, 72]
var('a, b, t')
model(t) = a*t+b
data = zip(sal,wins)
fit = find_fit(data, model, solution_dict=True)
model.subs(fit)
p = plot(model.subs(fit),(t,30,130))+points(data,size=25,color='red')
p.save('moneyball.pdf')
\ No newline at end of file
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment