Orthogonal Projection Matrix

[coarse gyro]

Hello! I'm a bit confused on where to start for 4A. i already was able to prove 3, but i dont know how we would begin with 4a. does anyone have a hint?

[versed valley]

1. Do not ping the Moderators, unless someone is breaking the rules.
2. Do not ping the Helper Moderators, unless there is a conflict between helpers.
3. Do not ping other members randomly for help.
4. Ask your question and show the work you've done so far. If you've posted a screenshot of a question, specify which part you need help with.
5. Wait patiently for a helper to come along.
6. If the Helper has answered your question, remember to thank them with the Mathematics Ranks bot and close the thread with:

+close
Feel free to nominate the person for helper of the week in #helper-nominations
If you're happy with the help you got here, and the server overall, you can contribute financially as well:

[coarse gyro]

unfortunately my professor has not gone over anything like this in class :/

i should say i'm working on 4a.a

my guess is so show something is the sum of proj_{e1}(v) + proj_{e2}(v)

or something like that

[magic bronze]

This may be useful to you.
https://math.stackexchange.com/questions/112728/how-do-i-exactly-project-a-vector-onto-a-subspace/112743#112743

[coarse gyro]

would i be able to find the columns of that matrix with that? bc as far as i know that's the equation for a vector in w

[magic bronze]

Well, do you know how you can represent dot products with matrices?

[coarse gyro]

(U^T)(U)

i believe?

[magic bronze]

Well, that's for the same vector, but yeah.

u·v = u^T v.

[coarse gyro]

oh shoot

[magic bronze]

In any case, you can try to adapt that to the formula in the post I shared above.

I haven't done it myself, but it should work.

[coarse gyro]

i'll try that, thank you! can i leave this open while i try it?

[magic bronze]

Of course!

[coarse gyro]

oh one more question, should i be saying the vector (in the link you sent) should be a matrix?

i believe i understood you wrong bc i'm stuck

bc for a 2 by 2 matrix (a, b, c, d) i got that v in the basis w is a + d, which doesnt sound right at all

[magic bronze]

Hm...
Try stacking vectors in some way.

As in, by rows or columns.

[coarse gyro]

or maybe i should be doing a vector in R3 actually

[magic bronze]

Then it will be a matrix.

[coarse gyro]

i'll try a matrix 3 x 2 matrix (v1, u1, v2, u2, v3, u3)

oh i see what i did wrong

i used e1 and e2 in terms of R2 instead of R3

okay so i got the projection of v in W is (u1, v1) + (u2, v2)

[magic bronze]

What are u1, v1, u2 and v2, again?

[coarse gyro]

theyre the original entries from the original matrix i used
(v1 u1)
(v2 u2)
(v3 u3)

i stacked 2 arbitrary vectors in R3

i have a feeling that wasnt the right thing to do now 💀

[magic bronze]

Oh, that's what you mean.

[coarse gyro]

i think i found a theorem we havent gone over in class that might help?

the dash is how they denote a dot product

since W has an orthogonal basis

just need to show its orthonormal maybe?

[magic bronze]

Yeah, that!

We need to arrive to that.

[magic bronze]

Yeah, orthonomality is needed, I believe.

[coarse gyro]

perfect!

oh well we know its orthonormal bc theyre bases

so their magnitude must be 1?

and its given that theyre orthogonal?

no wait thats not right

oh wait we know W is orthonormal bc they have the std basis vectors have magnitude 1 and are orthogonal

[magic bronze]

I believe we can do it like this.

That sum should be equal to UU^T.

[coarse gyro]

yep that makes sense

and so each of those results being summed up can be a column in our matrix?

or would sum ui*ui^t be our matrix

[magic bronze]

Yeah.

I think...

[coarse gyro]

hmm, okay

[magic bronze]

But, as you can see here, the vectors do need to be normalized.

[coarse gyro]

yeah exactly

[magic bronze]

Oh, I found a version where that's not needed.

In that case the matrix becomes U(U^T U)^(-1) U^T.

[coarse gyro]

ill try that way bc ive seen that equation mentioned before

[magic bronze]

In statistics, perhaps?

Because that's where I just remembered it from 😄

[coarse gyro]

possibly!

hm i think i just want to go with U^TU

i just dont know how to derive that honestly

[magic bronze]

Yeah, same. I did derive it some time ago myself, but I don't quite remember how to do that now.

[coarse gyro]

thats super fair

[magic bronze]

I wonder if it's possible to reason about the properties of this matrix just from this form.

As in, only 0 and 1 as eigenvalues, etc.

[coarse gyro]

thats what the next question is about

i have to show that the only possible eigenvalues are 0 and 1

and that A^2 = A

this is hurting my head though, im so confused

wait i think i did this wrong

if {b1,...,bk} are the orthonormal bases

and n = 3

wait no nvm

we're doing that rn

my only issue is we have now have the matrix of W terms of W, which is just [e1 e2]

i just want to say rhats the matrix 😭

[magic bronze]

Well, this one is actually easy if you use A = U(U^T U)^(-1) U^T.

As for eigenvectors/-values... Well, maybe geometric interpretation can help?

Any vector in the subspace stays where it was, while every vector perpendicular to it becomes zero.

[coarse gyro]

thats what i was thinking

but unfortunately i cant do those until i do 4A.a and b

[magic bronze]

Yeah, I see.

[coarse gyro]

bc i have to use my previous work

[magic bronze]

I'm a bit rusty on the theoretical linear algebra stuff, as I don't really use it specifically. I just use formulas.

I'm a chemical technology student, so the applications are more important for me, usually.

[coarse gyro]

i wish! this is an elementary linear algebra class

[quick charm]

On a theoretical standpoint, this is the characterization of a projection

a linear map f: V -> V is a projection if and only if f² = f

[coarse gyro]

exactly

[quick charm]

But I'm assuming that you don't have access to basic linalg?

[coarse gyro]

nope

my professor is very focused on proofs, which is very different from the other professors

anyway, i need to work on this matrix problem

[quick charm]

What do you realistically have access to?

as in, how do you define a projection, and how do you define an orthogonal projection?

[coarse gyro]

well we have a projectionon a vector in an orthogonal basis as the sum of the projections in each basis on that vector

[quick charm]

you kind of have something circular here

[coarse gyro]

sorry let me find the actual definition in my textbook

[quick charm]

That's a projection on a line, alright

an orthogonal projection on a line*

[coarse gyro]

yes

so i somehow have to find a general matrix for that

[quick charm]

so you define orthogonal projections only?

as in, you don't concretely define projections even

[coarse gyro]

yeah rhats all we have

just dot product over dot product

[quick charm]

that is... exotic

[magic bronze]

Why? That's also how I learned about projections.

[quick charm]

Well, (I am biased but) I find it quite atypical to start from orthogonal projections, without going through projections in general

since when you start with linear algebra, you might not have a good idea of the geometry of the vector space you work with

meaning that a notion of orthogonality might not be properly defined

[coarse gyro]

well we know what orthogonality is

[quick charm]

but that matters not, I'm here to help you out

First of all, to show that if A is an orthogonal projection matrix, then A^2 = A

[coarse gyro]

nonono thats not the problem im working on rn

i'm still working on my original one

[quick charm]

I thought it was?

Ahh ok

[coarse gyro]

thats a future problem

and i need to finish the one im working on now to be able to do it 😭

[quick charm]

Alright so

4b?

[coarse gyro]

4a

i've learned the matrix needs to be square and that it's U*U^T = proj_w(v)

[quick charm]

for 4a, I think you can pretty much eyeball it, let me explain

you have a vector in 3D, say v = [x, y, z]

and you want a transformation that projects v on Span(e1, e2), i.e. that sets the third coordinate to zero and leaves the other two as is

[coarse gyro]

so its [e1 e2 0]?

[magic bronze]

Moreover, if you do (4a) using the most fundamental approach - seeing the action on orts - you can even find out what the eigenvectors are.

[quick charm]

pretty much, yeah, but you gotta verify that your guess is correct

[coarse gyro]

okay, so i can verify by computing U*U^T and see if that gets us the same answer as proj_w(v) in equation 4?

[coarse gyro]

.

[quick charm]

well if U U^T is your projection matrix, you have to verify that: [x, y, 0]^T = U U^T v = <e1, v>e1 + <e2, v> e2 + <0, v> 0

[coarse gyro]

okay yes that makes sense

thank you

[quick charm]

which leads to question 4b

if you have a vector v, then if you express v in a certain basis B, the orthogonal projection will set some of the coordinates in that specific basis to zero, and leave the rest as is

[coarse gyro]

and that v is still [x, y, 0]^T?

[quick charm]

nah, I think question 4b wants you to generalize

[coarse gyro]

that makes more sense

[quick charm]

v = [x1, ..., x_n] in the standard basis

but it can be, say, v ~ [y_1, ..., y_n] in another basis

[coarse gyro]

yes yes

[quick charm]

and a projection would just seek to set some of the y_i to zero

the question is, which basis?

[coarse gyro]

yes

[quick charm]

Think about that, and it'll give you good answer elements for question 4b

[coarse gyro]

i dont know how we would continue i'll be honest

we know that it needs to somehow have orthogonal columns in it i think?

since the 0 blocks

[quick charm]

You can always construct an orthogonal basis of any subspace, the question revolves entirely on how

[coarse gyro]

we havent learned about the grant schmidt process yet unfortunately

[quick charm]

[coarse gyro]

i know 😭

i only know about it bc i read ahead

[quick charm]

Ok, what about: construct a basis (not even orthogonal) of a vector space?

[coarse gyro]

i know that yes

[quick charm]

cool

we can work with that

[coarse gyro]

sick

[quick charm]

so the point is, you project on a subspace W

so what you want to do first of all is to get a basis of W composed of unit norm vectors

say, (w_1, ..., w_k)

[coarse gyro]

yep

[quick charm]

then you can complete it with some more vectors to make a basis of R^n

(w_1, ..., w_k, v_1, ..., v_l)

the point is that a projection on W will have no coordinate in v_1, ..., v_l

as it can be expressed directly as a linear combination of w_1, ..., w_k

and not just that, but the coordinates on w_1, ..., w_k don't change

[coarse gyro]

yeah that makes sense

hm, okay

[quick charm]

try to use these pieces of info

to construct the matrix of the projection

[quick charm]

in this basis

[coarse gyro]

well my immediate guess is to go (w_1, ... ,w_k, 0, 0, ... )

[quick charm]

if you're listing the coordinates by column

then the first k columns wouldn't be w_1, ..., w_k no

[coarse gyro]

i dont know i'll be completely honest

[quick charm]

remember that this is a coordinate system

for example, x ~ [x_1, ..., x_k, ..., x_n] expressed in this basis

basically means that x = x_1 w_1 + ... + x_k w_k + ... + x_n v_l

the point is that now, the projection Px will be x_1 w_1 + ... + x_k w_k

[coarse gyro]

oh yes that makes sense

[quick charm]

so its new coordinates in that basis are [x_1, ..., x_k, 0, ..., 0]

[coarse gyro]

so why was my original suggestion wrong?

[quick charm]

Now question of the century: what matrix turns [x_1, ..., x_k, ..., x_n] into [x_1, ..., x_k, 0, ..., 0] ?

if you can answer that, then you got what you were looking for

it's similar to what we did earlier: what matrix turns [x, y, z] into [x, y, 0] ?

[coarse gyro]

the identity matrix w/ a 0 column?

or wait no

we turned e3 into a column of 0's

[quick charm]

it looks similar

if you want a definitive hint

just read question 4b again

[coarse gyro]

so its that block mateix

matrix

[quick charm]

yeah, simple and easy

the question is just about presenting it a bit better than I did, with the tools of your lecture notes

[coarse gyro]

simple and easy 😭

[quick charm]

but that's the idea

you construct a basis of W, complete it however you want

then the projection matrix in that full basis is just that

for the reasons I specified

[coarse gyro]

okay, please dont get mad, so my guess is if we have any orthonormal basis for W, the projection is always similar to that block matrix?

i'm so sorry if thats wrong

[quick charm]

it doesn't matter if it's orthogonal or not

[coarse gyro]

oh just any basis??

thats insane

[quick charm]

getting an orthonormal basis of W is done with the gram schmidt process

but you told me you can't use it

[coarse gyro]

yeah unfortunately

[quick charm]

so you just need a basis change even if the new basis is not orthonormal

basis changes are invertible

so it's fine

but yes, to answer your question, no matter the basis of W that is selected, and how it is completed

the projection matrix in that basis will look like that

you don't need to write the basis change matrix explicitly

as said in the question, it suffices to show what basis would lead to such a representation

[coarse gyro]

sick, thanks yall :]

+close

[hearty oyster]

Please thank your Helpers before closing!

Please thank the helpers who assisted you by clicking the buttons below. You can thank each helper only once. Once you're done, click "Close Post" to close this thread.

[hearty oyster]

Thank you for your feedback! Rion has been awarded 1 . They now have 64 . They have 3 daily left for today.

[hearty oyster]

Thank you for your feedback! LordDarpinger has been awarded 1 . They now have 910 . They have 2 daily left for today.