ive been using a class on udemy to learn linear algebra and its been going great so far
#linear-algebra
cinder basalt
costs money but so does a book (unless you can find it for free)
why not both?
granite mirage
native tide
ok i worked out some concrete examples in R^n and i think I understand now
When you choose an orthonormal basis of course B = I because each b_ij of B is the inner product of the ith and jth basis vectors
That makes a lot more sense now
Thank you so much for your help!
royal harbor
Okay so I just finished a course this semester which was multivariable calc and series. I cant lie I got fucked a bit but yeah.
To you guys, how was the difficulty of linear algebra compared to multivariable?
hardy adder
Depends on what type of linear algebra but general (computation based) linear algebra I found much easier than multivar
royal harbor
Depends on what type of linear algebra but general (computation based) linear al...
interesting
I mean i know both courses where I go are hard, and the multi course I took was 50% multi and 50% series which really really sucked cause it was a ton of material in a short time
but idk im hoping linear is a bit easier for me to grasp
hardy adder
If it's more theoretical it may be a little difficult but that is what this server is for 
you can always ask for people to help contribute to your intuition
royal harbor
Yeah I suppose. Im bored so im going over some linear material rn
before the next semester
hardy adder
Very good 
royal harbor
we used this resource partially for multi so hopefully its good for linear https://cruzgodar.com/teaching/notes/linear-algebra
hardy adder
Looks pretty standard based on the toc
Seems pretty sparse in terms of explanations (outside of chapter 1), it'll be important that you make sure to do at least some exercises each time they appear
royal harbor
Seems pretty sparse in terms of explanations (outside of chapter 1), it'll be im...
yeah
usually the prof would have reading from this then problems from another textbook
royal harbor
@hardy adder any knowledge of the emory one i sent? or thoughts
hardy adder
Not particularly other than it's more standard
You'll probably find it nicer to work out of
Especially for self-study
I think the explanations are much clearer and the pacing more reasonable
as compared to the first
I much prefer the presentation of vector spaces in this one to the other
Although at the cost of a little generality
Much more concrete (less abstract)
potent reef
The honor is yours :3
jagged pendant
oop
sand hedge
👀
potent reef
Why did you open a separate linear algebra channel
potent reef
lol
jagged pendant
enough people were asking for it
potent reef
lobbying always works
sand hedge
u can't ban me
jagged pendant
potent reef
Thats what seperates good lobbists from bad ones 
jagged pendant
get out before 
stoic pythonBOT
Colen:
ember pilot
sand hedge
how calculate determinant
wintry steppe
cool channel
random wasp
oh cool
random wasp
is this advanced maths? o:
hoary osprey
so this is calculus 2 right?
sand hedge
calc 1.5
sand hedge
I did group theory before limits 
hoary osprey
i did calculus b4 limits
wintry steppe
uwu new linear channel
hoary osprey
lol
sand hedge
u w u
broken hawk
no channel description ‼
broken girder
yay now i know 'advanced maff'
half ice
Wow I never realized how much I needed this until I got it
sand hedge
see what good happens when woog listens to a wild Colen 
broken hawk
so, reminder to everyone: cayley-hamilton is magic
half ice
Yes of course
tranquil ermine
broken hawk
imo it makes sense, but perhaps I’d throw it into the other category
some linalg is often taken e.g. in high school or by non-math majors who may not care about e.g. group theory
it makes sense to separate it out imo
dreamy fiber
Is LA really higher level math tho 🤔
proper crescent
So the tricky thing here is that there's two things you can mean when you say linear algebra
You can mean, how do I compute matrix stuff?
Which would be under the low math category
Or you could be doing more theoretical stuff, which probably would technically fall under abstract algebra
broken hawk
I feel like since even theoretical linalg is fairly low-level on the grand scale of things it’d make sense to
a) have it in this channel
b) move this channel to the MATHEMATICS category
I mean you have number theory there too, and stats, both topics that at least where I am are largely university topics
ember zenith
loo
I'm doing up to Linear algebra and differential equations in high school
So that's pretty guud
jagged pendant
sure, I don't mind moving it to the other section.
I wasn't too sure myself.
moved @broken hawk @trim ermine @proper crescent @dreamy fiber
frigid bronze
Hey baby
jagged pendant
ok I just deleted the channel description lol
broken girder
#MakeLAHigherMaffAgain
sand hedge
broken girder
We've been accustomed to the LA being in the higher math category for god knows how long now...
sand hedge
exactly its been like it for i'd say at least 3 hours, this change is unacceptable @jagged pendant
broken girder
A change like this would wreak havoc upon our already-established socio-economic structure...
One does not simply break tradition
stiff oasis
Is this a new channel?
trim ermine
Step 1: Make a matrix mapping our anger at LA being removed from higher-mathematics to the complex plane
sand hedge
sorry I can only make woogian matrices
trim ermine
Nah, it should be a power series
wintry steppe
smh
broken hawk
consider: power series with matrix entries
frosty vapor
wth
sand hedge
ember zenith
Move linear algebra above Calculus dammit
broken hawk
it… is?
rocky cove
f* and f Are endomorphisms
barren plank
sure
rocky cove
And its Matrix is the transpose of the f one
broken hawk
f* is the dual of f I assume?
rocky cove
Yeah
glass sluiceBOT
🆙 | Mintman agent 47 leveled up!
barren plank
$\langle f(x, y \rangle = \langle x, f^*(y) \rangle$
stoic pythonBOT
mniip:
barren plank
I was trying to make sense of this
rocky cove
In the scalar product something was missing
But this i did demonstrate
The 3rd is the point where i am stuck
$\langle f(x), y \rangle = \langle x, f^*(y) \rangle$
stoic pythonBOT
Mintman agent 47:
barren plank
okay, consider $y \in \ker f^*$
stoic pythonBOT
mniip:
rocky cove
This $Ker(f*)=(Im(f)T)$
stoic pythonBOT
Mintman agent 47:
barren plank
for all x, $\langle f(x), y \rangle = \langle x, f^*(y) \rangle = 0$
stoic pythonBOT
mniip:
rocky cove
Yeah
barren plank
that is, $\forall z \in \operatorname{Im} f, \langle z, y \rangle = 0$
stoic pythonBOT
mniip:
barren plank
that is $y \in (\operatorname{Im} f)^\bot$
rocky cove
stoic pythonBOT
mniip:
rocky cove
Yup!
barren plank
this argument works both ways
so we've proved 3.1
for 3.2 you can use 3.1 with a bit of algebraic manipulation
barren plank
$(\ker f)^\bot = (\ker f^{**})^\bot = ((\operatorname{Im} f^)^\bot)^\bot = \operatorname{Im} f^$
rocky cove
Is the demonstration rigourous
stoic pythonBOT
mniip:
rocky cove
?
barren plank
which demonstration
rocky cove
3.1
barren plank
which part are you doubting?
stoic pythonBOT
mniip:
barren plank
that's pretty normal
barren plank
I said the argument works both ways
rocky cove
Oooh
barren plank
the other direction is just the same steps in backwards order
I could place <=> signs at each step to make it rigorous
rocky cove
Neat proof ?
barren plank
short and elegant
rocky cove
Panda!
Pandou
Pandi
@barren plank i'll see
I am already struggling with 3.2 too
hollow wolf
$$\langle AX, Y \rangle = \mathrm{Tr}( (AX)^T Y) = \mathrm{Tr}(X^T (A^T) Y)^T) = \langle X, A^T Y \rangle$$
stoic pythonBOT
Astréas:
hollow wolf
A = Mat_B (f)
rocky cove
Je pouvais faire avec les matrices ?
hollow wolf
Oui
rocky cove
Pck j'ai fait la 1.a avec les matrices mais pour ça j'ai pas osé
hollow wolf
Oui on a le droit
barren plank
yuck, why do you have to introduce a basis in this otherwise completely universal theorem
hollow wolf
You can always introduce a basis since we are working in a finite dimensional space
barren plank
yea but that's kinda ugly to be honest
rocky cove
I dont Even know if it's the same name in american
barren plank
what is?
hollow wolf
$$y \in \mathrm{Ker}(f^) \Leftrightarrow \forall x \in E, \langle x, f^(y) \rangle = 0 \Leftrightarrow \forall x \in E, \langle f(x), y \rangle = 0 \Leftrightarrow y \in \mathrm{Im}(f)^{\perp}$$
stoic pythonBOT
Astréas:
hollow wolf
Inner Product Space
barren plank
either that or euclidean space, yes
rocky cove
3.2 is the same ?
barren plank
3.2 is slightly different, I haven't looked at how to prove it in terms of points yet
rocky cove
Seems not intuitive imo
barren plank
it's probably the same
rocky cove
But i am crap so
broken hawk
I learned euclidean space as real inner product space (as opposed to unitary for example, which would be complex inner product)
broken hawk
“them” being what?
barren plank
me? I'm 20
hollow wolf
I think Inner product space is meant to be a general space with an inner product. Euclidean means the dimension is finite
At least that is the difference once translated in french
rocky cove
Panda have you checked 3.2 ?
hollow wolf
La 2 cest f = f**
avec un orthogonal en plus
$$\mathrm{Im}(f^*) = \mathrm{Ker}(f^{**})^{\perp} = \mathrm{Ker}(f)^{\perp}$$
D'après la 3a)
rocky cove
Ah ok!
stoic pythonBOT
Astréas:
broken hawk
I think Inner product space is meant to be a general space with an inner product. Euclidean means the dimension is finite
idk about the dimension but I’m pretty sure euclidean implies that it’s a space over ℝ (and not ℂ or 𝔽₂)
hollow wolf
Oh yeah sure
But I think finite dimension is also underlined
Function space are not called Euclidean, I think
Et pour la dernière
$$\langle x, f(f^(x)) \rangle = \langle f^(x), f^(x) \rangle = | f^(x)|^2$$
stoic pythonBOT
Astréas:
rocky cove
Whut ?
hollow wolf
Donc si x est dans Ker(ff*), il est aussi dans Ker(f*) vu que le produit scalaire cest 0 scalaire x qui est nul
Et le dernier une inclusion suffit, on peut conclure ensuite par dimension
rocky cove
Juste comment tu justifies le passage de la première à le deuxième inégalité ?
hollow wolf
cest la définition de l'adjoint
rocky cove
Mais non on a pas l'adjoint c'est Hp
hollow wolf
$$\langle x, f(y) \rangle = \langle f^*(x), y \rangle$$
stoic pythonBOT
Astréas:
rocky cove
Je pense que le but de l'exo c'est de nous faire voir les propriétés de l'adjoint mais je peux pas vraiment admettre un truc HP si ?
hollow wolf
Cest pas admis
Cest défini juste au dessus ...
Tas le droit de changer le f de place à condition de lui ajouter une étoile ...
Et vu que l'ajout de deux étoiles cest lui retirer l'étoile 😄
hollow wolf
C'est la combinaison de la formule de 1b) et de la troisième propriété de 2)
rocky cove
Est ce que si f est symétrique alors f* est antisymetrique ?
hollow wolf
Non
rocky cove
Enfin la mat
hollow wolf
Non plus
rocky cove
Ok d'acc
hollow wolf
Si la matrice est symétrique, sa transposée est égale à elle même
rocky cove
Mdrr et la 4b c'est n'importe quoi les hyperplan c'est pas au programme aussi
Ah ouais je suis con...
4b....
Ouahhj
hollow wolf
Un hyperplan cest un sous espace de dimension n-1
rocky cove
C'est la définition complète ?
hollow wolf
En dimension finie, ca suffit comme definition
broken hawk
what’s it you have to show? λ is an eigenvalue, show that E(λ)^⊥ is a subspace?
unless I’m missing something it won’t always be one of dimension n-1, since E(λ) could be of any dimension ≤ n
but if that’s what you have to show, that’s just the same proof as showing that S^⊥ is a subspace for any set S
which would just go:
let v, u be orthogonal to any element in S. pick s∈S arbitrarily. Then ⟨v + αu, s⟩ = ⟨v, s⟩ + α⟨u,s⟩ = 0, so v+αu is also orthogonal to s. but since s is arbitrary, it is orthogonal to all elements in S. Therefore, S^⊥ is a subspace
broken hawk
what is vect(U)
broken hawk
ah okay, yea, so you really only need the propery that Span(U) is a one-dimensional subspace then
that it’s an eigenvector etc is irrelevant
broken hawk
do you have proven the theorem that if W is a subspace of V then
V = W ⊕ Wᵀ?
or, at least that dim(W) + dim(Wᵀ) = dim(V)?
rocky cove
broken hawk
really, you need to prove two things:
- span(u)ᵀ is a subspace
- dim(span(u)ᵀ) = dim(V) - 1
rocky cove
Yeah this theorem has already been demonstrated in class
broken hawk
okay so you have 2) already then
rocky cove
Done!
broken hawk
so you just need to show span(u)ᵀ is a subspace, which I’ve demonstrated above how to do
rocky cove
Feel delighted man
I dont need to dem that span(U)t is a subspace no?
Oh yeah an hyperplane needs to be a subspace and have a dim =n-1
broken hawk
yes you do, that’s step one
rocky cove
Done thanks!
sharp lodge
If i do the QR decomposition of A=VP. And then the QR decomposition of transpose(P). I will obtain A=VMU. Is M diagonal? Will i obtain the singular value decomposition?
ember zenith
can someone explain to me eigenvalues and eigenvectors?
half ice
A fairly long conversation of me explaining the concept. Feel free to come back and ask!
winter reef
there are some great courses about linear algebra on youtube, 3blue1brown, maththebeuatiful and other professor I forgot the name but he looks kinda like the flex tape guy
I'd recommend all
dusky mango
I missed our last lecture aboutthe "festlegungslemma" which states that an isomorphism of vectorspaces is sufficiently described by the images of the basis vectors. But neither on google nor in the books can I find this lemma, can anyone tell me how it's actually called / link to an an article about it?
wintry steppe
linear transformations are described by the images of the basis vectors
This is because of the linearity of the transformations and the fact that any vector in the space can be written as a linear combination of the basis vectors.
dusky mango
yup I know, I would like to read up on it though, do you know the name by any chance?
wintry steppe
"Change of basis"? There isn't really a name for this fact.
dusky mango
It seems to be pretty important though, I can't find it anywhere
But thanks for trying to help :)
broken hawk
I‘ve never heard of that and I did linalg in german
it is very important though, yes
west widget
wat
broken hawk
pretty easy to see too:
let’s say (v1, v2… vn) is a basis. then there’s a unique way to write any v∈V as a linear combination of the basis elements and so you can write
T(v) = T(Σaᵢvᵢ) = ΣaᵢT(vᵢ) by linearity
qed
there’s not much more to it, maybe do some exercises to convince yourself of its truth
mellow hull
Anyone who can clarify a few basic concepts for me? 😃
mellow hull
Is it understood correctly that we have N observations, each of which consists has its own M dimensional vector?
Also, is there a specific reason that vector x is transposed, and all elements of vector X are transposed? 😃
vague thicket
First of all, transposing a column vector basically gets you the same vector, but written as a row vector; and I think this is pretty clear to you, since it's not part of your questions.
Now, I'm pretty sure it's standard notation to consider vectors as column vectors by default. Now look at the way the matrix X is defined: each row of the matrix must be a row vector x_i; but since vectors are all column vectors by default, you need to transpose it first, if you want to fit it in the matrix that way.
When it says that x = [x_1, ..., x_M]^T, your book is basically saying what I said above, that is that all vectors are written in column form by default: in fact, if you transpose the vector on the right side of the equality, what you get is a column vector. Writing [x_1, ..., x_M]^T, or writing [x_1, ..., x_M] vertically, is exactly the same thing.
mellow hull
So it is correctly understood that an entry in X i.e x_1^T, corresponds to a row vector x?
and the reason that x = [x_1...x_M]^T, is because we by default write vectors as column vectors, and thus it is equivalent?
vague thicket
Well... I wouldn't say that an entry of the matrix X is a row vector: remember that a matrix is ultimately an array of numbers, so each of those numbers are the actual entries of X. It's just that, when you look at the rows of this matrix, they happen to be the row vectors x_i that were discussed earlier (because this is how you built the matrix X, after all).
The reason why you find x_1^T, x_2^T, ..., in the definition of X instead of just x_1, x_2, ..., is that, as you correctly say, these would be column vectors by default: and stacking column vectors vertically... well, that would basically give you a longer column vector, and certainly not the N x M matrix you want to define instead. So first you transpose them so they can be written horizontally, then you stack them one on top of another... and bam!, that's how you get the matrix X.
mellow hull
Ok, thanks, got it now
rare siren
yo guys i got that really quick question
if i got a matrix lets say 2x2 matrix to make it simpler, and it has 2 eigenvectors
and i want to find the basis for the eigenspace
do i use both the eigenvectors or just one of them?
broken hawk
depends
each eigenvalue has its own space
so e.g. if they're eigenvectors with different eigenvalues, then each will be a basis vector of its own space
and there are two eigenspaces
rare siren
but lets say they want me to find A=PDP^-1
broken hawk
but if they have the same eigenvalue, you have to check if they're independent, and if yes, use both
broken hawk
oh, yea, you need to have a separate, linearly independent vector in each column of P
rare siren
yeah excatly
broken hawk
each corresponding to its value
rare siren
can i take 1 from the first lamdba and one from the second?
broken hawk
so if you have twice the same value, then you put it there twice in D
rare siren
and build a basis of those two?
but if they have the same value u cant build a basis in R^n
broken hawk
but you need to make sure that the vectors are independent
it is entirely possible for a matrix to not be diagonalizable btw
rare siren
yeah i know
broken hawk
but assuming there are n linearly independent eigenvectors
rare siren
im reading about it right now, i used eigenvalues to try find a space, but the vectors wernt linierly independtent
broken hawk
you put those in P, and then write the corresponding values into D
rare siren
yeah i know that, but if im trying to find the vectors
am i allowed to take some of them from one of the lamdba
and some from the other lamdba?
broken hawk
you will have to
rare siren
i always have to ?
broken hawk
lemme make a concrete example
rare siren
like lets say my eigenvalues are (L-4)(L-3)
A-(L-3)I
and then try to find vectors
and then i go A-(L-4)I
and then find one vector?
combine those and thats my P?
broken hawk
the 3x3 matrix A has eigenvalues 1 and 2.
let's say rank(A- 1I) = 1. So you know there are two lin. indep eigenvectors here, and you need both. And rank(A-2I)=2, so you need to find one here
rare siren
whats rank :O?
broken hawk
and the diagonal matrix will be (1,1,2)
rare siren
im stil lhella new to this m8
broken hawk
...how are you looking for eigenvectora and don't know what the rank of a matrix is?
rare siren
well im studying in swedish
broken hawk
thats's an incredibly weird progression
rank of a matrix = how many independent columns it has
rare siren
yeah im missing alot of words
broken hawk
but yea you need to find as many vectors as you're missing to get full rank. e.g. if you have a 4x4 matrix and rank(A - λI) = 2 then you have to find two vectors
broken hawk
then there was only one eigenvalue to begin with
rare siren
i see
ive been doign 2x2 matrix
so that explains why ive only been needing to use one eigenvalue
broken hawk
yea there the options are:
-two values, one vector each
-one value with two vectors
-one value with only one vector -> not diagonalizable
broken hawk
np
sand hedge
off-topic is best topic tbh
mellow hull
Anyone familiar with PCA? 😃
glossy hemlock
@mellow hull I know a bit about it
stone drum
Same
tulip galleon
hey guys
how can i compute 3D unit vector using two angles ?
horizontal and vertical
can someone give me the formula ?
half ice
That's the common "math" way to do it, but it might not be what you're looking for if you're a programmer. Let me know.
tulip galleon
i am programmer
grave plank
what is the rigorous definition of rank?
half ice
The dimension of the subspace the column vectors span
grave plank
and what is span
half ice
All linear combinations of a set of vectors
The span of a set of vectors is itself a set
grave plank
My book defines rank as the integer r that satisfies the conditions 1) The matrix A has a minor of order r which does not vanish. and 2) Every minor of the matrix A of order r+1 and higher (if such exists) vanishes
Matrix A is just a general matrix of size nxm
broken hawk
I mean that's not wrong, but it's also not very enlightening
rank of a matrix = number of linearly independent columns = number of linearly independent rows = dimension of the span of the column vectors = dimension of the image of L_A, where L_A is the linear transformation given by left multiplication with A
each of these is a sensible and rigoeous definition of rank
glass sluiceBOT
🆙 | Sascha Baer leveled up!
deep cedar
@broken hawk can also relate it to the number of features under PCA
for the stats boyos
broken hawk
I have no idea what that acronym stands for
deep cedar
Principle Component Analysis
https://en.m.wikipedia.org/wiki/Principal_component_analysis
Principal component analysis - Wikipedia
meh all it does is find the eigenvectors
broken hawk
okay, I also have no idea what that concept is ^^ is it in any way related to singular value decomposition?
deep cedar
https://en.m.wikipedia.org/wiki/Singular_value_decomposition
Singular value decomposition - Wikipedia
broken hawk
yea I know what the svd is
oh yea it says here “the principal components transformation can also be associated with the svd”
deep cedar
ya its all related
its when u dont have a square matrix or have sparsity that there become distinctions
broken hawk
svd works for nonsquare though?
deep cedar
they are basically the same thing cept U can be unnecessarily large in svd because of the latter
u can drop the unnecessary columns tho i think
tulip galleon
hey guys , so i am a programmer and i use
this equation
this.direction = [
Math.sin(vAngle) * Math.cos(hAngle),
Math.sin(vAngle) * Math.sin(hAngle),
Math.cos(vAngle)
];
but no matter what is my input
it outputs values that belongs to the positives x , y ,z 3D space
shouldnt it output a values in negtive if i input values larger than 90 ?
half ice
Cool, glad to see that equation worked for you! Your programming language likely works in radians, not degrees.
Check to see if there's a degree version of sin and cos, or multiply vAngle and hAngle by 2π/360 before using them
tulip galleon
i convert them in radins in the lines before this
vAngle *= glMatrix.toRadian(vAngle);
hAngle *= glMatrix.toRadian(hAngle);
tbh i think the equation is fine and the code is also fine
but i think i dont know how the view matrix works
which might be what is causeing the problem but i see it normal
half ice
The view matrix?
tulip galleon
Camera Matrix i its 4*4 Matrix that is used to transform the whole 3d space from the world space to the camera space
its like moving the world around the camera not the other way
grave plank
Use quaternions
tulip galleon
i tried to but keeping track of this extra vector in my code might cause me some troubles later
idk
tidal nebula
Use a matrix.
wintry steppe
quaternions are dank af
grave plank
dont know much about them but ive heard they are computationally tedious
half ice
Easier than matricies, but can do less than matrices. Matricies usually win out
balmy coral
how was determinant discovered? especially for generalized nxn
or it is just randomly thought up rules that seems to work
half ice
Note the denominator of any system of equations is the determinant of the matrix. I imagine they played with that for a while, and it evolved from there
balmy coral
I mean it's easy to see for 2x2, but how did they extend that to nxn
trail valley
The déterminant of nxn is the alternating sum of the elements in column p times the determinant of the (n-1)x(n-1) matrix formed by all the elements not in the row or column of p
dull kettle
hello guys, All square matrices without a zero row are invertible, this 'zero row' is referring to a row consisting of entirely zeros right?
hollow cedar
yeah
wintry steppe
All square matrices without a zero row are invertible? That's incredibly false.
dull kettle
yeah, I am supposed to evaluate if its true or false
wintry steppe
$$\left[
\begin{array}{cc}
1 & 1 \
1 & 1
\end{array}
\right]$$
stoic pythonBOT
TendentiousTorturousTopics:
dull kettle
What about this? I know that statement 3 is definitely wrong, not sure about 1 and 2
empty copper
A^T + B^T = (A + B)^T, and the inverse of a transposed matrix is the transposed of the inverse
So 2 is definitely true
I'm wondering if there's a counterexample to 1
Oh yeah here's an easy one
dull kettle
ohh yeahh, (A^T)^-1 = (A-1)^T
empty copper
Take the 2x2 identity matrix and split it into two nonzero (lower triangular) matrices A and B that each contain only a single 1, the other entries being 0
Then both A and B have determinant 0, thus not invertible
1 false, 2 true, 3 false
dull kettle
thank you! @empty copper
empty copper
Np 🍮
olive breach
I'm taking this course https://courses.edx.org/courses/course-v1:UTAustinX+UT.5.05x+1T2019/course/
strong snow
@wintry steppe You here?
broken hawk
that doesn’t look like linalg to me
strong snow
It's not
broken hawk
I mean it is but it’s not what I’d expectg to be taught in linalg
broken hawk
but rather in like,well, (the high school version of) algebra
it annoys me that there’s two entirely distinct subjects typically called algebra
I barely even see how solving equations for x and group theory is related at all
like sure it’s all connected by stuff like abel-ruffini theorem etc
but high school algebra should prolly be called sth like “intro to equations” or “intro to variables”
ember zenith
um excuse me?!!??
x + 2 = 7 is OBVIOUSLY as complicated as inversing transposed matrices
You may think
x = 5 because 7-2=5
but how do you know
broken hawk
the funny thing is that’s not even a linear equation
normal gust
you don't know what you even know
normal gust
oof
leaden ermine
Hello, could someone show me an example using numbers of finding the L2 norm? Meaning if we had two points, say (3,2) and (4,7), how would we calculate the L2 norm of these points? I get how to calculate the L2 norm of just the x, or just the y, but how about both? Thx
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
In this case, you have a norm on elements of the vector space $\mathbb R^2$, so you have a metric, i.e. Euclidean distance, taking in two points.
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
but your question isn't mathematically well-formed, since you're saying something like calculating the L2 norm of two points.
leaden ermine
Hello sorry I am not discribing it properly
describing
So a bit more background
Lets say we have a d-dimensional multivariate Gaussian distribution (X1....Xd)
with a mean vector u in R^d and covariance matrix Rdxd
where u subscript i denotes the ith element of u, and summation ij to denote the element at i'th row and j'th column of summation. (summation is the variance here I guess?)
is it making sense so far? @wintry steppe
wintry steppe
wait what does that have to do with calculating a norm?
leaden ermine
Ok so here is that part lol
Let x, y ∈ Rd be two independent samples drawn from N (µ, Σ). Give expression for E�x�2
2
and E�x − y�2
2. Express your answer as a function of µ and Σ. �x�2 represents the ℓ2-norm
of vector x.
wintry steppe
not of summation; $\Sigma$ means a covariance matrix in this case.
leaden ermine
the question marks are Ellxll2
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
Yeah sorry i am having ahard time describing it because i dont understand the question well
I know the basic l2 norm vector but i think the question is a bit more complicated, not sure how to approach it
i guess the covariance only comes into play when we are looking at x and y
Ellxll2 can probably be calculated without that right
wintry steppe
it's asking for $\mathbb E[||x||^2_2]$ and $\mathbb E[||x-y||^2_2]$
glass sluiceBOT
🆙 | Shocks leveled up!
wintry steppe
yes
leaden ermine
sorry i cant post the signs properly 😦
That is correct though
well
actaully
It should be squared as well
squared of each of them
wintry steppe
ah I was a bit confuddled by it not being squared
leaden ermine
Ellxll2^2
wintry steppe
that would be something hard to calculate
leaden ermine
;P
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
So lets start with the first one, just the x one
wintry steppe
okay
leaden ermine
Yep!
wintry steppe
so you recognize that the squared l2 norm is just the sum of squares of the components, right?
leaden ermine
Yes
wintry steppe
expectation is linear
leaden ermine
i am getting confused with the x,y and dimensions etc
wintry steppe
so it's just the sum of expectations of the squares
leaden ermine
Is this just the norm of two points?
or a vector with 2 values
I am just having trouble understanding what exactly the norm is, i know its a magnitude
wintry steppe
the L2 norm of a vector is defined as $||x||^2_2 = \sum\limits_{i=1}^n x_i^2$
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
I know if we had (0,1,3) it would just be sqrt(1+9)
leaden ermine
Ok great, so in that picture you posted, x itself is just a vector
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
well there are two parts, the first one is just x, the second is x-y
wintry steppe
yes
leaden ermine
so what does the y represent? simply the y coordinate of the two points?
leaden ermine
ah hmm..
wintry steppe
and then you get a quantity z=x-y from them
and it's asking what the expectation of the squared l2 norm of z is
wintry steppe
Let's say you draw two samples (0,0,0) and (1,1,1) from the distribution
leaden ermine
ok so this is a 3 dimensional distrib
wintry steppe
then the value of ||x-y|| = sqrt 3
leaden ermine
is x your first point and y your second?
wintry steppe
I said that we sampled x and y from the distribution
leaden ermine
ok, let me ask a very dumb question here haha
It doesnt tell me how many dimensions this is
simply d-dimensional
so here x,y doesnt mean x coord, y coord, it just means two diff samples
leaden ermine
so x could be (3,6,1) or (1), it doesnt say and doesnt matter, correct?
wintry steppe
no
leaden ermine
so x and y represent two different vectors on a plane
and the dimensionality is unknown
wintry steppe
the dimensionality is determined by the dimensionality of the covariance matrix
yeah it doesn't matter in the end
leaden ermine
ok great, i was confusing it because i kept thinking (x,y) were points
when in reality they are vectors
so further than that, now if we speak about the covariance matrix, which is just how x varies with respect to y right?
well..
Actually no forget that
Covariance would be when one dimension changes, how it affects the others?
wintry steppe
covariance matrix is defined as $\Sigma_{ij} = \text{Cov}(x_i, x_j)$
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
where $x_i$ and $x_j$ are the ith and jth components of the vector, respectively
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
if its a vector, how does it have row and column?
shouldnt it just have one row and many column
or one column and many rows
sorry i know these are silly questions
but im just trying to understand the problem and not very familiar with LA
wintry steppe
a vector just can be indexed
leaden ermine
I know in statistics a covariance would be how one variable changes with respect to another
wintry steppe
A vector-valued random variable is just a measurable function from the sample space to some space R^d
which really means that a vector-valued random variable with dimension 1 is the same as a scalar random variable
leaden ermine
where d represents dimensionality, and R is just the space itself?
yes
that makes sense
In this problem, we dont know the dimensionality as we said before, so these vectors are drawn in a multi-dimensional field. But if they are vectors, they only change on one plane?
wintry steppe
yes d is the dimensionality
it turns out that in general, things will depend on the dimensionality
but if you solve this problem, you'll find a nice expression at the end that only sorta depends on the dimensionality
in that you don't really need to write the dimensionality in there
leaden ermine
Yes youre correct, im just trying to envision this problem and what it looks like 😃 But ok just for the sake of an example im going to assume its a three dimensional space
So in this three dimensional space, we have two vectors, x and y
correct so far?
wintry steppe
yes
leaden ermine
awesome, so we want to first calculate the l2 norm of x, squared
so if the x vector had the coordinates (3,6,-1) the l2 norm would be sqrt(3^2 + 6^2 + -1^2)
then if we square that, it would get the l2 norm squared?
id try to use the LaTeX but it would take me a while and i dont want to make you wait
wintry steppe
yes
so the squared l2 norm of a vector is just the sum of squares of its coordinates
wintry steppe
yes
leaden ermine
However, this problem wants me to express it as a function of the mean and variance. So in this case, the mean
what does that mean? no pun
How do we take the mean of a vector
Or how is that possible I guess
wintry steppe
It wants you to express $\mathbb E[||x||_2^2]$ as a function of $\mu$ and $\Sigma$
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
of course you can take the mean of several vectors
leaden ermine
yes, of several
wintry steppe
just like how you can talk about expectation of a vector-valued random variable
wintry steppe
x is a sample of a random variable
we can certainly talk about $\mathbb E[X]$ if $X \sim N(\mu, \Sigma)$
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
right?
leaden ermine
so we have a normal distribution of this random variable
so what is this X vector made up of? we just took a sample three timesfrom this RV?
and the Y vector is another sample of three from this RV?
wintry steppe
hm
leaden ermine
or maybe X is a random sample of one RV
wintry steppe
pretty much
wintry steppe
no x and y are samples of the same R.V.
wintry steppe
no
the distribution is of vector-valued r.v.s
so we can assign a density $f(\mathbf x)$, where $\mathbf x$ is a vector
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
when we take a sample, we're sampling from a vector-valued distribution
wintry steppe
I suppose you can think of it as sampling d random variables $x_1, x_2, x_3, ..., x_d$, where each is distributed normally, but then they're not independent; they have covariance with one another
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
where x1, x2, x3 are different RVs but do affect eachother
i just keep getting so caught up on where i think x1, x2, x3 would be the i'th pull from the same variable
i kind of get the stat part, but the LA part with the matrices just throws me for a loop
so X is made up for three normally distributed random variables that are not independent of eachother
glass sluiceBOT
🆙 | Shocks leveled up!
leaden ermine
and Y is made up of the same three normally distributed RVs that are not independent of eachother
but with diff values
each RV represents one dimension
wintry steppe
yes
leaden ermine
awesome I got it now
ALRIGHT
So
this first Vector X
we will be computing the L2 norm squared
so its simply X1^2 + X2^2 + X3^2
and for the L2 norm of X - Y squared it would be (X1-Y1)^2 + (X2-Y2)^2+(X3-Y3)^2
wintry steppe
yes
leaden ermine
great, it makes sense 😃 So then how do I express them as a function of the mean and covariance
mu and summation
er
sigma
wintry steppe
so $\mathbb E[||X||_2^2] = \mathbb E[X_1^2 + X_2^2 + X_3^2] = \mathbb E[X_1^2] + \mathbb E[X_2^2] + \mathbb E[X_3^2]$
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
yep
wintry steppe
We have $\mathbb E[X_1^2] = \text{Var}(X_1) - \mu_1^2$
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
Ahh, yes
wintry steppe
so $\mathbb E[X_k^2] = \Sigma_{kk} - \mu_k^2$
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
the expectation of the variable squared, is the variance of the variable minus the expectation of the variable, then squared
wintry steppe
and therefore $||X||2^2 = \sum\limits{i=k}^d \mathbb E[X_k^2] = \sum\limits_{i=k}^d \Sigma_{kk} - \sum\limits_{i=k}^d \mu_k^2$
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
youre so good at LaTeX
leaden ermine
im going to try to write my hw in LaTeX you inspired me haha
wintry steppe
$\sum\limits_{i=k}^d \Sigma_{kk} - \sum\limits_{i=k}^d \mu_k^2 = \text{tr}(\Sigma) - ||\mu_k||_2^2$
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
and you're done
wintry steppe
$\Sigma$ is a dxd matrix
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
For example, if you want the standard Gaussian distribution, we set $\mu = 0$ and $$
\Sigma = \left[
\begin{array}{ccc}
1 & 0 & 0 \
0 & 1 & 0 \
0 & 0 & 1
\end{array}
\right]
$$
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
and the trace is the sum of the elements along the diagonal
$\Sigma_{ij} = \text{Cov}(x_i, x_j)$, so if $i=j$, then $\Sigma_{ij} = \text{Cov}(x_i, x_j) = \text{Cov}(x_i, x_i) = \text{Var}(x_i)$
stoic pythonBOT
TendentiousTorturousTopics:
leaden ermine
and that last statement is for our first questio nright
where we just use X
err wait..
no.
so i and j represent the rows and columns right
of our matrix
or mayb enot
wintry steppe
yes
leaden ermine
we had x1 x2 and x3
wintry steppe
i is the ith row, and j is the jth column
wintry steppe
no
the matrix is always 2 dimensional
$$\Sigma = \left[
\begin{array}{cc}
\text{Cov}(X_1, X_1) & \text{Cov}(X_1, X_2) \
\text{Cov}(X_2, X_1) & \text{Cov}(X_2, X_2)
\end{array}
\right]
$$
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
that's for 2-dimensional vectors
leaden ermine
so it just becomes sigma = Cov(X1,X2) * Cov (X2,X1)?
maybe its just me, but it feels so much harder when they dont specify the dimensionality
i know it doesnt matter
but if we have a ton of dimension, wouldnt there be a lot more variables
and more Covariances?
wintry steppe
yes
the number of covariances is d^2
Sigma is a matrix so that's what sigma is equal to
wintry steppe
nah
leaden ermine
Great thank you so much, this part doesnt look as difficult and ill attempt to answer it first
Find the distribution of Z = αiXi + αjXj , for i ∕= j and 1 ≤ i, j ≤ d. The answer will belong
to a familiar class of distribution. Report the answer by identifying this class of distribution
and specifying the parameters.
alpha represents this parameter im assuming
so each dimension has a parameter
drawing a bit of a blank, I know for a normal distribution it should be something like aX + b right? not sure if that applies here
I know all of our RVs are Normal
wintry steppe
I'm a bit sleepy right now, so I'm also drawing a bit of a blank
Expectation comes linearly
leaden ermine
Ah its ok, i can try to figure it out on my own and if i run into a wall ill post it again tomorrow 😃 But i would assume the sum of two normally distributed variables would also be some ort of normal distributed variable
not 100% sure what the question is gettin at but yeah
Sum of normally distributed random variables
This means that the sum of two independent normally distributed random variables is normal, with its mean being the sum of the two means, and its variance being the sum of the two variances (i.e., the square of the standard deviation is the sum of the squares of the standard deviations).
wintry steppe
ehhh sum of two normally distributed random variables is only normal if they're independent
leaden ermine
ah
True..
Found this piece
If they are dependent you need more information to determine the distribution of the sum.
If X
and Y are iid and X+Y and X−Y are independent then X and Y are normally distributed (and then so are X+Y and X−Y
).
If X
and Y form a bivariate normal distribution, then their sum is normal. This implies that the conditional distribution of Y given X is normal, the regression of Y on X is a straight line, and the variance of Y conditional on X does not depend on X. Similarly for the distribution of X given Y.
for whatever thats worth haha.
wintry steppe
oh if X and Y form a bivariate normal distribution, then their sum is normal. They do form a bivariate normal.
leaden ermine
How does this alpha parameter come into play ? Their coefficients
or whatever it means by specifying the parameters
wintry steppe
the alpha just scales it
scaling a normal just gives another normal with mean $\mu\alpha$ and variance $\sigma^2\alpha^2$
stoic pythonBOT
TendentiousTorturousTopics:
wintry steppe
well, I suppose they're bivariate normal
so the covariance matrix looks like: $$
\left[
\begin{array}{cc}
\Sigma_{ii}\alpha_i^2 & \Sigma_ij\alpha_i\alpha_j \
\Sigma_{ji}\alpha_j\alpha_i & \Sigma_{jj}\alpha_j^2
\end{array}
\right]
$$
stoic pythonBOT
TendentiousTorturousTopics:
sand hedge
why is stats tainting my precious linear algebra channel 
wintry steppe
because vectors?
the variance of the sum is actually just the sum of all of the entries in the covariance matrix lololol
leaden ermine
dont worry i will have more linear algebra questions tomorrow 😃
broken girder
help pls
I know it cant be C because there are two y values for x = 0, hence its not a function
but what about between A and B?
south hound
Hmm i think it's A because there are many points of y=0 in B, which makes it a higher degree polynomial. I'm sorry if I'm wrong I'm not good at this.
broken girder
I also think so. I think it must make U turns many times to hit those points, so it has to be higher degree.
wintry steppe
@broken girder weirdly, C contains two points having the same x coordinate: (0,0) and (0,-1)
so there's no polynomial that goes through all points of C
so there are two things:
- either we forget about C because it can't be interpoloated
- we consider the interpolatng polynomial of C{(0,0)} or C{(0,-1)}. Both would have the same degree: 2
if we consider 2), the answer would have to be C
but maybe cuz they say "choose the set that can be interpolated"
we'd have to consider A
oh sry I didn't see you already said "I know it cant be C because there are two y values for x = 0, hence its not a function"
hmm then yeah I agree with cat-lover: A
the idea is indeed what he said, but we'd have to "prove it" I guess
well A has 5 points so the interpolating polynomial would have degree <=4
B has 6 points so the interpolating polynomial would have degree <=5. let's call it P(X)
the thing is
as Cat-lover said
there are points having the same value
this gives information, by Rolle's theorem, that the derivative P'(X) equals 0 somewhere there
ok -2 and -1 have the same value, 0
thus P'(c_1)=0 for some c_1 between -2 and -1
same argument we get c_2 between -1 and 0, c_3 between 0 and 2, and finally c_4 between 3 and 4
notice that Rolle's theorem says that c_i is inside the open interval between the two points
this shows that c_1,...,c_4 are all distinct points
hence P'(X) has at least 4 distinct roots
wintry steppe
so this really proves, without calculations, that A is the correct answer.
broken girder
yup thats a really rigorous proof, thanks
wintry steppe
It's a pleasure :)
hollow cedar
√(λ(M^2)) is always real for any
i)real matrix M
ii)real, symmetric matrix M
Could someone help prove why either or both are true?
Here, λ represents Eigenvalues.
I think part i is true, and part ii isn't.. for one, part i can have a rectangular matrix but for part ii, since it says symmetric matrix, it must be square.. Am I thinking straight?
broken hawk
if part i is true, so is part ii
M² only makes sense for square matrices in the first place
since the product of an n×m matrix with itself only makes sense when m=n
broken hawk
okay, so first of all to rephrase this, the question is whether the eigenvalues of M² are nonnegative.
a matrix has nonnegative eigenvalues if it’s positive semidefinite
if it is positive semidefinite, then vᵀAv ≥ 0 for all v
so vᵀMMv = (Mᵀv)ᵀ(Mv) ≥0
if M is symmetric, then this is the dot product between Mv and itself, so it is ≥ 0 (and 0 iff Mv=0)
so (ii) is true
as for (i) I believe a rotation matrix by 90 degrees should be a counterexample
its square will have eigenvalues -1
broken hawk
what does MvM mean?
broken hawk
that’s a fundamental property of the dot product
since it is an inner product
v·v ≥ 0, and =0 iff v=0
hollow cedar
right, I can't think of a counter example to that so it is true.
broken hawk
you could show (ii) a bit differently too:
since M is symmetric, it is diagonalizable with real eigenvalues (spectral theorem). write M = QΛQ⁻¹. Then M² = QΛQ⁻¹QΛQ⁻¹ = QΛ²Q⁻¹, where Λ² now is a diagonal matrix with nonnegative values
hollow cedar
secondly, I didn't understand what you did for part i there
broken hawk
so M² is diagonalizable and has only nonnegative eigenvalues
hollow cedar
why should rotation matrix by 90 dgrees squared have eigenvalue -1?
yeah, got that part
broken hawk
may I just say proof left as an (easy) exercise to the reader?
but the basic idea was that a rotation matrix by 90 degrees is not diagonalizable (over ℝ) and so the argument about the diagonal matrix won’t work. further, that matrix is essentially analogous to i, and i² = -1
squaring it will straight up give you the negative identity
which is trivially a matrix with negative eigenvalues
hollow cedar
but umm
oh right
negative eigenvalues rooted would be complex
not real
so i is false
broken hawk
yea, basically instead of thinking about real/complex roots I found it easier to just think about positive/negative values ^^
for (ii) I’d say the diagonalization proof is nicer, but it requires heavier tools
broken hawk
the fact that the dot product is positive-definite is pretty fundamental, the spectral theorem (all symmetric matrices are diagonalizable) may not be
hollow cedar
I see, I'm not comfy with spectral theo cuz I just did it a couple days back
broken hawk
oh actually it’s pretty easy to see that the dot product is positive definite
hollow cedar
but I'll delve deeper and come back to this
hollow cedar
yeah all elements squared and added
broken hawk
that’s v₁² + v₂² + …
broken hawk
each of those are nonnegative
broken hawk
oof svd
hollow cedar
suppose, there wasn't SVD in there
broken hawk
1 can definitely not be true because M may not be diagonalizable at all
glass sluiceBOT
🆙 | Sascha Baer leveled up!
broken hawk
why is this bot a thing
hollow cedar
if it was just √(λ(M^2)) = λ(M)
would the question change at all?
1 can definitely not be true because M may not be diagonalizable at all
Why?
broken hawk
rotation matrix isn’t diagonalizable
but has an svd
…isn’t diagonalizable in ℝ that is
it is in ℂ
but even in ℂ there are matrices you can’t diagonalize
while the svd always exists
hollow cedar
oh right, orthogonal matrix can't have eigenvalue decomposition
is what you meant
right?
broken hawk
yea it can, identity is orthogonal
hollow cedar
umm?
broken hawk
I simply mean that the matrix $\begin{bmatrix} 0&-1\1&0 \end{bmatrix}$ has only complex eigenvalues
stoic pythonBOT
Sascha Baer:
hollow cedar
where are the complex eigen values?
broken hawk
so λ(M) = i, -i
hollow cedar
I see 1 and 1 as its eigen values?
broken hawk
yea, diagonalization is this
hollow cedar
why can't rotational matrix be diagonalised again?
broken hawk
think about it
what would it mean for a rotation matrix to have an eigenvector?
specifically in 2D
broken hawk
eigenvectors must remain parallel to how they started under the action
anything that is rotated is not an eigenvector
only stretching
broken hawk
except for those that rotate by 0 or 180 degrees
hollow cedar
why is that?
broken hawk
but note: diagonalizable does not imply symmetric
I’ll let you think it through why those are symmetrical
as for the others, well, because they don’t have (a full set of) eigenvectors
so they can’t be symmetrical
because if they were they could be diagonalized
and also from a purely pragmatic perspective, rotation matrices always have a sin(θ) on one side of the diagonal and a -sin(θ) on the other
hollow cedar
So any symmetrical matrix is a rotational matrix that rotates by 360 or 180 degrees, correct?
broken hawk
how did you come to that conclusion now?
hollow cedar
wait
broken hawk
definitely not
the matrix that is all 2s is a symmetric matrix
and does not at all rotate ^^
hollow cedar
they aren't cuz the columns aren't orthonormal
yeah well
does not at all rotate
is equivalent to
rotating by 0 or 360 degrees, no?
broken hawk
well, I mean, sure, but that matrix does other things
not all matrices are rotations ^^
hollow cedar
I mean, would there be any counter example to the statement I made?
broken hawk
which one?
hollow cedar
all symmetrics are rotational, by 0 or 180 deg?
broken hawk
any matrix that is symmetrical but not orthogonal?
broken hawk
the only rotation matrices that are symmetrical are the identity, and the identity but with two -1s somewhere on the diagonal
which correspond to 0 degrees rotation, and 180 degree rotation in some plane
as you can obviously see, most symmetric matrices are not those
broken hawk
sure is not, yea ^^
broken hawk
I don’t think you’re being pedantic in the right way
hollow cedar
so that statement, as dumb as it sounds, is still always right, right?
broken hawk
in my opinion, that matrix does not rotate at all
as in
not “it rotates by 0”
“it does something which cannot be described as a rotation”
hollow cedar
uhmm
broken hawk
I mean it moves all base vectors to the vector (2,2,…2)
so every vector did a completely different motion
how can that possibly be called rotation?
rotation is a rigid motion in a 2-dimensional subspace
but no 2-Dimensional subspace is even kept alive
broken hawk
I’ve seen two definitions of rotation