numerical-analysis

[brave crypt]

I have been reading books about neural network and ML.

[wide spear]

[brave crypt]

And I understand this structure somewhat:

But I have a few doubts. May I ask you?

[wide spear]

Ok

@brave crypt also knows about ML I think

[brave crypt]

Thank you 😊

8da, you please also help me.

[wide spear]

Well, 8da is at work right now

[brave crypt]

It's okay.

You are smart enough 😀

My first question is I know there is upscale happening in encoding.

But why we there are four outputs? 😕

[wide spear]

So a model output is produced at each scale level

Do you remember how the loss formula calculated the loss using all four scale levels

Why they choose to do this, I cannot answer

[brave crypt]

Yes, Gradient Descent.

So, Will it calculate gradient descent in last three outputs and update the parameters.

After that we will get final output.

[wide spear]

Not sure

[brave crypt]

I see 😀

No worries.

Thank you for clearing out many things.

[brave crypt]

@brave crypt Would it be possible for you to help me? 😀

I'll probably be here and there in about 4 hours or so. Are you talking about the same pdf you posted earlier? (I might take a look earlier if I have some downtime at work)

Yes, The same PDF.

Thank you very much 😀

Also, If you can help me to figure out this formula then I would be grateful to you 😀

[tall solar]

Can you repost the pdf?

[brave crypt]

Sure

@tall solar https://arxiv.org/pdf/1905.12120.pdf

[fleet sail]

does parallel computing allow things like using givens rotation in jacobi method to be theoretically reduced from O(n^2) to O(n) per sweep?

well acctually no

because u would need n processors or smthn lol

idk how many processors those supercomputers have

oh damn 100k processors

[pine jetty]

fajitas

[wide spear]

[tall solar]

Hey sorry I was testing something.

I do wanna ask something. Suppose I had a third order tensor $T \in R^{n\times m\times p}$ with slices along the first mode $T^{(i)}$.

Is running svd on each slice equivalence to running svd on a block diagonal matrix with block diagonals as the slices?

Recall that the eigenvalues of a block diagonal matrix are the eigenvalues of the diagonal matrices.

[pine jetty]

fajitas

[wide spear]

So you're asking if you have $X=U_1\Sigma_1V_1$ and $Y=U_2\Sigma_2V_2$ then does $\begin{bmatrix}X&0\0&Y\end{bmatrix}=\begin{bmatrix}U_1&U_2\end{bmatrix}\begin{bmatrix}\Sigma_1&0\0&\Sigma_2\end{bmatrix}\begin{bmatrix}V_1\V_2\end{bmatrix}$?

[pine jetty]

黏土人

[wide spear]

This is true

[pine jetty]

fajitas

[wide spear]

Why do you think it wouldn't be the case?

[tall solar]

Eh I just was thinking about block diagonalization. In hindsight the thing I was describing isn't true in general since the singular values of Y could all be smaller than those of X lol

[fleet sail]

ah so ur goal is to map the tensor into a larger 2d array

[wide spear]

Be careful about losing some structure

[fleet sail]

in NN there is no need for bias in 1st layer?

since its the input layer

[wide spear]

The first layer is still a layer?

[fleet sail]

yea i think so

thing is im trying to figure stuff out and dont want to get it wrong

[wide spear]

My boyfie and I see no reason there wouldn't be one

[fleet sail]

cool that doesnt add much complication

but since theres no weights

the first layer would just be a^(1)+b^(1)

unless theres weights?

but the weights would depend on previous layer but there is none

[wide spear]

?

Why wouldn't there be weights

Why do they need to depend on the previous layer

[fleet sail]

they don't depend on previous layers since there are none,

so how is weights relevant, like how would you apply the weights on the first layer

[wide spear]

To the input

[fleet sail]

oh i think my first layer refer to input layer

vectorized

[tall solar]

Yess

[prime kraken]

wdym "rank projection"

what you wrote there gives you a rank X_k + rank Y_k (assuming X_k and Y_k lin indep), but it really just stacks the two matrices in a larger one, not as diagonal blocks

it's just a big SVD (again assuming independence; otherwise, it isn't even an SVD)

for block diag, you'd have to do some projections onto the null spaces of the opposite blocks

[tall solar]

By rank projection I mean that when you use the truncated svd with rank=k it's a projection onto the set of matrices of rank-k matrices

But hmm this is interesting. Any idea where I can read about the latter of what you're talking about? Specifically the part about projections onto the null space of opposite blocks

[prime kraken]

i'm a lowly engineer, so my references will reflect that

https://kth.diva-portal.org/smash/get/diva2:1167908/FULLTEXT01.pdf

chapter 4 in that seems to explain it ok

4.2.1 and 4.2.2

[tall solar]

Don't put yourself down like that, this is good thanks

[prime kraken]

oh, i think in your case the projections can be done by just appending 0s to the singular vectors

[prime kraken]

this is unfortunately still wrong, but the fix should come from putting the U and V matrices in a block diagonal form as well

$\begin{bmatrix}X&0\0&Y\end{bmatrix}=\begin{bmatrix}U_1&0\0&U_2\end{bmatrix}\begin{bmatrix}\Sigma_1&0\0&\Sigma_2\end{bmatrix}\begin{bmatrix}V_1^T & 0\0&V_2^T\end{bmatrix}$

[pine jetty]

Edd

[prime kraken]

which you might recognize is useless 😛

it's just two SVDs in a trench coat

[fleet sail]

lol

edd u mind helping me on some basic ML things?

[prime kraken]

i can try

[fleet sail]

so im trying to figure out stuff

heres my summary of forward pass or whatever its called

if u see something wrong let me know lol

and this is the backward pass or backprop

wait i didnt finish writing this

[prime kraken]

first one looks ok

your gradient is wrong

wait lemme reread

[fleet sail]

ye i realized

like the second part 1/2f(x)^t f(x)

wait

[prime kraken]

gradient w.r.t. what?

[fleet sail]

(W,b)

which is just x

[prime kraken]

what is y

input data?

[fleet sail]

the output

or i mean

[prime kraken]

this notation is cursed

[fleet sail]

lol

[prime kraken]

the target

[fleet sail]

yea

[prime kraken]

and f(x) is its estimate

ok

[fleet sail]

i think i figured where im wrong

[prime kraken]

so - x^T y + total derivative(f(x))^Ty

[fleet sail]

(1/2)f^t(x)f(x)

[prime kraken]

yeap

[fleet sail]

nice thanks

now im just wondering a few things

u can probably figure out an explicit or recursive formula for each component of grad f?

and also is the method i outlined the way the basic NN works or no

[prime kraken]

recursive with chain rule

start at the last layer and work your way backwards

[fleet sail]

ic, is it possible to give like an example? say you solved the w's 2nd last layer, how do you get to the 3rd last layer

[prime kraken]

i'll link you to something

the important part is to be able to (sub)differentiate the activation function

https://brilliant.org/wiki/backpropagation/#:~:text=Backpropagation%2C short for "backward propagation,to the neural network's weights.

that and "automatic differentiation" are what you want

you would never do it by hand for something with more than 2 or 3 layers and just like 6 variables per layer lol

[fleet sail]

i see, thanks for the help!

also i just want to ask im not misunderstanding anything in the notes i wrote right?

for the most simple case of backprop with arbitrary number of neurons in each layer

[prime kraken]

looks ok to me

[fleet sail]

also i just wonder do u conventionally call the first layer the input layer

or is that the 0th layer, and im assuming the input layer has no weights and bias?

[prime kraken]

input layer sounds right

[brave crypt]

can maple return the general term of a succession given a few inputs?

idk if terminology is ok

[night urchin]

Sorry to take over with another question (I don't know Maple well enough to answer). So I'm wondering if anyone knows Posits (https://posithub.org/docs/Posits4.pdf for reference), and specifically knows about two properties: (1) is it true that if the bitwise representation of posits is strictly increasing within a given bit sign (i.e. positives are strictly increasing, as are negatives)? (2) is there an accepted algorithmic approach for adding Posits? Cause I discovered a weirdly cool property of Posits that allows me to "reconstruct them", but only sometimes it seems like. Any help / thoughts would be appreciated: I can go into more detail on (2) if anyone's curious, still working out the math

[balmy grotto]

so i have this function and points

and im being asked to find this

since n =3 we have the fourth derivative right

how would i go about finding the maximal value of that

[wide spear]

Can you compute the 4th derivative

[balmy grotto]

Yes

[wide spear]

Ok

Can you find the maximum of a function on an interval

[balmy grotto]

well yeah find critical pts and stuff

how would I go about the fast route through mathematica without having to build a silly sign chart

[wide spear]

Ok

So first D[1/(1+x^2),{x,4}] computes the 4th derivative

Or

f[x_]=D[1/(1+x^2),{x,4}]

Then you do

MaxValue[{f[x],-4<=x<=4},x]

And this spits out your answer

For future questions about using mathematica #computing-software

[hollow surge]

anyone know a good resource to learn matlab for linear algebra

[wide spear]

Does anyone have any pretty pde simulation videos lying around

I'm giving a brief talk about pdes to a bunch of incoming first years

And I think having some pretty animations would be nice

[fallow salmon]

Is anybody able to help explain stability analysis in a little more depth with me for a couple of differential equation methods as i'm real confused about them, so there is a differential equation, f''(t)+10f'(t) + 21f(t) = 0, and i was wondering how do i distinguish between whether the following are stable for it, so Explicit Euler, Implicit Euler, Heun Method, 2nd Order Milne, and RK4 Runge-Kutta Method

I may be in the wrong place im not 100% sure

[wide spear]

Do you know which type of stability you are considering?

[fallow salmon]

Just selecting which numerical methods are stable for the linear ODE given dt=1/3

[wide spear]

@quiet sparrow can help

[fallow salmon]

Thanks for getting somebody who can help!

[quiet sparrow]

I hope thats a joke

[fallow salmon]

what would be a joke?

[wide spear]

mniip I've actually forgotten stability analysis

You are the only hope now

[fallow salmon]

Lecturer has taught about 20 different numerical analysis for differential equations and has killed everyone off with it

[wide spear]

Sounds about right

[fallow salmon]

He expects us to be able to do a theoretical exam on it when 95% of the examples are coding examples

[quiet sparrow]

Im struggling in my own numerics class

[quiet sparrow]

You need to check whether h lambda for all lambda of your differential operator lies in the stability region of the method

[fallow salmon]

This is the part which is catching me out lol, I dont really understand the ordinary part of this course

the partial part is fine, ordinary on the other hand

[fleet sail]

can i continue to ask u abt this, i got confused at one point

nvm i got it

it is hard to code

at least from scratch

[prime kraken]

what was it

ah

yeah you probably shouldn't code it yourself

especially not automatic differentiation

regular backpropagation, you can do it yourself for small models

depending on how much sanity you have

[wide spear]

[glossy yoke]

Is anyone familiar with doing statistics on non-Euclidean data?

[wide spear]

What do you mean by non-Euclidean?

[glossy yoke]

@wide spear let’s say I have some points that I know are sampled from a 3D manifold. Can I compute the goodness of fit for the hypothesis that they were sampled from a torus?

Or maybe I have a distribution over a space X which carries a G-action. Can I do a hypothesis test for whether the distribution is G-invariant?

[wide spear]

Yeah these sound possible

You might need to rederive some formulas

For the second one you’ll hope that X has a norm or metric

[glossy yoke]

The context I’m interested in for the second is L^2(G) for a locally compact abelian group G so that’s a non-issue

There’s some moderately obvious things to do, like define a test statistic that measures the deviation of x_i from the mean value across the orbit of x_i under G (which will be zero everywhere iff the function is invariant to G)

But this seems like the kind of thing there’d be literature about, I would think

I’m having a lot of trouble finding things because Google likes using a very different meaning of the word “group” in the context of goodness of fit tests lol

And searches involving “symmetry” are bringing up symmetrical distributions over R^k

[wide spear]

For the first one, can't you embed your manifold in R^4

I don't know anything about manifolds

[glossy yoke]

@wide spear Yes, all manifolds can be embedded in R^k for some k. In fact, it can be embedded in R^3 (that’s what it means to be a 3D manifold!) but the problem is that such an embedding is not isometrically isomorphic.

It’s pretty easy to make two points arbitrarily close in R^k but far apart on a manifold.

[wide spear]

Sure which is why you embed it in some higher R^n

Does that work? No clue

[glossy yoke]

Nope 😛 if it were algebraic topology and differential geometry wouldn’t exist as fields

[wide spear]

Oh rip

[glossy yoke]

They’d be called “geometry” and “calculus” respectively

[dawn viper]

dont you need like R^7 or smt

idk anything so i shouldnt talk but isit possible to "fatten" up each point and compute like its simplicial homology or smt along those lines

[wide spear]

[wide spear]

[inland solstice]

Hi, I have a question considering second derivative of Newtons interpolation polynomial (with constant differences)

This is probably a better question for calculus section but nvm

How did they get that 1/h^2 in Pn''(x)?

[brave crypt]

why maple sucks so hard

can someone explain me this pls?

https://gyazo.com/8be3837170d21fbe17616a5079ce27f3

It was working before, and suddenly it gives me this error

like wtf

nvm, i closed maple and opened again and it works

what a shiit

[wide spear]

You can ask about maple in #computing-software

[brave crypt]

oh

[tall solar]

I don't think anyone actually uses maple. It seems like a ponzi scheme

[wide spear]

[pine jetty]

PandaMan-AMB

[grave spear]

It wont give me a solution

claiming there is no solution

[grave spear]

ignore this I got it

[fleet sail]

can someone help me with neural net math

[wide spear]

[fleet sail]

im looking for how to optimize a step in back prop

[wide spear]

What do you mean by optimize

[fleet sail]

its very computation heavy

[wide spear]

Right

[fleet sail]

like ill send u a summary of the method i use

[wide spear]

NNs are very computationally intensive

[fleet sail]

there was like no dependency on d

except for 1 part

so wonder if I have to iterate the same thing nonstop here

sorry for bad hand writing i might tex soon, but the relevant part is page 3 last bit

[fleet sail]

i see but i guess maybe im missing something i can do to shorten the last part

"possibility to shorten computation here" since only the $\delta^{(m-1)}$ differs per varying d

[pine jetty]

Anticipation

[fleet sail]

and the recursive bit is exactly the same

well at least the weights they use

[wide spear]

https://github.com/pytorch/pytorch

You can try to figure out how pytorch implements backprop

[fleet sail]

ah ok thx

btw the last layer

is the rectifying function no longer the sigmoid right?

or whatever that maps to (0,1)

or do you even need a rectifying function for last layer

[wide spear]

For the last layer?

[fleet sail]

yea

[wide spear]

I guess it depends on what sort of network you have

Like if you have a binary classifier

You won't want a sigmoid

[fleet sail]

ah that makes sense

[wide spear]

So it depends on the problem

[fleet sail]

but im also confused about the middle layers, like ReLU doesnt seem to make sense

since it doesnt map to (0,1)

but the neurons should take values btwn (0,1)?

[wide spear]

Why do the neurons need to take values between 0 and 1?

[fleet sail]

at least 3b1b said i think

ok then ig they dont?

[wide spear]

Yeah

A common thing, batch normalization, is incompatible with values between 0 and 1

Because it normalizes so that the mean is 0

And sd is 1

[fleet sail]

that makes sense then

thanks!

i guess i got 1 more question for now

so how do you deal with functions that r like binary classifiers as last layers, or those that are flat piecewise

assume derivative is 0 everywhere?

but that looks sus

[wide spear]

Hmmmm

Well if you have a binary classifier

[cursive surge]

For binary classifiers you usually output a probability between 0 and 1

[wide spear]

I guess you can return a probability that it is a thing

Yeah

[cursive surge]

generally you use softmax to force everything between 0 and 1, and also have the probabilities of each class sum up to 1

[fleet sail]

ah ok, makes sense

thanks!

so basically various form of output have different method of remedy, and perhaps they r heuristic?

[cursive surge]

Yeah you can use different functions in different situations.

[fleet sail]

i am struggling to find relevant code it is ok tho ill try look elsewhere

[wide spear]

[cosmic karma]

Does this mean that this method gives 3 roots at the same time? I'm confused on the multi-point iterative method.

Are x,y,z different roots of the function?

[cosmic karma]

so what does multi-point method mean?

because i found this paper that "solves for multiple roots of nonlinear equations"

Ohhh okay yeah they mentioned multiplicity a lot

got it thank you!

nice this is a package in C?

i will just try to find the algorithm and try translating it in Matlab

thanks for the resource

[fallow salmon]

when coding the numerical solution for Successive over relaxation for an altered version of Helmholtz' PDE, is there a good amouot iterations that i should expect? i got around ~~101 for a coefficient of w=1.5, on an Nx x Ny = 21 x 21 grid

is this a lot for the size and differential type?

[wide spear]

What error tolerance

[grave sand]

I have to implement a boundary value method, which is equivalent to solving a system such as
$
y0 = Y_0\
y_{n+1} - 2hCy_n - y_{n-1}=0\
y_N - hCy_N - y_{n-1}=0
$
where each $y_i \in \mathbb{R}^m$ for some m

[pine jetty]

Nicky

[grave sand]

C is a matrix.

Obviously, i could easily implement it by writing out the mn linear equation, but this leads to a massive but sparse system. I was wondering what the best way is to implement this in python. the only way i can see it happening right now is to solve this system, but i was wondering if theres a better method, given the 'nice' relation of the equations

[wide spear]

What do you know about the matrix

And what diff eq is it coming from

[grave sand]

its for the heat equation

so C is D_2c

[wide spear]

Try SuperLU

Included in scipy

You precompute a sparse LU decomposition of C

Then use it whenever you need to solve

You can also try conjugate gradient which might be faster, depending on the condition number

[grave sand]

ok let me have a look

speed isnt really an issue

its more how to implement it

[wide spear]

If it’s in python all of these will be in scipy

So you don’t need to implement anything

[grave sand]

what does the LU decomposition give me exactly?

[wide spear]

Read the documentation

https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.SuperLU.html

[grave sand]

i see but how does that solve my issue?

[wide spear]

You were wondering how to solve your linear system right?

[grave sand]

hm, well if implement the mn matrix, its no problem

ok let me reformulate the question

so i have N linear equations

but each equation is m dimensional

is there a way to implement this without 'expanding' everything and creating the Nm dimensional matrix and then solving it

like, is there a module which i can enter a matrix with higher dimensional entries to solve the system

[wide spear]

You matrix will be a band matrix right

[grave sand]

yes

[wide spear]

https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.diags.html#scipy.sparse.diags

So you construct your matrix in a sparse manner by just declaring the bands

[grave sand]

okay thats great actually

thanks!

[fallow salmon]

as i was saying

[wide spear]

Ok so you have $u_{xx}+u_{yy}=f(x,y)$

[pine jetty]

主要场景进行

[wide spear]

Have you discretized it

[fallow salmon]

badly most likely but yes

[wide spear]

How did you discretize it

[fallow salmon]

Central FD for the 2 partials and then left f as f_i,j

[wide spear]

Ok

You get a matrix from this

[fallow salmon]

ya in the form L+D+U ?

[wide spear]

Ok

How does SOR work

[fallow salmon]

works similar to GS by shortening no of linear eqns?

[wide spear]

Yes it is similar to Gauss Seidel

Wikipedia has pseudo-code

[fallow salmon]

Ive implemented a little bit of code for solving the diffeq

but im not sure about it at all

mainly bc it took 101 iterations instead of around 75 as expected

[wide spear]

Oh

101 iterations instead of 75

Hmmmm

Are you using the optimal relaxation parameter

[fallow salmon]

we was advised to estimate over optimise

this was what i implemented thus far

[wide spear]

a is the grid spacing?

What is 4+a^2?

Shouldn't it be 4*a^2?

[fallow salmon]

how so?

[wide spear]

What does 4+a^2 represent

[fallow salmon]

a rearrangement of the discretisation

[wide spear]

Can you elaborate

[fallow salmon]

each CFD leaves -2f_[i,j]

but has a a^-2 multiplied by each

so -4a^(-2)f_[i,j]

then the f_[i,j] has left too

so factorise it

[wide spear]

Ok so the centered difference you have is $f_{i-1,j}-2f_{i,j}+f_{i+1,j}$ and $f_{i,j-1}-2f_{i,j}+f_{i+1,j}$ and all of these are multiplied by $h^{-2}$ right

[pine jetty]

主要场景进行

[fallow salmon]

ya

[wide spear]

So you factor out 1/h^2?

[fallow salmon]

essentially

[wide spear]

Ok

So how does that become 1/(4+a^2)?

[fallow salmon]

then f[i,j] = 1/(4+h^2)*(...)

[wide spear]

Are you using the right norm

[fallow salmon]

in what sense

ima be honest, im not 100% sure i even know what im doing

[wide spear]

Ummmm

[fallow salmon]

its extremely new content to me

[wide spear]

Is the number of iterations important

Tbh 101 iterations versus 75 is not that big of a deal

[fallow salmon]

not really no, but im not sure it even yields a correct solution

[wide spear]

Oh

Ok

What if you use someone else's SOR code

[fallow salmon]

hm?

[wide spear]

https://stackoverflow.com/questions/53251299/successive-over-relaxation

Tbh this sounds like a code debugging issue

Which I am very bad at

[fallow salmon]

I dont even understand half of that stack lol

[wide spear]

Have you checked that all the signs are correct

[fallow salmon]

i believe so

[neon snow]

Yo

I have an ellipse in 2D space

And I want to find the semi minor and major axes

I did:

S = [X',Y'] % Nx2 matrix
c = cov(S) % covariance matrix
[V, D] = eig(c) %V-columns are eigenvectors with corresponding diagonal entries of eigenvalues in D

And I get

Directions are correct, but how do I get the length of semi and minor and major axis?

Those eigenvalues don't say much

Or maybe I don't know how to interpret them

[tall solar]

I'm looking for a tight matrix $A\in R^{m \times n}$ norm inequality.

I know between the nuclear norm and frobenius norm $||A||_* \leq min(m,n) ||A||_F$
And between frobenius norm and Manhattan norm $||A||_F \leq ||A||_1$

Anyone know if a matrix norm inequality tighter than $||A||_* \leq min(m,n) ||A||_1$?

[pine jetty]

fajitas

[brave crypt]

I think identity matrix achieves the equality right?

[fleet sail]

Is there conjugate-gradient like methods for arbitrary functions/convex functions? that converges in a fixed iteration and does optimal work in each iterations

[brave crypt]

That sounds too powerful

My guess is that you could construct a counterexample

[bright palm]

is this an okay place to ask about finite differences method?

I'm curious about the optimal h explanation

I have found some explanations on google but they use much much more technical language than I'm equipped to deal with

$error(h) = \left|f'(x_0) - \frac{\bar{f}(x_0+h) - \bar{f}(x_0-h)}{2h}\right| = \left|f'(x_0) - \frac{f(x_0+h) - f(x_0-h)}{2h} + \frac{\delta(x_0+h) - \delta(x_0-h)}{2h}\right| \le \left|f'(x_0) - \frac{f(x_0+h) - f(x_0-h)}{2h}\right| + \left|\frac{\delta(x_0+h) - \delta(x_0-h)}{2h}\right| \le \frac{h^2M_3}{12} + \frac{2\epsilon}{2h} = \frac{h^2M_3}{12} + \frac{\epsilon}{h} := g(h)$

[pine jetty]

jan Niku

[bright palm]

so we assume that both the function itself and the derivative are bounded by M and epsilon

[bright palm]

<@&286206848099549185>

[wide spear]

Yeah

CG for arbitrary functions is too OP

So if we specifically consider linear systems

Any Krylov subspace method will converge in a fixed number of iterations

For A a n by n matrix and b in R^n then after n iterations the Krylov subspace will include all of R^n and then it will give the correct value

However, CG converges very quickly, much faster than something like gradient descent

However, the solution is not exact until the Krylov subspace spans all of R^n

[brave crypt]

"Optimal direction and optimal distance algorithm" sounds like it would have a name though

[wide spear]

However, because of floating point error, you don't need all n iterations

Even GD doesn't do optimal direction/distance

[brave crypt]

I just meant optimal direction as in along the gradient (locally optimal). I guess you could call it "gradient descent with exact line search" (I was hoping for a shorter name though)

[wide spear]

Gradient descent is locally optimal

[fleet sail]

yea

ic

[tall solar]

I've heard about people doing crazy stuff like adding a threshold t to the singular values of the hessian to make each singular value positive. Can't say if this is non sense or not

[fleet sail]

yea not sure how that works lol

[wide spear]

There's a lot of modern optimization theory

It's all very complicated

And technical

[brave crypt]

Hello

[wide spear]

Hello

[brave crypt]

I am trying to infer the formula of dilation to understand its role in CNN.

[wide spear]

Well, do you know what dilation does

[brave crypt]

Yes, it capture higher information.

And we don't have to sacrifice resolution.

[wide spear]

Uh sure

Were you given this formula

[brave crypt]

I found it from this blog: https://www.inference.vc/dilated-convolutions-and-kronecker-factorisation/

[wide spear]

Ok sure

And do you understand the formula

The notes you linked seem quite comprehensive

[brave crypt]

I am sorry, I don't understand.

[wide spear]

O

Ok

Do you know what k and f are

[brave crypt]

Nope. I am sorry.

[wide spear]

Read through this: https://towardsdatascience.com/understanding-2d-dilated-convolution-operation-with-examples-in-numpy-and-tensorflow-with-d376b3972b25

[brave crypt]

Okay 👍

Thank you very much @wide spear 😀

[wide spear]

[fleet sail]

can someone take a look at this slides, so they broke proving power method for symmetric matrices in 2 stages, 1. show if converge then converge to eigenpair, and 2. to show it converges

I feel like the first part proving that if power method converges, then it converges to eigenpair, is redundant?

cuz I think the second part seems to proved that it converges to the eigenpair already

(unless I didn't figure out that it used first part implicitly)

[brave crypt]

Could someone help me implementing the algorithm in this pdf? https://www.researchgate.net/publication/264422777_Image_Reconstruction_from_Legendre_Moments You can scroll a bit down to see it, not much to read

[static sail]

Hello everyone, I am new here. I am struggling with an assignment and was wondering if anyone has ever used chebyshev collocation in Python to solve Ordinary Differential Equations. Any help by chance?

[wide spear]

If it's a coding question, then #computing-software or the CS server or the Python server

Is it a theory question?

[static sail]

Understand the theory, having trouble implementing it to code

Thanks again

[wide spear]

The python server is quite extensive

[sage dew]

I don't understand what are they expecting as an answer?

[flint rover]

Hi is anyone familiar with 2 phase flow in 1D and how to implement the solution? This paper gives the solution but I'm having trouble grasping how to implement the solution, S(x,t).

specifically, I'm having trouble evaluating the integral in 3.10

[wide spear]

Iceberry do you know what it means for something to be ill-conditioned with respect to perturbation?

Aethos do you know what delta t is, because I don't

[flint rover]

delta is just a constant I believe

[wide spear]

Ok

Beta and gamma are constants as well?

[flint rover]

yes

[wide spear]

Ok so you have your function S(xi,t) which you integrate with respect to xi from 0 to x

[flint rover]

yes but S(xi,t) is dependent on xbar which is the part I find confusing

[wide spear]

How so?

[flint rover]

I cannot evaluate xbar because I end up with an expression like
xbar = some function of xbar

[wide spear]

What is the expression for S(xi,t)?

[flint rover]

it is equation 3.13 with x substituted for xi

[wide spear]

So wouldn't it be partial xi bar partial xi?

[flint rover]

I believe it would be partial x bar partial xi

[wide spear]

x bar is a function of x, no?

[flint rover]

yes but I dont think substitution of x for xi applies to xbar

[wide spear]

But it doesn't make sense to write x bar as a function of xi I don't think

Anyways

Integrating 3.10 using the expression given in 3.13 is circular

Because they're the same thing, just rewritten

[flint rover]

so how would you go about implementing S(x,t) ?

because partial xbar partial x requires me to first evaluate xbar wouldnt it?

[wide spear]

You don't have another expression for S(xi,t)?

[flint rover]

I don't think I do, all the expressions are in the screenshot I sent

[wide spear]

And what are you trying to implement?

[flint rover]

trying to plot S(x,t)

which is the saturation of a fluid at (x,t)

[wide spear]

Ok

So you have 3.13

And what remains is to find x-bar(x,t)

[flint rover]

yes

[wide spear]

You know that $\phi(\bar{x},t)=e^{ax}$

[pine jetty]

鬼怪

[wide spear]

And you have the expression for phi(x-bar,t)

Given in 3.12

So given x and t, you can solve for x-bar

It'll be a bit messy so you'll need to do it numerically

[flint rover]

how would I do that numerically? That may be a bit out of my skillset

[wide spear]

So you have given x and t

And you want to find x-bar

Have you heard of newton's algorithm?

[flint rover]

I have not

[wide spear]

I see

https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fsolve.html

You can use this if you're working in python

[flint rover]

wouldnt I need to evaluate xbar to get an initial guess?

[wide spear]

No

You can set your initial guess to be 0 or something

[flint rover]

hmm okay, so I would do something like
fsolve( phi(xbar ,t) - e^ax, 0)

[wide spear]

Modulo some coding details, yes

[flint rover]

okay, I'll give it a go

Thanks so much for the help

[wide spear]

[hushed lark]

Anyone good at Macaulay2?

Im trying to figure out how to make a generic upper triangular Matrix

[wide spear]

#computing-software probably

[hushed lark]

Ty

[brave crypt]

Hey mate

I understood what deep dilation does.

But I am still unable to grasp this formula.

Would you please, please help me?

[wide spear]

Give me 4 hours

[brave crypt]

Sure

Thank you 😊

[wide spear]

Ok

I'm sorry it's been 5 and a half hours

Ummmm

I think this is a bad dilation convolution formula

https://towardsdatascience.com/review-dilated-convolution-semantic-segmentation-9d5a5bd768f5

See section 1 of this

The formula you've posted comes up in 1d signal processing

And is closely related to a dilated convolution in CNNs

But conventions are different

So it's weird

[wide spear]

@balmy grotto you should read LeVeque's Numerical Methods for Conservation Laws

I think it will answer your questions

[balmy grotto]

How do I show a finite difference scheme is conservative and consistent

[wide spear]

Do you know what it means for a scheme to be consistent

[balmy grotto]

no

I am tryingto read that book yes

[wide spear]

See sections 12.1 and 12.2

[brave crypt]

@wide spear Thank you for your help.

I know I should search this on internet.

But would you please help me to know what F, l and K is here.

[wide spear]

l is the dilation factor

F is your input

k is the convolution filter

Unless it's the other way around

If might be F filter and k input

[brave crypt]

L is the dilation factor.

[wide spear]

Yes

[brave crypt]

Thanks.

😀

And would you please tell me what is P here?

[wide spear]

p is the p-th entry of the output

[brave crypt]

Got it.

So, It's basically applying different convolutions on the input and summing up.

[brave crypt]

@wide spear

[wide spear]

?

Yes

[brave crypt]

Great

One last question what is s and t here?

[wide spear]

s is an index of the filter and t is an index of the input

[brave crypt]

Great.

Thank you very much for your help.

😀

[neon snow]

I guess this is the correct place to ask

[pine jetty]

Victor H

[neon snow]

I have this recursion and I want to find the time complexity of it; I'm suspecting exponential

So what I've thought of so far is: it's linear time complexity to find the max of n numbers

However, every call to p(n) requires me to calculate p(n-i)

How would I set up the recurrence formula?

[
T(n)=n*\sum_{i=1}^{n}T(n-i)
]

[pine jetty]

Victor H

[neon snow]

Like this?

[distant sky]

for part b) when they say p_0 when x = 0 does it mean at the origin point (0,0,0) so i can let x,y,z = 0

[wide spear]

redd yes

Victor wouldn't that be n^2?

You don't need to recompute p(n-i) every time

You can store the values

So you have n steps

And at the i-th step you have i work

So it's like n^2/2

[neon snow]

Yes but that's not what I'm trying to do

That I will do later

[wide spear]

Ok so you want to recompute p(n-i) everytime?

[neon snow]

Yeah, it's an assignment about dynamic programming

So first do it without

then with

[wide spear]

Ah ok

[neon snow]

Is my recurrence formula correct?

[wide spear]

The sum should be from 1 to n-1

Wait no

[neon snow]

Ah yes!

Thanks

[wide spear]

Ok you do it in reverse

[neon snow]

No wait

[wide spear]

Wait 1 to n is fine

[neon snow]

Yeah 😄

[wide spear]

Wait why do you multiply by n

[neon snow]

Don't I need to find the max

[wide spear]

Wouldn't that be adding n work

[neon snow]

Yes you're so correct

[wide spear]

Ok so it would be n+sum

[pine jetty]

Victor H

[wide spear]

That looks fine I think

[neon snow]

Can I apply Master's theorem?

[wide spear]

No I don't think so

[neon snow]

Yeah I figured I couldn't

hmm

Any idea on how to solve this?

[wide spear]

Mathematica doesn't present a solution

Rip

[neon snow]

Yep

My wolfram isn't understanding me

,w T(n) = n + \sum_{i=1}^{n}T(n-1)

[pine jetty]

Results provided by WolframAlpha

Click here to refine your query online
Upgrade to WolframAlpha Pro!

Requested by Victor H#9602

[neon snow]

big sad

Thinks its T*(n-1)

[wide spear]

Lol rip

[neon snow]

The biggest of the rippest

[wide spear]

https://en.wikipedia.org/wiki/Akra–Bazzi_method

This should work

[neon snow]

What type of magic is this

Let me check it out

[neon snow]

I don't think I can use this

[wide spear]

Rip

[neon snow]

b_i<1

for me it would be

b_i=1

[neon snow]

Master's applies easily now! 🙂

[wide spear]

Oh nice

[summer bridge]

I'm looking for an MCMC algorithm with the following properties

It needs to sample 2D distribution, and I would rather not have rejects

I.e. I'm looking to sample a distribution and I need the points to not be repeated

and if they are repeated, I can discard them without statistical impact

[balmy grotto]

I read the sections over and over and i'm suffering huge from imposter syndrome

[wide spear]

Oh no

Did you read the first 11 chapters as well

[balmy grotto]

no

[wide spear]

Oh

[balmy grotto]

the excercise is supposed to be self contained

[wide spear]

That might be why

KurtDee what about Gibbs sampling

[summer bridge]

I thought about that, but I don't think I can take conditional samples

And I was thinking about MH for each, but it would have to be for each condition right

The 2D space is dependent

[wide spear]

MH also has rejects, which you said you were trying to avoid?

[summer bridge]

So let me describe the problem

[wide spear]

This is very much not my area of expertise

[summer bridge]

Er okay, I'm basically seaching for something based on distribution of previous knowns

in a 2D space

so MH repeats would repeat the search, which is useless

and if I discard the rejects it impacts the statistics of it

Hmmmm actually

Yeah I thought about it and probably the gibbs sampler will work

data -> histogram -> spline -> normalized

can get dependent distributions

that should be sufficient, correct?

[wide spear]

Yes that seems sensible

[summer bridge]

I'll give it a shot

[summer bridge]

Hmmmmm so I'm generating my histogram

what's the best way to determine the width of it

Variable width maybe?

Yeah I like that idea

[wide spear]

[summer bridge]

Actually no that's terrible

[wide spear]

[summer bridge]

I was thinking like a 4d gradient threshold

but that makes no sense

DON'T FUCKING LAUGH AT ME

AAAAAAAAAAAAAAAA

[wide spear]

Ok

[summer bridge]

is bounding $(\frac{\partial ^4 }{\partial x^4} + \frac{\partial ^4 }{\partial y^4} )f$ proper?

[pine jetty]

KurtDee

[summer bridge]

I just want to make sure there are no gaps

[wide spear]

What do you mean by proper

[summer bridge]

Let me draw my consideration

[wide spear]

I see

[summer bridge]

there will probably be gaps in my data, so I need to choose a stepsize that's appropriate

[wide spear]

Sure

[summer bridge]

I'm thinking the 4th derivative should be a good benchmark, but the form I have is like uhh

I dont even know how to describe it

it's not $\nabla \cdot \nabla ^3 f$

[pine jetty]

KurtDee

[wide spear]

Wait why do you need to specify a stepsize?

Like if you're using matplotlib

I think it can pick a good bin size

[summer bridge]

So I have this data that I'm going to histogram, and with the histogram I'm going to spline it to approximate the distribution

[wide spear]

Oh I see

[summer bridge]

Oh fuck I'll check it out

but the distribution is 2d?

[wide spear]

matplotlib.pyplot.hist2d

[summer bridge]

I wanted to describe it mathematically, but perhaps getting the solution first is more important

Yeah it describes bins as a default param

[wide spear]

Yeah I misread that

Anyways

[summer bridge]

Yeah there seems to be a bunch of rules

I'm reading it now

[wide spear]

Wait if you have 2d data

And want to recover the surface

Why don't you just put a cone

At each point where you have a point

[summer bridge]

Like gauss map it?

[wide spear]

Yeah

It doesn't necessarily need to be a Gaussian distribution

[summer bridge]

hmmmm yeah that would give me a distribution

[wide spear]

It can even be like

A cone

With radius 1

[summer bridge]

hmmmmmmm'

[wide spear]

But I guess you want smooth

So yeah

At each point you put a gaussian with sd 1

[summer bridge]

I mean if the radius is big enough

[wide spear]

And then at the end you normalize it

[summer bridge]

it will be sufficiently smooth

I thought about that before but yeah maybe

it would make normalizing easier

[wide spear]

Like

Your histogram -> spline idea isn't bad

But I have concerns about how sensitive it is to small perturbations

[summer bridge]

True

[wide spear]

Like

You don't want a slightly different bin size to result in a very different interpolated function

[summer bridge]

Ughhhhhh fuck

Aight think of this

I get the gauss/cone map

Log scale that shit

adjust radius as needed

[limber rampart]

Hi are there any economists here?

need some help with a question

[wide spear]

[limber rampart]

What does perfect competition do to the mpk and mpl?
Also how do i compute (d)

[wide spear]

Is this a test

[limber rampart]

past year

[wide spear]

Ok

[limber rampart]

but no answers

:/

[wide spear]

@mortal dagger

Rip I don't remember any macro

[limber rampart]

I know A_t represents the total factor productivity but cant seem to get my head round how to get it

[mortal dagger]

Waaaat

[limber rampart]

Help me pls @mortal dagger

T-T

[mortal dagger]

I don't know this stuff

I do finance, not economics

[wide spear]

Oh rip

[limber rampart]

RIP

<@&286206848099549185> Anyone?

[brave crypt]

how can i know if an image has been "painted"?

like, if it has a blue-ish color overlapping it or something

idk how to explain sorry

better with an example

this one has blue overall

while this one doesnt

Like monochrome? My guess is that it lies some linear subspace of RGB

For example, grayscale is the subspace t(1,1,1)

(R=G=B)

it is not monochrome

Oh hm

on the blue one, if u add the oppsosite color of blue on the color wheel (i think is orange-ish), the pic will get same colors as the one below, let me photoshop

https://gyazo.com/ab694b28744683884ea7abbb59028075

What do you mean by "add a color"?

i calculated the average color of the picture, i inverted it, and i...

overlay? idk the name of the fussion mode in english sorry

blending* modes

i think it is called lighten

I guess I mean more like a mathematical formula for "adding a color"

no, is overly

i dont want a formula to apply this

i wanna know how can i know if a photo has a color overlaying

Like the first one, it has blue

But if you don't have a formula for how you got that blue picture, I don't know what is the relationship of the original to the blue one

that i wanna remove the blue color

but i need to know how can i know if a pic have this, like a main color

when it shouldnt

[wide spear]

What you're interested in is white balance

I think

[brave crypt]

mmmm

is it?

imagine i have a lemon

and i paint it red

cant i remove the red excess?

[wide spear]

No

How does a computer know that it’s yellow?

[brave crypt]

idk 🙂

thats what i am wondering

[wide spear]

It sounds like you might be interested in digital image processing

[prime kraken]

you have 2 options here. one of them is to do it rather arbitrarily at the behest of a user (e.g. you), who says they want to do something specific to an image

another is to do it in a data driven fashion, since there is no good model for this. you need to check reference images to discover what happened to the coloring/lighting and undo it

this is a pretty challenging inverse problem to solve "optimally"

[brave crypt]

i wanted to make this happen on an script :/

So i cant manually say "this yes" or "this no"

[prime kraken]

this is a so-called "inverse problem". not all of them can be solved, for starters. and you need 2 out of 3 of (input, transformation, output) in order to be able to do something

only very special problems can be inverted from just the output, and you still need "prior knowledge" of at least some properties of the input and the transformation

[brave crypt]

sad q.q

[wide spear]

[brave crypt]

i guess i cant then 😦

this is like going from black and white to color right?

[wide spear]

Yes

Which is very difficult

Because BW fundamentally has less information

[brave crypt]

but a blue-ish image has same amount of channels than the original :/

[wide spear]

Sure

But you can't tell if that's how it was taken

Or if it was edited

Consider a car

Each model of car comes in multiple colors

If you are given an image of a car, how can you determine if it was edited?

[brave crypt]

but i can see if the picture overall has been edited

[wide spear]

How

If I give you a blurry image, how do you determine if it was blurry when it was taken or if it was blurred after with a filter?

[brave crypt]

https://gyazo.com/2fbdc2da4821a19c674aedddca2a27d7

look at this one

do u think it was original or it has some blue added artificially?

[wide spear]

No clue

[brave crypt]

mmm

i see it clearly with some extra blue

[wide spear]

How do you know how much extra blue there is?

[brave crypt]

well, i dont know how much extra blue does it have

[wide spear]

If you don't know how do you want a computer to know

[brave crypt]

i mean, it can do the average color of the image and then overlay the opposite color :/

[wide spear]

Why is this the correct thing to do?

The image is very blue/green

So the average will be blue green

And then if you subtract blue green

[brave crypt]

if i had the answers i wouldnt be asking 🙂

[wide spear]

Then you won't have much color left

You're trying to automate something that can't be at this point in time

[brave crypt]

mhmmmm

so

there is no way i can turn this

[wide spear]

I mean

[brave crypt]

into this

[wide spear]

You can with human intervention

[wide spear]

You pick a point that you know should be white

[brave crypt]

but i want it on a script

[wide spear]

And then subtract that color

Right

You can't automate the process

[brave crypt]

and how do i know what should be white?

q.q

[wide spear]

Well, you know that the wing should be white

[brave crypt]

well, i can pick the most white pixel on the purple one

and assume it is white

mmm

actually

[wide spear]

You don't necessarily want the most white pixel

[brave crypt]

actually, im thinking

i can turn purple image to gray, then pick the 128 gray value, and see what color does that pixel have on the purple image, and that tone of purple is the one that has been used to paint the image?

cuz maybe if i know what purple color has been used, and what should be white, i can undo the paint

can i?

[prime kraken]

there is no optimality in what you're proposing

it's very much the same as the first approach i listed

someone (you) makes stuff up and applies it to the image

[brave crypt]

hdsbfla<hdf

but this is not what i want

[prime kraken]

it's already what you're doing tho

[brave crypt]

no

i am trying to find a pattern that can be automated so the computer does itself when ever it is needed

first i need to detect if an image has been painted or not

[prime kraken]

this pattern has to be learned from somewhere

[brave crypt]

and then, undo the paint

[prime kraken]

you would need like a physical simulation of the whole sensor setup

i am under the impression you are not ready to understand why this problem cannot be done easily

[brave crypt]

xd

[prime kraken]

you should read about inverse problems

[brave crypt]

i mean, i dont care if it is easy or not, coding is not the problem. I wanna know if it is possible, and if so, how

[prime kraken]

the easiest way out of this one is machine learning with hundreds of thousands of example images

[brave crypt]

dammit i wanted to avoid this

xD

cuz i am doing this cuz it will be the input of another NN

x)

[wide spear]

[prime kraken]

but what dantalion has been saying cannot be avoided. there are multiple inputs that will give the same output, because in some bases coloring an image in blue is something you want to undo, but in others, this blue hue is the actual ground truth

[brave crypt]

i mean, i could for the first pokemon generations

[prime kraken]

this is a really bad problem

[wide spear]

If you know what it's supposed to be, fine

[brave crypt]

but as long as u get further, there are less pokemon images

[prime kraken]

you are still misunderstanding what the problem is

but it seems this is a case in which you know input AND output

i.e. 2 of the big 3 things (input, transformation, output)

[wide spear]

Then it's a matter of pattern matching

[prime kraken]

and then MAYBE it can be done

[brave crypt]

no no

i dont know the input ofc

[prime kraken]

then you won't be able to solve it uniquely

xD

[brave crypt]

my principal goal is to guess that is this pokemon

on a nn

but with this colors it thinks it is a finneon

( a fish )

so i need to give it the correct colors

but since idk the pokemon, i cant restore it 🙂

[prime kraken]

i would suggest you do it in black and white, then

because you won't be able to recover the color

[brave crypt]

ive though, but the model is already pretrained (the data set used to train it is no longer available) and i couldnt manage to find a big image dataset of each pokemon, so i couldnt train it on my own

[wide spear]

You know the trick about applying transformations to images to get more data right?

Like

You can rotate each image

And then treat those as separate data points

[brave crypt]

yes, but still

5 images per pokemon arent still enough 🙂

[wide spear]

You can rotate each image 360 times....

[brave crypt]

for pikachu i have 100 while for meltan i have like 8 (?)

so i also have inbalanced data set

😄

thats why i went for an already trained model

[wide spear]

I assure you there are more than 8 images of meltan online

[brave crypt]

but the data set used on that model is gone from kaggle 🙂

[brave crypt]

sure but i am not gonna google image for each pokemon and download 100 manually

xD

[prime kraken]

you can additionally take each image and each of its rotations and input it after modifying its colors

this will augment the data set immensely and teach the network that color doesn't matter

[brave crypt]

i mean, i would !!!!!

but i dont have the data set

asjdfañsdf

[prime kraken]

you have to go and make it, then

you expect this to just work magically?

[brave crypt]

no, i wanted to remove the purple 🙂

[prime kraken]

well, good luck

the sad part is, you could actually remove the purple if you already knew what the pokemon is

so this is something that could be done at the output of the network that classifies lol

but not before

not enough info for that

[brave crypt]

^

[prime kraken]

would you understand better if i explained it to you in spanish?

[brave crypt]

no

u are saying i could paint it correctly if i knew what pokemon it is. And i am telling u the NN needs it to be correctly painted in order to classify it. So it doesnt help

[prime kraken]

that's only the last part. i mean, to explain to you why it is that you won't be able to just blindly correct the color

well, dantalion already explained that as well

so all i can say is good luck

[wide spear]

Like

You're trying to do something

And if you could do it accurately and efficiently

[brave crypt]

i mean, i know why i cant go from BW to color

[wide spear]

Then you would easily put adobe out of business

[brave crypt]

cuz RGB = (100,0,0) -> BW = 33 is the same as RGB = (0,100,0)

but in this purple thing, i still have the 3 channels, so i was wondering if i can somehow bring to life the RGB values hidden by the extra amount of purple 😦

[prime kraken]

the short answer is no

🔵 🔴

it's like me asking you right now which of those two circles is the original

just like that, with no other context

[brave crypt]

asdasda but on my purple image u see different pixels with different brigthness

[prime kraken]

doesn't matter

[brave crypt]

i can choose the closest one to white, and assume that one is white, like dant said

[wide spear]

That is very much not what I said

[brave crypt]

o.O

[prime kraken]

lol

[brave crypt]

u said i need a white pixels and substract it (?)

[wide spear]

Yes

I said you need to manually pick a pixel you know to be white

[brave crypt]

.

[prime kraken]

but you need to know it was white BEFORE it was colored in

[wide spear]

This is not necessarily the pixel that is closest to white

[brave crypt]

yes, thats what i say

[prime kraken]

in other words. the network needs to have already classified the image to know what it contains

[brave crypt]

i pink, on the purple image, the wing as white color

[prime kraken]

this can be done BACKWARDS

but not in this direction

it cannot be done

[brave crypt]

and say that one was white before the coloring