🔒 How do I efficiently filter/sort through vectors?

[prisma kiln]

Hello all,
I'm having difficult writing code for a project in python where I need to filter through a bunch of vectors e.g. <x,y,z> by satisfying an equation.
I have a function of three variables (s, theta, phi), where the variables are just equally spaced numbers (np.linspace). This function just multiplies a bunch of matrices together and outputs more matrices that represent 3D coordinates as vectors. Given an equation which I have included a picture of, the output vectors/points need to satisfy this equation. How can I go about doing this efficiently? I have millions of these points since my s,theta, phi parameters are very finely spaced apart.

[bold hatch]

@prisma kiln

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[cedar glacier]

def function(x: int, y: int, z: int) -> int:
    return x + y + z

satisfiers = list(filter(function, vectors))```

The only way to do this more efficiently would be to write a C or Cython function and expose it to python - but I'm not sure how much more efficient that would be, because there's a conversion step

If your predicate is multi step or involves particularly complex operations, it might be worth it

There's also numpy

[prisma kiln]

does the -> int mean that outputs should equal an int(e.g. 0 in this case)?

[cedar glacier]

Yes - the arrow followed by a type indicates the return type

If your predicate (the function you're using) is just math operations then numpy is definitely going to be the fastest

[prisma kiln]

ok

[cedar glacier]

With numpy you feed the entire table of vectors into the computer at once, and use the GPU to compute them all simultaneously

[prisma kiln]

i believe numpy has np.gradient which calculates derivatives for all variables

[cedar glacier]

Oh, is that your operation?

[prisma kiln]

yes, partial derivative with respect to s variables crossed (cross product) with partial derivative with respect to theta. and then the entire thing is dot product with partial derivative with respect to phi.

[cedar glacier]

https://tenor.com/view/james-corden-what-jwu-confused-what-happened-gif-8057539

Me, realizing the last few years was just a dream, and I'm still studying for a calculus exam in the morning

I enjoyed calculus, and I'm glad I took it. But I'm also glad its over

[prisma kiln]

i think ill just give it a try with numpy and see what happens. numpy already has np.cross and np.dot so im just gonna see how fast it does this.

[cedar glacier]

If you can do it in numpy, do so. That'll be on the GPU and enormously faster. If you can't, try Cython. For several purely mathematical operations, it'll be faster than just pure pythom. You can look into using a pool of processes to perform multiple operations at once

There's probably some extreme hacks you can use if performance is critical, but if you're just looking a sane level of speed, the above is good enough

[prisma kiln]

i did not know numpy ran natively on gpu

[cedar glacier]

Shit, I lied

I was thinking of torch

That's another option, though

[prisma kiln]

yea i tried using that once and just gave up lol

if numpy works ill leave it at that, if not ill try gpu computation

[cedar glacier]

🙂

Good luck and god speed

[prisma kiln]

thanks for the help

[cedar glacier]

I just looked it up by the way

Assuming a matrix of millions of purely numeric vectors, and a predicate composed of multiple purely mathematical operations (including cross and dot), torch would be the preferred method

[prisma kiln]

thanks, will be sure to try that

[bold hatch]

Python help channel closed for inactivity

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.