🔒 Optimize Numpy Matrix Roll

[azure talon]

    self.F=1+0.01*np.random.random((6,res,res,5)).astype(self.precision)])
    self.cx  =     [  0,    1,   0,   -1,    0]
    self.cy  =     [  0,    0,  -1,    0,    1]
    [...]
    for a in range(6):
      for i in range(0,5):
        self.newF[a,:,:,i]=np.roll(self.F[a,:,:,i],(self.cy[i],self.cx[i]),axis=(0,1))

I have the following code, and this np.roll is really slow. Can anyone recommend a faster way?

[plain tangle]

@azure talon

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[azure talon]

To explain the code: It takes shifts a part of the matrix either up, down, left, or right by 1 step

[lime cove]

I'm no expert, but are you sure roll is what you're looking for? It wraps entries from one line onto the next line:py x2 array([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]) np.roll(x2, 1) array([[9, 0, 1, 2, 3], [4, 5, 6, 7, 8]])
Notice how the 4 from the first line ends up on the start of the second line?

I'm not sure about this, especially with the different line wrap around, but couldn't you do all a's at the same time with py self.newF[:,:,:,i]=np.roll(self.F[:,:,:,i],(0, self.cy[i],self.cx[i]),axis=(0,1))

[azure talon]

Yeah but I over-write the rolled value later

[azure talon]

Technically yes, but it doesn't save any time

Also you'd need to change the axis

[lime cove]

Really? That really surprises me. The #1 most important thing about optimizing numpy code in my experience is avoiding pure python code (especially for loops) and doing as much as possible using numpy methods.

[azure talon]

Yeah the roll operation is way slower than the loops

[lime cove]

What about shifting using slices? py np.concatenate([np.array([False]*shift_by),np.equal(arr[shift_by:],arr[:-shift_by])])

[azure talon]

I don't get it

[lime cove]

This is a way to shift a regular list, the above was just a numpy equivalent:a = [1,2,3,4] shifted = a[-1:] + a[-1:]

[azure talon]

I'm going to try views maybe

I feel like there must be a simple way to do it

[lime cove]

Hmm,

got that out of a stackoverflow thread... now that I re-read it, I'm not really sure what they're doing with the equal stuff either

[azure talon]

I have the following idea:
I could make my matrix 2 larger, and then just copy everything within the right window, and ignore everything that overflows 😉

[lime cove]

You could also try an index remapping, like what they did here: py m,n = a.shape idx = np.mod((n-1)*np.arange(m)[:,None] + np.arange(n), n) out = a[np.arange(m)[:,None], idx]
https://stackoverflow.com/a/42101326

[azure talon]

self.newF[a,1+self.cx[i]:-2+self.cx[i],1+self.cy[i]:-2+self.cy[i],i]=self.F[a,1:-2,1:-2,i]

I tried this trick to make the matrix have a border, and then shift the copy this way. It didn't speedup

I don't really understand how that is supposed to work

how do I plug in my cX and cY value

like he has it varying so that each array is shifted by a different amount, I guess, but doesn't shift vertically, just horizontally

I will invent a super fast method

All of these copying etc is probably just wrong

I bet I can do it with no copying

[lime cove]

From a previous project where I did index remapping to shift images:

xmesh, ymesh = np.meshgrid(
            np.arange(img.width), np.arange(img.height), sparse=False
        )
xmesh = (xmesh + 5) % img.width
ymesh = (ymesh + 7) % img.height
np_img = np.array(self.img)
np_img = np_img[ymesh.flatten(), xmesh.flatten()].reshape(np_img.shape)```

[azure talon]

I'm thinking something like this https://numpy.org/devdocs/reference/generated/numpy.lib.stride_tricks.sliding_window_view.html

I'm going to try effectively making my own "window view" into the data, instead of copying it

class LatticeFrame:
  def __init__(self, res, dir):
    self.precision = np.float32
    self.F = np.random.random((res,res)).astype(self.precision)
    self.index=np.array([0,0])
    self.dir=np.array(dir)
  def rotate(self):
    self.index+=np.mod(self.dir,res)

this is my trick solution

I let the data stay where it is, and I move the origin point instead

Tested, and indeed it's basically infinitely fast. I just need to write my helper functions so I can "fill in" the rotated out data in the correct place

[lime cove]

Nice

[plain tangle]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.