What I worked on in week 2: I refactored the code for draft PR #7440 to fix the weak speedup ratios I had last week. The parallel version now uses the same two level band setup that r.neighbors uses. It works through the map band by band, splits each band’s rows across the threads, and each thread only holds a small rolling window of rows instead of a large part of the map. I also limited memory usage with the standard memory= option and added mask handling. On a 100 million cell raster the peak went from about 620 MB down to about 330 MB. The output is still identical to the serial module’s output at one thread and with a mask.
While reworking the module I found and fixed a bug that’s in the original serial module too. When the window is larger than the region, it used to write past the end of the map. The parallel version handles that case cleanly now.
I tested the refactored code with a bunch of different benchmarking scripts and different window sizes, every test case was with a 100 million cell raster. For one of the test cases, I used window size 31 and I saw about 1.91x at 2 threads, 3.37x at 4, and 4.55x at 8. I added the benchmark script and a script to generate the test raster to the PR so mentors can reproduce it on their own machines.
What I’m doing this week: Finalizing #7440 for merge, responding to review feedback, and clearing the remaining CI checks, and fixing any other bugs that I come across after stress testing.
What’s blocking me: Nothing right now, I’ll work on this week’s tasks and also address mentor feedback as it comes in.
Thanks,
Kaushik