What I worked on in week 8:
The r.param.scale PR got merged this week with the lu.c race fix from last week, so that whole workstream is done
The rest of the week I wasn’t able to work on r.proj as much as I wanted to but I was still able to accomplish a lot. The first thing finished was the pole map fix, so the parallel r.proj now handles maps with a north or south pole inside the output frame. That closed the final map case for the nearest method.
Then I found that the non-nearest methods were silently broken in the parallel path. If you asked for methods like “bilinear”, “bicubic”, or “lanczos”, the code silently uses the “nearest” method instead and gives that output with no warning. The parallel path needed its own versions of the bilinear, bicubic, and lanczos code, so I wrote those by carefully copying the math from the original module and hooking them up the same way the original picks between methods. While checking their output against the original I found a second bug. Some output rows landed at a very slightly different position, smaller than any real map difference but enough that the outputs were not exactly identical. The cause was that the original module calculates each row’s position by subtracting step by step, row after row, while my code jumped straight there with one multiplication. On a computer those two ways can give answers that differ in the last decimal place. I changed my code to calculate it step by step like the original, and after that the outputs match exactly.
The third thing was the memory cap edge case. There are inputs so wide that even one output row’s input strip cannot fit under a small cap, and my code used to just cancel there, while the old r.proj would finish slowly. Now the parallel r.proj falls back to the old serial tile cache for those runs and prints the exact memory value that would keep the parallel path. This way if the user wanted the benefits of the parallelization and could spare some more memory they would know exactly how much memory they need to activate the parallel path for their map. However if they really can’t spare more memory, then it just falls back to the tile cache, but this fallback only occurs on extremely rare and purposely wide input tests. I swept through different memory caps to see how often this fires, and it never triggered on any real projection pair down to 1 MB. Only a deliberately wide test input hits it.
To close the week I ran a full verification and benchmark suite overnight. The full list of benchmarks I ran are at my latest comment within the #7627 draft PR.
What I’m doing in week 9: I’ll create a small separate PR fixing two global variables in the projection library that are technically a data race, with thread sanitizer as the proof. Create new pytest cases for the interpolation methods, pole maps, and the fallback. Then two performance items from previous mentor review, keeping the input resident across bands and overlapping the output write with compute. I’m trying to bring down the phases in my parallelization that still run in serial as they bring down the total speedup, even though the compute speed is high.
What’s blocking me? Just waiting on a couple of review answers but nothing that really stops me from focusing on the list of tasks above.
Thanks,
Kaushik