[Geoserver-devel] outline of OpenCV proposal

Jeff_Brower · November 3, 2015, 6:17pm

All-

Here is the outline of my opencv proposal, along with a bit of rationale.

Please let me know when there are questions. Thanks.

-Jeff

1) For proof of concept / demo purposes, identify a compute intensive (i.e. slow) image processing chain / sequence of
operations frequently required by GeoServer users. So far, Jody, Andrea, and Jukka have mentioned crop, affine, warp,
transform (pls clarify -- coordinates? or transform like FFT or DCT?), resize (resample), rendering maps from vector
data...

2) The demo should be something that:

-everyone knows what to expect; i.e. not
difficult to tell it has been done right

  -any speed improvement would have a major
   impact on GeoServer users, especially
   those with limited server resources
   or those using cloud servers

  -if the operation(s) requires video encode
   or decode (e.g. H.264 or H.265), for
   example due to cloud based operation,
   that's fine (our accelerated version
   of OpenCV handles that)

-if the operation(s) requires network I/O,
that's also fine (for same reason)

3) I assume that given the above input, my team can identify which JAI APIs are used. We would then implement an
interface layer between GeoServer and JAI that translates those APIs into equivalent or combination of OpenCV APIs.
For debug, results comparison, and benchmarking, it would be possible to control operation of each API.

In this step we may need community help in being sure we've got correct JAI APIs and correct usage. Obviously we
don't need any help on the OpenCV side.

4) This is a Signalogic effort, in close cooperation with Texas Instruments. We would be giving the interface layer
source code and any modified GeoServer source code back to the GeoServer community. If it works and gains approval, I
would expect it to become part of the GeoServer main branch.

5) The result would run with (i) x86 OpenCV alone (i.e. OpenCV open source, unmodified), and (ii) c66x OpenCV, which
means the OpenCV we have ported to Signalogic TI c66x CPU accelerator cards.

6a) I have no idea whether x86-only OpenCV will be faster than JAI or not.

6b) To be clear, Signalogic's major interest is in huge speed improvements (20 to 50 times) using the aforementioned
accelerator cards. This is typically called "hardware acceleration". However, this is *not* the GPU, FPGA, or ASIC
type of acceleration that might come to mind. The TI CPUs on the card have similar fundamental architecture as Intel
CPUs, can run any C, C++, and OpenMP code, are connected to their own NICs, have 2 GB of DDR3 memory each, etc. One
card has 64 cores; assuming a simple 1-to-1 core comparison with x86, the performance advantage can be quite
significant for a server with several cards.

jive · November 4, 2015, 6:21am

A few comments inline, but first up thanks for engaging the community and attending last nights meeting.

For proof of concept / demo purposes, identify a compute intensive (i.e. slow) image processing chain / sequence of

···

operations frequently required by GeoServer users. So far, Jody, Andrea, and Jukka have mentioned crop, affine, warp,
transform (pls clarify – coordinates? or transform like FFT or DCT?), resize (resample), rendering maps from vector
data…

Their are two transforms:

co-ordinate transform between two coordinate reference systems (for example a global 4326 dataset being projected to an albers equal area projection)
affine transform used to position the final image, or perform a rotation etc…

The demo should be something that:

-everyone knows what to expect; i.e. not
difficult to tell it has been done right

The above sequence is based on the common processing chain used for map generation. It is the only one that qualifies as frequently required by GeoServer users.

-any speed improvement would have a major
impact on GeoServer users, especially
those with limited server resources
or those using cloud servers

-if the operation(s) requires video encode
or decode (e.g. H.264 or H.265), for
example due to cloud based operation,
that’s fine (our accelerated version
of OpenCV handles that)

-if the operation(s) requires network I/O,
that’s also fine (for same reason)

From last night call it sounded like your users may want more of an image processing (i.e. data product) generation.

An easy one that would be of benefit to the community is the production of hillshade visualization from a digital elevation model. The calculation is online: from raw dem to the production of slope and gradient and then generate the final colours.

If that is not enough of a challenge (only one fork) you could try production of hypsometric hillside (so the colors used for the calculation change depending on elevation and environment factors). Natural Earth has a good example.

I assume that given the above input, my team can identify which JAI APIs are used. We would then implement an interface layer between GeoServer and JAI that translates those APIs into equivalent or combination of OpenCV APIs.

I hope last nights meeting identified the approach to use (add additional operators to the JAI-Ext project) acting as a “native” implementation that can be selected if the appropriate libraries are in place.

I really like this approach, it is nice and clean and gives us a strong performance boost for those in position to install.

The trick here though is that JAI-Ext is a seperate project (many of the same players are here, just wanted to point out you would be working in a different codebase).