[GRASS-dev] Image Segmentation - Summer of Code

GRASS GIS GRASS GIS developer
1

Hi,

Thanks everyone for the nice greetings, I’m very impressed with how polite and helpful the GRASS community is.

There is a wiki [1] and source code repository [2] for the Image Segmentation - Google Summer of Code project.

The mentors for the project have already provided a lot of advice, if there is anyone else with an interest in segmentation you are welcome to add to the Specifications or other areas of the wiki.

For the “code” I have started an outline, but I think there are still too many questions to call it pseudocode yet!. Many of the questions are for myself to research and answer. Of course, I welcome input and suggestions for all of it, but some of the bigger questions are:

  1. How to best deal with images that are larger then available memory. (I assume too much disk I/O would be slow, but simple tiling could lead to poor segmentation at the tile borders.)

  2. How to find neighbors. (Is this already available somewhere in GRASS?)

  3. Would any functions would be useful as a general library function.

  4. If library functions are already available for any steps.

I saw GSoC is on the agenda for the Sprint, if it would be helpful (and isn’t too early in the day!) I can join by Google video / IRC / etc.

Regards,
Eric

[1] http://grass.osgeo.org/wiki/GRASS_GSoC_2012_Image_Segmentation

[2] https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.segment

2

On Thu, May 24, 2012 at 7:29 PM, Eric Momsen <eric.momsen@gmail.com> wrote:

Hi,

Thanks everyone for the nice greetings, I'm very impressed with how polite
and helpful the GRASS community is.

There is a wiki [1] and source code repository [2] for the Image
Segmentation - Google Summer of Code project.

The mentors for the project have already provided a lot of advice, if there
is anyone else with an interest in segmentation you are welcome to add to
the Specifications or other areas of the wiki.

For the "code" I have started an outline, but I think there are still too
many questions to call it pseudocode yet!. Many of the questions are for
myself to research and answer. Of course, I welcome input and suggestions
for all of it, but some of the bigger questions are:

1. How to best deal with images that are larger then available memory. (I
assume too much disk I/O would be slow, but simple tiling could lead to poor
segmentation at the tile borders.)

There are two tile cache mechanisms in GRASS, once the read-only cache
used by r.proj and i.rectify, once the more flexible and read/write
cache of the segment library. In this case I would opt for the segment
library because the data stored there can be any size, determined on
run-time. You could e.g. put all input bands (all maps in a group)
into one array and cache that array with the segment lib. The size of
the array corresponds thus to the number of input bands.

2. How to find neighbors. (Is this already available somewhere in GRASS?)

What neighbors do you mean? The eight adjacent neighbors or neighbours
in a larger neighborhood? You could look at r.neighbors for a general
solution of any neighborhood size and at r.watershed for the eight
adjacent neighbors.

3. Would any functions would be useful as a general library function.

What functions exactly do you have in mind?

4. If library functions are already available for any steps.

I hope the above comments help a bit. A lot of useful functionality is
implemented not only in libraries but also in modules.

I saw GSoC is on the agenda for the Sprint, if it would be helpful (and
isn't too early in the day!) I can join by Google video / IRC / etc.

We address items on the agenda more in an ad-hoc approach, no fixed
schedule, but you can contact us any time.

Hope that helps,

Markus M

Regards,
Eric

[1] http://grass.osgeo.org/wiki/GRASS_GSoC_2012_Image_Segmentation

[2]
https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.segment

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

3

Follow up on some of the questions below.

Also, I posted a brief project report to the other mail list:
http://lists.osgeo.org/pipermail/soc/2012-May/001747.html

On Fri, May 25, 2012 at 3:47 AM, Markus Metz <markus.metz.giswork@googlemail.com> wrote:

On Thu, May 24, 2012 at 7:29 PM, Eric Momsen <eric.momsen@gmail.com> wrote:

  1. How to best deal with images that are larger then available memory. (I
    assume too much disk I/O would be slow, but simple tiling could lead to poor
    segmentation at the tile borders.)

There are two tile cache mechanisms in GRASS, once the read-only cache
used by r.proj and i.rectify, once the more flexible and read/write
cache of the segment library. In this case I would opt for the segment
library because the data stored there can be any size, determined on
run-time. You could e.g. put all input bands (all maps in a group)
into one array and cache that array with the segment lib. The size of
the array corresponds thus to the number of input bands.

OK, I’ll take a look at these, my biggest concern is how to deal with the interior borders, to make sure segmentation can continue across those artificial boundaries.

  1. How to find neighbors. (Is this already available somewhere in GRASS?)

What neighbors do you mean? The eight adjacent neighbors or neighbours
in a larger neighborhood? You could look at r.neighbors for a general
solution of any neighborhood size and at r.watershed for the eight
adjacent neighbors.

Does 4 or 8 neighbors make more sense for segmentation? (I’m thinking with diagonal neighbors, you could end up with two larger regions only joined at a diagonal in the center - but maybe that is over thinking what will occur with real images)

Finding pixel neighbors will be just the first step. After the segments are growing larger, the both the center region can be irregular (not just one pixel), and the set of possible neighbors will include irregular shapes and pixels.

So I guess this is probably similar to finding a 1 pixel buffer, then reducing those pixels to just check the unique segments… so I’ll look at r.buffer as well.

I saw GSoC is on the agenda for the Sprint, if it would be helpful (and
isn’t too early in the day!) I can join by Google video / IRC / etc.

We address items on the agenda more in an ad-hoc approach, no fixed
schedule, but you can contact us any time.

Sure, but my schedule is pretty flexible, I wouldn’t need much warning to get online.

Hope that helps,

Yes, thanks
Eric

Markus M

Regards,
Eric

[1] http://grass.osgeo.org/wiki/GRASS_GSoC_2012_Image_Segmentation

[2]
https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.segment

grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev

4

Eric,

On 24/05/12 19:29, Eric Momsen wrote:

There is a wiki [1] and source code repository [2] for the Image
Segmentation - Google Summer of Code project.

[...]

For the "code" I have started an outline, but I think there are still
too many questions to call it pseudocode yet!. Many of the questions
are for myself to research and answer. Of course, I welcome input and
suggestions for all of it, but some of the bigger questions are:

[...]

[2]
https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.segment

I've just added a series of comments (all starting with 'ML:') to your pseudo-code. Mostly on general questions. For the more technical coding questions Markus M. will be of more help.

Moritz

5

On Sat, May 26, 2012 at 8:10 AM, Moritz Lennert <mlennert@club.worldonline.be> wrote:

[…]

[2]
https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.segment

I’ve just added a series of comments (all starting with ‘ML:’) to your pseudo-code. Mostly on general questions. For the more technical coding questions Markus M. will be of more help.

Moritz

Thanks, from all the comments and suggestions I should be heading in the right direction this week. I added answers and follow up for most of the discussion oriented comments you had in the pseudo-code.

-Eric

6

On 28/05/12 06:14, Eric Momsen wrote:

On Sat, May 26, 2012 at 8:10 AM, Moritz Lennert
<mlennert@club.worldonline.be <mailto:mlennert@club.worldonline.be>> wrote:

    [...]

        [2]
        https://trac.osgeo.org/grass/__browser/grass-addons/grass7/__imagery/i.segment
        <https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.segment&gt;

    I've just added a series of comments (all starting with 'ML:') to
    your pseudo-code. Mostly on general questions. For the more
    technical coding questions Markus M. will be of more help.

    Moritz

Thanks, from all the comments and suggestions I should be heading in the
right direction this week. I added answers and follow up for most of
the discussion oriented comments you had in the pseudo-code.

And I've added some more of my own. It would be great to have some feedback from others in this discussion. However, maybe doing this in svn might be a bit too hidden... Maybe at least parts of the discussion should be moved back to the wiki ?

Moritz

7

On Tue, May 29, 2012 at 5:08 AM, Moritz Lennert <mlennert@club.worldonline.be> wrote:

On 28/05/12 06:14, Eric Momsen wrote:

On Sat, May 26, 2012 at 8:10 AM, Moritz Lennert

<mlennert@club.worldonline.be mailto:[mlennert@club.worldonline.be](mailto:mlennert@club.worldonline.be)> wrote:

[…]

[2]

https://trac.osgeo.org/grass/__browser/grass-addons/grass7/__imagery/i.segment

<https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.segment>

I’ve just added a series of comments (all starting with ‘ML:’) to
your pseudo-code. Mostly on general questions. For the more
technical coding questions Markus M. will be of more help.

Moritz

Thanks, from all the comments and suggestions I should be heading in the
right direction this week. I added answers and follow up for most of
the discussion oriented comments you had in the pseudo-code.

And I’ve added some more of my own. It would be great to have some feedback from others in this discussion. However, maybe doing this in svn might be a bit too hidden… Maybe at least parts of the discussion should be moved back to the wiki ?

Moritz

Thanks. I replied some in svn. The discussions that seemed most related to specifications I’ve put on the wiki, we can continue there or in email:

http://grass.osgeo.org/wiki/GRASS_GSoC_2012_Image_Segmentation#Questions

-Eric

8

On Tue, May 29, 2012 at 5:23 PM, Eric Momsen <eric.momsen@gmail.com> wrote:

On Tue, May 29, 2012 at 5:08 AM, Moritz Lennert
<mlennert@club.worldonline.be> wrote:

On 28/05/12 06:14, Eric Momsen wrote:

On Sat, May 26, 2012 at 8:10 AM, Moritz Lennert
<mlennert@club.worldonline.be <mailto:mlennert@club.worldonline.be>>
wrote:

[...]

   \[2\]

https://trac.osgeo.org/grass/__browser/grass-addons/grass7/__imagery/i.segment

<https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.segment&gt;

I've just added a series of comments (all starting with 'ML:') to
your pseudo-code. Mostly on general questions. For the more
technical coding questions Markus M. will be of more help.

Moritz

Thanks, from all the comments and suggestions I should be heading in the
right direction this week. I added answers and follow up for most of
the discussion oriented comments you had in the pseudo-code.

And I've added some more of my own. It would be great to have some
feedback from others in this discussion. However, maybe doing this in svn
might be a bit too hidden... Maybe at least parts of the discussion should
be moved back to the wiki ?

Moritz

Thanks. I replied some in svn. The discussions that seemed most related to
specifications I've put on the wiki, we can continue there or in email:

http://grass.osgeo.org/wiki/GRASS_GSoC_2012_Image_Segmentation#Questions

I have added comments to some technical aspects to outline and
modified the Makefile a bit to include all needed libs.

Markus M

9

Hi Eric,

great work with the i.segment, I just made a test with the NC Landsat
sample data:

i.segment -w -l group=lsat7_2000 output=testsegment threshold=4 \
               method=region_growing similarity=euclidean minsize=20

The result looks already pretty nice without further parameter tuning.

It would now be cool to susequently perform a unsupervised classification
with, say, i.cluster/i.maxlik in order to group the segments to a certain
number of classes.

Would that be feasible from the resulting image statistics?

thanks
MarkusN

10

On 06/09/12 15:44, Markus Neteler wrote:

Hi Eric,

great work with the i.segment, I just made a test with the NC Landsat
sample data:

i.segment -w -l group=lsat7_2000 output=testsegment threshold=4 \
                method=region_growing similarity=euclidean minsize=20

The result looks already pretty nice without further parameter tuning.

It would now be cool to susequently perform a unsupervised classification
with, say, i.cluster/i.maxlik in order to group the segments to a certain
number of classes.

Would that be feasible from the resulting image statistics?

I'm not sure that i.cluster is the right tool here. I think it might be more efficient / easier to actually transform the segments into vectors and fill up the attribute table with a whole series of statistics concerning spectral, shape, context, etc characteristics of these segments.

I guess you could also create a whole series of new bands, one for each of the characteristics mentioned above, in which for each pixel you put the statistic of its segment concering the respective characteristic, and submit that to i.cluster. But somehow that doesn't sound as efficient to me...

Moritz

11

On Thu, Sep 6, 2012 at 8:59 AM, Moritz Lennert <mlennert@club.worldonline.be> wrote:

On 06/09/12 15:44, Markus Neteler wrote:

Hi Eric,

great work with the i.segment, I just made a test with the NC Landsat
sample data:

i.segment -w -l group=lsat7_2000 output=testsegment threshold=4
method=region_growing similarity=euclidean minsize=20

The result looks already pretty nice without further parameter tuning.

It would now be cool to susequently perform a unsupervised classification
with, say, i.cluster/i.maxlik in order to group the segments to a certain
number of classes.

Would that be feasible from the resulting image statistics?

I’m not sure that i.cluster is the right tool here. I think it might be more efficient / easier to actually transform the segments into vectors and fill up the attribute table with a whole series of statistics concerning spectral, shape, context, etc characteristics of these segments.

I guess you could also create a whole series of new bands, one for each of the characteristics mentioned above, in which for each pixel you put the statistic of its segment concering the respective characteristic, and submit that to i.cluster. But somehow that doesn’t sound as efficient to me…

Moritz

It is nice to hear more people are trying out i.segment!

This week I updated my local copy of r.univar to include number of segments as the last column of the output table. Just a quick hack, if it were to be considered to be added in there are definitely some error checking, etc that should be added too. I started there because of the zone functionality in r.univar: I could use the segment results as the zone map (and input as the single raster map I had segmented) and then r.univar could find statistics for each segment.

I’m not familiar (yet) with i.cluster/i.maxlik to be able to comment on that part. I’m certainly interested to hear the most pressing needs for using the outputs from i.segment, and if some updates like that should be polished a bit more to be considered for everyone.

(I need to run to a meeting, if there is interest in what I did to r.univar I can send it out later.)

-Eric