[GRASS-dev] r.in.gdal changes range of values for sentinel bands (?)

GRASS GRASS developer
,
1

Forgot the list, sorry…

···

2018-02-05 19:49 GMT+01:00 Markus Metz <markus.metz.giswork@gmail.com>:

On Mon, Feb 5, 2018 at 1:14 PM, Veronica Andreo <veroandreo@gmail.com> wrote:

I tested also with 74 release branch… same result

2018-02-05 13:05 GMT+01:00 Veronica Andreo <veroandreo@gmail.com>:

Hi devs

I’m testing Martin’s new add-on, r.sentinel.import. I created an UTM21S location for my data and imported them with:

r.sentinel.import input=Downloads/S2B_MSIL1C_20171212T134159_N0206_R124_T21KZP_20171212T201017.SAFE/GRANULE/L1C_T21KZP_A004011_20171212T134158/IMG_DATA/

The problem arose when I tried to use i.color.enhance to display an RGB: I got blank screens, so I thought there must be something with the range of values.

r.univar map=T21KZP_20171212T134159_B02@PERMANENT
total null and non-null cells: 120560400
total null cells: 0
Of the non-null cells:

n: 120560400
minimum: 15
maximum: 17547
range: 17532
mean: 913.778
mean of absolute values: 913.778
standard deviation: 134.322
variance: 18042.4
variation coefficient: 14.6996 %
sum: 110165471238

that already seemed odd…
I checked the original band2 jp2 file… Values are different…

gdalinfo T21KZP_20171212T134159_B02.jp2
Driver: JP2OpenJPEG/JPEG-2000 driver based on OpenJPEG library
Files: T21KZP_20171212T134159_B02.jp2
…
Band 1 Block=1024x1024 Type=UInt16, ColorInterp=Gray
Min=602.000 Max=1576.000
Minimum=602.000, Maximum=1576.000, Mean=916.799, StdDev=123.509
Overviews: 5490x5490, 2745x2745, 1372x1372, 686x686
Overviews: arbitrary
Metadata:
STATISTICS_MAXIMUM=1576
STATISTICS_MEAN=916.7992
STATISTICS_MINIMUM=602
STATISTICS_STDDEV=123.50943477872
Image Structure Metadata:
COMPRESSION=JPEG2000
NBITS=15

Can you try again with gdalinfo -mm? This forces computation of the actual min/max values, which can differ from what is stored in the metadata (an approximation).

There can also be something wrong with i.color.enhance

Markus M

Just to check, I imported with r.in.gdal and I’m getting exactly the same as with r.sentinel.import.

What am I doing wrong here? Why would r.in.gdal change values like that? What else needs to be set? I don’t remember to have this problem before and it happens in all bands as far as I’ve checked.

Ah, I’m using trunk r72203

Thanks in advance for any help :slight_smile:

Cheers,
Vero

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

2

Sorry, sorry… that was a different band… I realize now. So, for the record, no issues,
gdalinfo -mm T21KZP_20171212T134159_B02.jp2
reports the same range as the one I see after importing in grass.

Computed Min/Max=15.000,17547.000

Again, sorry for the noise and thanks for the -mm hint, MM :slight_smile:

best,

Vero

···

2018-02-05 20:54 GMT+01:00 Veronica Andreo <veroandreo@gmail.com>:

Forgot the list, sorry…

Hi Markus,

here the output of gdalinfo -mm for the same band reported earlier:

gdalinfo -mm T21KZP_20171212T134159_B01.jp2

Driver: JP2OpenJPEG/JPEG-2000 driver based on OpenJPEG library
Files: T21KZP_20171212T134159_B01.jp2
[…]

Band 1 Block=192x192 Type=UInt16, ColorInterp=Gray
Min=908.000 Max=1511.000 Computed Min/Max=934.000,6934.000
Minimum=908.000, Maximum=1511.000, Mean=1138.806, StdDev=74.933
Overviews: 915x915, 457x457, 228x228, 114x114
Overviews: arbitrary
Metadata:
STATISTICS_MAXIMUM=1511
STATISTICS_MEAN=1138.8064
STATISTICS_MINIMUM=908
STATISTICS_STDDEV=74.933459275813
Image Structure Metadata:
COMPRESSION=JPEG2000
NBITS=15

still differs a lot from what I get after importing into GRASS.

System Info
GRASS version: 7.5.svn
GRASS SVN revision: r72203
Build date: 2018-01-01
Build platform: x86_64-pc-linux-gnu
GDAL: 2.1.3
PROJ.4: 4.9.3
GEOS: 3.6.1
SQLite: 3.20.1
Python: 2.7.14
wxPython: 3.0.2.0
Platform: Linux-4.14.14-200.fc26.x86_64-x86_64-with-fedora-26-Twenty_Six

best,

Vero

2018-02-05 19:49 GMT+01:00 Markus Metz <markus.metz.giswork@gmail.com>:

On Mon, Feb 5, 2018 at 1:14 PM, Veronica Andreo <veroandreo@gmail.com> wrote:

I tested also with 74 release branch… same result

2018-02-05 13:05 GMT+01:00 Veronica Andreo <veroandreo@gmail.com>:

Hi devs

I’m testing Martin’s new add-on, r.sentinel.import. I created an UTM21S location for my data and imported them with:

r.sentinel.import input=Downloads/S2B_MSIL1C_20171212T134159_N0206_R124_T21KZP_20171212T201017.SAFE/GRANULE/L1C_T21KZP_A004011_20171212T134158/IMG_DATA/

The problem arose when I tried to use i.color.enhance to display an RGB: I got blank screens, so I thought there must be something with the range of values.

r.univar map=T21KZP_20171212T134159_B02@PERMANENT
total null and non-null cells: 120560400
total null cells: 0
Of the non-null cells:

n: 120560400
minimum: 15
maximum: 17547
range: 17532
mean: 913.778
mean of absolute values: 913.778
standard deviation: 134.322
variance: 18042.4
variation coefficient: 14.6996 %
sum: 110165471238

that already seemed odd…
I checked the original band2 jp2 file… Values are different…

gdalinfo T21KZP_20171212T134159_B02.jp2
Driver: JP2OpenJPEG/JPEG-2000 driver based on OpenJPEG library
Files: T21KZP_20171212T134159_B02.jp2
…
Band 1 Block=1024x1024 Type=UInt16, ColorInterp=Gray
Min=602.000 Max=1576.000
Minimum=602.000, Maximum=1576.000, Mean=916.799, StdDev=123.509
Overviews: 5490x5490, 2745x2745, 1372x1372, 686x686
Overviews: arbitrary
Metadata:
STATISTICS_MAXIMUM=1576
STATISTICS_MEAN=916.7992
STATISTICS_MINIMUM=602
STATISTICS_STDDEV=123.50943477872
Image Structure Metadata:
COMPRESSION=JPEG2000
NBITS=15

Can you try again with gdalinfo -mm? This forces computation of the actual min/max values, which can differ from what is stored in the metadata (an approximation).

There can also be something wrong with i.color.enhance

Markus M

Just to check, I imported with r.in.gdal and I’m getting exactly the same as with r.sentinel.import.

What am I doing wrong here? Why would r.in.gdal change values like that? What else needs to be set? I don’t remember to have this problem before and it happens in all bands as far as I’ve checked.

Ah, I’m using trunk r72203

Thanks in advance for any help :slight_smile:

Cheers,
Vero

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

3

On Tue, Feb 6, 2018 at 10:25 AM, Veronica Andreo <veroandreo@gmail.com> wrote:

Sorry, sorry… that was a different band… I realize now. So, for the record, no issues,
gdalinfo -mm T21KZP_20171212T134159_B02.jp2
reports the same range as the one I see after importing in grass.

Computed Min/Max=15.000,17547.000

Again, sorry for the noise and thanks for the -mm hint, MM :slight_smile:

Good to hear that gdalinfo -mm and GRASS stats are indeed identical.

A hint: use r.info -s instead of r.univar -g for simple stats (number of non-null cells, min, max, mean, stddev) on the full raster map, it’s faster.

The reason why gdalinfo with and without -mm might report different values is that some software creating raster datasets for some obscure reason estimates min, max, mean, stddev and for some even more obscure reason writes these estimates to metadata. At least regarding min, max one would expect actual min, max, not some estimate with estimated min > actual min and estimated max < actual max.

Markus M

best,
Vero

2018-02-05 20:54 GMT+01:00 Veronica Andreo <veroandreo@gmail.com>:

Forgot the list, sorry…

Hi Markus,

here the output of gdalinfo -mm for the same band reported earlier:

gdalinfo -mm T21KZP_20171212T134159_B01.jp2
Driver: JP2OpenJPEG/JPEG-2000 driver based on OpenJPEG library
Files: T21KZP_20171212T134159_B01.jp2
[…]

Band 1 Block=192x192 Type=UInt16, ColorInterp=Gray
Min=908.000 Max=1511.000 Computed Min/Max=934.000,6934.000
Minimum=908.000, Maximum=1511.000, Mean=1138.806, StdDev=74.933
Overviews: 915x915, 457x457, 228x228, 114x114
Overviews: arbitrary
Metadata:
STATISTICS_MAXIMUM=1511
STATISTICS_MEAN=1138.8064
STATISTICS_MINIMUM=908
STATISTICS_STDDEV=74.933459275813
Image Structure Metadata:
COMPRESSION=JPEG2000
NBITS=15

still differs a lot from what I get after importing into GRASS.

System Info
GRASS version: 7.5.svn
GRASS SVN revision: r72203
Build date: 2018-01-01
Build platform: x86_64-pc-linux-gnu
GDAL: 2.1.3
PROJ.4: 4.9.3
GEOS: 3.6.1
SQLite: 3.20.1
Python: 2.7.14
wxPython: 3.0.2.0
Platform: Linux-4.14.14-200.fc26.x86_64-x86_64-with-fedora-26-Twenty_Six

best,
Vero

2018-02-05 19:49 GMT+01:00 Markus Metz <markus.metz.giswork@gmail.com>:

On Mon, Feb 5, 2018 at 1:14 PM, Veronica Andreo <veroandreo@gmail.com> wrote:

I tested also with 74 release branch… same result

2018-02-05 13:05 GMT+01:00 Veronica Andreo <veroandreo@gmail.com>:

Hi devs

I’m testing Martin’s new add-on, r.sentinel.import. I created an UTM21S location for my data and imported them with:

r.sentinel.import input=Downloads/S2B_MSIL1C_20171212T134159_N0206_R124_T21KZP_20171212T201017.SAFE/GRANULE/L1C_T21KZP_A004011_20171212T134158/IMG_DATA/

The problem arose when I tried to use i.color.enhance to display an RGB: I got blank screens, so I thought there must be something with the range of values.

r.univar map=T21KZP_20171212T134159_B02@PERMANENT
total null and non-null cells: 120560400
total null cells: 0
Of the non-null cells:

n: 120560400
minimum: 15
maximum: 17547
range: 17532
mean: 913.778
mean of absolute values: 913.778
standard deviation: 134.322
variance: 18042.4
variation coefficient: 14.6996 %
sum: 110165471238

that already seemed odd…
I checked the original band2 jp2 file… Values are different…

gdalinfo T21KZP_20171212T134159_B02.jp2
Driver: JP2OpenJPEG/JPEG-2000 driver based on OpenJPEG library
Files: T21KZP_20171212T134159_B02.jp2
…
Band 1 Block=1024x1024 Type=UInt16, ColorInterp=Gray
Min=602.000 Max=1576.000
Minimum=602.000, Maximum=1576.000, Mean=916.799, StdDev=123.509
Overviews: 5490x5490, 2745x2745, 1372x1372, 686x686
Overviews: arbitrary
Metadata:
STATISTICS_MAXIMUM=1576
STATISTICS_MEAN=916.7992
STATISTICS_MINIMUM=602
STATISTICS_STDDEV=123.50943477872
Image Structure Metadata:
COMPRESSION=JPEG2000
NBITS=15

Can you try again with gdalinfo -mm? This forces computation of the actual min/max values, which can differ from what is stored in the metadata (an approximation).

There can also be something wrong with i.color.enhance

Markus M

Just to check, I imported with r.in.gdal and I’m getting exactly the same as with r.sentinel.import.

What am I doing wrong here? Why would r.in.gdal change values like that? What else needs to be set? I don’t remember to have this problem before and it happens in all bands as far as I’ve checked.

Ah, I’m using trunk r72203

Thanks in advance for any help :slight_smile:

Cheers,
Vero

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev