[Geoserver-users] Error while creating store with raster data -image mosaic

Hi,

So we are trying to switch to Geoserver 2.22.2 version, but running into following problem while trying to create a store with image mosaic. I tried to test on the 2.21 version, it works on that with our properties. However, on 2.22 and above, it’s the same error I am seeing. Attached are the logs and a screenshot of the UI. Please let me know if you have solution for this. I have seen online that users are having some error while upgrading to the 2.22 and above. Please let me know if you have any question for me. Thank you

Ashish Jonchhen

Software Engineer I

C : +1 620.757.3966

ashish.jonchhen@anonymised.com

He/Him/His

Raytheon Intelligence & Space

1717 E. Cityline Drive

Richardson, TX 75082

RTX.com | LinkedIn | Twitter | Instagram

geoserver.log (592 KB)

geoserverError3.JPG

I wanted to provide a little more detail on this question.

We’re working with an ImageMosaic Store that we’ve used successfully with every major GeoServer version from 2.17 - 2.21 but we are having the problems Ashish mentioned on GeoServer 2.22 or 2.23. We configure our Store to index the granules in Postgres using the ImageMosaic configuration files.

[datastore.properties]

StoreName=cite:ImageMosaicPostGIS

[BrowseImageMosaicStore.properties]

ExpandToRGB=false

TypeName=BrowseImageMosaicStore

SuggestedFormat=org.geotools.gce.image.WorldImageFormat

Name=BrowseImageMosaicStore

SuggestedSPI=com.sun.imageio.plugins.jpeg.JPEGImageReaderSpi

LevelsNum=1

PathType=ABSOLUTE

Heterogeneous=false

Caching=false

HeterogeneousCRS=false

LocationAttribute=location

Levels=1.2793554128E-4,1.0810019425E-4

CheckAuxiliaryMetadata=false

MosaicCRS=EPSG:404000

[indexer.properties]

Caching=false

AbsolutePath=true

Schema=*the_geom:Polygon,location:String

PropertyCollectors=

···

From: Jonchhen, Ashish RTX via Geoserver-users geoserver-users@lists.sourceforge.net
Sent: Friday, April 21, 2023 3:19 PM
To: geoserver-users@lists.sourceforge.net
Subject: [External] [Geoserver-users] Error while creating store with raster data -image mosaic

Hi,

So we are trying to switch to Geoserver 2.22.2 version, but running into following problem while trying to create a store with image mosaic. I tried to test on the 2.21 version, it works on that with our properties. However, on 2.22 and above, it’s the same error I am seeing. Attached are the logs and a screenshot of the UI. Please let me know if you have solution for this. I have seen online that users are having some error while upgrading to the 2.22 and above. Please let me know if you have any question for me. Thank you

Ashish Jonchhen

Software Engineer I

C : +1 620.757.3966

ashish.jonchhen@anonymised.com

He/Him/His

Raytheon Intelligence & Space

1717 E. Cityline Drive

Richardson, TX 75082

RTX.com | LinkedIn | Twitter | Instagram

Thanks for the detail, the information in screen snap (and the information when a raster cannot be accessed) is not always useful for troubleshooting.

I assume you updated to Java 11, is the com.sun.imageio.plugins.jpeg.JPEGImageReaderSpi still available in Java 11?

What happens if you take a few granules and place them in a folder to generate a new image mosaic? You could compare a newly generated datastore.properties to the one you have been using.

Jody

···


Jody Garnett

We have tried Java8 and Java11. If I just make an ImageMosaic with some granules, my generated BrowseImageMosaicStore.properties is the same. I don’t know how to auto generate the datastore.properties file.

Some other errors we will see is:

Why is it trying to create the schema? I just want to connect to it.

22 Apr 12:09:12 ERROR [gce.imagemosaic] - Schema ‘BrowseImageMosaicStore’ already exists

java.lang.IllegalArgumentException: Schema ‘BrowseImageMosaicStore’ already exists

at org.geotools.jdbc.JDBCDataStore.createSchema(JDBCDataStore.java:731)

at org.geotools.jdbc.JDBCDataStore.createSchema(JDBCDataStore.java:150)

at org.geotools.gce.imagemosaic.catalog.AbstractGTDataStoreGranuleCatalog.createType(AbstractGTDataStoreGranuleCatalog.java:563)

at org.geotools.gce.imagemosaic.catalog.RepositoryDataStoreCatalog.createType(RepositoryDataStoreCatalog.java:40)

This is a bit odd. I am providing the folder containing my properties files as usual.

22 Apr 12:09:12 DEBUG [gce.imagemosaic] - This plugin accepts File, URL or String. The string may describe a File or an URL

org.geotools.data.DataSourceException: This plugin accepts File, URL or String. The string may describe a File or an URL

at org.geotools.gce.imagemosaic.ImageMosaicReader.(ImageMosaicReader.java:219)

at org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:563)

at org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:104)

at org.geoserver.catalog.ResourcePool.getGridCoverageReader(ResourcePool.java:1594)

···

From: Jody Garnett <jody.garnett@…84…>
Sent: Saturday, April 22, 2023 3:14 AM
To: Mort, Ashley C RTX <Ashley.C.Mort@…11756…>
Cc: geoserver-users@lists.sourceforge.net
Subject: [External] Re: [Geoserver-users] Error while creating store with raster data -image mosaic

Thanks for the detail, the information in screen snap (and the information when a raster cannot be accessed) is not always useful for troubleshooting.

I assume you updated to Java 11, is the com.sun.imageio.plugins.jpeg.JPEGImageReaderSpi still available in Java 11?

What happens if you take a few granules and place them in a folder to generate a new image mosaic? You could compare a newly generated datastore.properties to the one you have been using.

Jody

On Fri, Apr 21, 2023 at 8:41 PM Mort, Ashley C RTX via Geoserver-users <geoserver-users@lists.sourceforge.net> wrote:

I wanted to provide a little more detail on this question.

We’re working with an ImageMosaic Store that we’ve used successfully with every major GeoServer version from 2.17 - 2.21 but we are having the problems Ashish mentioned on GeoServer 2.22 or 2.23. We configure our Store to index the granules in Postgres using the ImageMosaic configuration files.

[datastore.properties]

StoreName=cite:ImageMosaicPostGIS

[BrowseImageMosaicStore.properties]

ExpandToRGB=false

TypeName=BrowseImageMosaicStore

SuggestedFormat=org.geotools.gce.image.WorldImageFormat

Name=BrowseImageMosaicStore

SuggestedSPI=com.sun.imageio.plugins.jpeg.JPEGImageReaderSpi

LevelsNum=1

PathType=ABSOLUTE

Heterogeneous=false

Caching=false

HeterogeneousCRS=false

LocationAttribute=location

Levels=1.2793554128E-4,1.0810019425E-4

CheckAuxiliaryMetadata=false

MosaicCRS=EPSG:404000

[indexer.properties]

Caching=false

AbsolutePath=true

Schema=*the_geom:Polygon,location:String

PropertyCollectors=

From: Jonchhen, Ashish RTX via Geoserver-users <geoserver-users@lists.sourceforge.net>
Sent: Friday, April 21, 2023 3:19 PM
To: geoserver-users@lists.sourceforge.net
Subject: [External] [Geoserver-users] Error while creating store with raster data -image mosaic

Hi,

So we are trying to switch to Geoserver 2.22.2 version, but running into following problem while trying to create a store with image mosaic. I tried to test on the 2.21 version, it works on that with our properties. However, on 2.22 and above, it’s the same error I am seeing. Attached are the logs and a screenshot of the UI. Please let me know if you have solution for this. I have seen online that users are having some error while upgrading to the 2.22 and above. Please let me know if you have any question for me. Thank you

Ashish Jonchhen

Software Engineer I

C : +1 620.757.3966

ashish.jonchhen@…11756…

He/Him/His

Raytheon Intelligence & Space

1717 E. Cityline Drive

Richardson, TX 75082

RTX.com | LinkedIn | Twitter | Instagram


Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Jody Garnett

Is there anything else I can do to help investigate this issue?

Our simplest test is:

  1. Put 1 granule (we use 1 jpg and 1 jgw world file) in a folder (data_dir/myFolder) w/ a datastore.properties
    [datastore.properties]
    StoreName=cite:ImageMosaicPostGIS

cite:ImageMosaicPostGIS is a PostGIS datastore points to a local vanilla Postgres (15) that has postgis installed.

  1. Manually create an ImageMosaic Store pointing to data_dir/myFolder using the GeoServer web interface.

GeoServer versions prior to 2.22 will create the table in Postgres and create a .properties file for the Store then send you to the Edit Layer page. GeoServer 2.22 will stacktrace a variety of problems w/ GeoTools developer logging. I tried remote Java debugging GeoTools 28.2 with GeoServer 2.22.2 in intellij but I end up with a variety of stack traces that aren’t explicitly helpful and stepping through was a bit confusing. Creating the Image Mosaic Store without the datastore.properties (and using the Shapefile indexing) works fine.

Is there anything else I can do to document the problem? I could make a videocast or something but idk if that’s helpful at all.

Thanks again!

···

From: Jody Garnett <jody.garnett@…84…>
Sent: Saturday, April 22, 2023 3:14 AM
To: Mort, Ashley C RTX <Ashley.C.Mort@…11756…>
Cc: geoserver-users@lists.sourceforge.net
Subject: [External] Re: [Geoserver-users] Error while creating store with raster data -image mosaic

Thanks for the detail, the information in screen snap (and the information when a raster cannot be accessed) is not always useful for troubleshooting.

I assume you updated to Java 11, is the com.sun.imageio.plugins.jpeg.JPEGImageReaderSpi still available in Java 11?

What happens if you take a few granules and place them in a folder to generate a new image mosaic? You could compare a newly generated datastore.properties to the one you have been using.

Jody

On Fri, Apr 21, 2023 at 8:41 PM Mort, Ashley C RTX via Geoserver-users <geoserver-users@lists.sourceforge.net> wrote:

I wanted to provide a little more detail on this question.

We’re working with an ImageMosaic Store that we’ve used successfully with every major GeoServer version from 2.17 - 2.21 but we are having the problems Ashish mentioned on GeoServer 2.22 or 2.23. We configure our Store to index the granules in Postgres using the ImageMosaic configuration files.

[datastore.properties]

StoreName=cite:ImageMosaicPostGIS

[BrowseImageMosaicStore.properties]

ExpandToRGB=false

TypeName=BrowseImageMosaicStore

SuggestedFormat=org.geotools.gce.image.WorldImageFormat

Name=BrowseImageMosaicStore

SuggestedSPI=com.sun.imageio.plugins.jpeg.JPEGImageReaderSpi

LevelsNum=1

PathType=ABSOLUTE

Heterogeneous=false

Caching=false

HeterogeneousCRS=false

LocationAttribute=location

Levels=1.2793554128E-4,1.0810019425E-4

CheckAuxiliaryMetadata=false

MosaicCRS=EPSG:404000

[indexer.properties]

Caching=false

AbsolutePath=true

Schema=*the_geom:Polygon,location:String

PropertyCollectors=

From: Jonchhen, Ashish RTX via Geoserver-users <geoserver-users@lists.sourceforge.net>
Sent: Friday, April 21, 2023 3:19 PM
To: geoserver-users@lists.sourceforge.net
Subject: [External] [Geoserver-users] Error while creating store with raster data -image mosaic

Hi,

So we are trying to switch to Geoserver 2.22.2 version, but running into following problem while trying to create a store with image mosaic. I tried to test on the 2.21 version, it works on that with our properties. However, on 2.22 and above, it’s the same error I am seeing. Attached are the logs and a screenshot of the UI. Please let me know if you have solution for this. I have seen online that users are having some error while upgrading to the 2.22 and above. Please let me know if you have any question for me. Thank you

Ashish Jonchhen

Software Engineer I

C : +1 620.757.3966

ashish.jonchhen@…11756…

He/Him/His

Raytheon Intelligence & Space

1717 E. Cityline Drive

Richardson, TX 75082

RTX.com | LinkedIn | Twitter | Instagram


Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Jody Garnett

Hey Ashley:

Did you get it working with only a couple of granules and no PostGIS? From your description it sounded like you skipped straight to using a database?

  1. It does not appear anyone else has replied to your email saying “I have that problem also”. We could try clearly asking if anyone is using PostGIS + image mosaic in GeoServer 2.22.x

  2. You should check if the issue is already reported, and if not please report the issue with the information you have managed to collect thus far.

  3. If resolving this matter is time sensitive you may wish to seek help from some of the commercial support providers.

I am not greatly familiar with the image mosaic code myself. As a developer I would step through the code in the older release to see how it is intended to work. And then perform the same steps in the newer release.

Jody

···


Jody Garnett

I will try to create a bug report on the issue.

It looks like the core problem is that with a couple of granules and no PostGIS, it uses ShapefileFeatureStore which is able to determine the bounds and crs properly and works fine. When I add a datastore.properties to use PostGIS, it uses JDBCFeatureStore instead and the crs on that is null which is the cause of my NPE.

java.lang.NullPointerException
at org.geotools.gce.imagemosaic.RasterManager$SpatialDomainManager.prepareCoverageSpatialElements(RasterManager.java:236)
at org.geotools.gce.imagemosaic.RasterManager$SpatialDomainManager.(RasterManager.java:217)
at org.geotools.gce.imagemosaic.RasterManager.initialize(RasterManager.java:1581)
at org.geotools.gce.imagemosaic.RasterManager.initialize(RasterManager.java:1542)
at org.geotools.gce.imagemosaic.ImageMosaicConfigHandler.indexingPostamble(ImageMosaicConfigHandler.java:1082)
at org.geotools.gce.imagemosaic.ImageMosaicDirectoryWalker$MosaicDirectoryWalker.(ImageMosaicDirectoryWalker.java:139)
at org.geotools.gce.imagemosaic.ImageMosaicDirectoryWalker.run(ImageMosaicDirectoryWalker.java:218)
at org.geotools.gce.imagemosaic.Utils.createMosaicInternal(Utils.java:576)
at org.geotools.gce.imagemosaic.Utils.createMosaic(Utils.java:525)
at org.geotools.gce.imagemosaic.Utils.checkSource(Utils.java:1656)
at org.geotools.gce.imagemosaic.ImageMosaicReader.initReaderFromURL(ImageMosaicReader.java:264)
at org.geotools.gce.imagemosaic.ImageMosaicReader.(ImageMosaicReader.java:217)
at org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:563)
at org.geotools.gce.imagemosaic.ImageMosaicFormat.getReader(ImageMosaicFormat.java:104)
at org.geoserver.catalog.ResourcePool.getGridCoverageReader(ResourcePool.java:1593)

···

From: Jody Garnett <jody.garnett@…84…>
Sent: Wednesday, April 26, 2023 11:56 PM
To: Mort, Ashley C RTX <Ashley.C.Mort@…11756…>
Cc: geoserver-users@lists.sourceforge.net
Subject: Re: [External] Re: [Geoserver-users] Error while creating store with raster data -image mosaic

Hey Ashley:

Did you get it working with only a couple of granules and no PostGIS? From your description it sounded like you skipped straight to using a database?

  1. It does not appear anyone else has replied to your email saying “I have that problem also”. We could try clearly asking if anyone is using PostGIS + image mosaic in GeoServer 2.22.x

  2. You should check if the issue is already reported, and if not please report the issue with the information you have managed to collect thus far.

  3. If resolving this matter is time sensitive you may wish to seek help from some of the commercial support providers.

I am not greatly familiar with the image mosaic code myself. As a developer I would step through the code in the older release to see how it is intended to work. And then perform the same steps in the newer release.

Jody

On Wed, Apr 26, 2023 at 6:45 PM Mort, Ashley C RTX <Ashley.C.Mort@…11756…> wrote:

Is there anything else I can do to help investigate this issue?

Our simplest test is:

  1. Put 1 granule (we use 1 jpg and 1 jgw world file) in a folder (data_dir/myFolder) w/ a datastore.properties
    [datastore.properties]
    StoreName=cite:ImageMosaicPostGIS

cite:ImageMosaicPostGIS is a PostGIS datastore points to a local vanilla Postgres (15) that has postgis installed.

  1. Manually create an ImageMosaic Store pointing to data_dir/myFolder using the GeoServer web interface.

GeoServer versions prior to 2.22 will create the table in Postgres and create a .properties file for the Store then send you to the Edit Layer page. GeoServer 2.22 will stacktrace a variety of problems w/ GeoTools developer logging. I tried remote Java debugging GeoTools 28.2 with GeoServer 2.22.2 in intellij but I end up with a variety of stack traces that aren’t explicitly helpful and stepping through was a bit confusing. Creating the Image Mosaic Store without the datastore.properties (and using the Shapefile indexing) works fine.

Is there anything else I can do to document the problem? I could make a videocast or something but idk if that’s helpful at all.

Thanks again!

From: Jody Garnett <jody.garnett@…84…>
Sent: Saturday, April 22, 2023 3:14 AM
To: Mort, Ashley C RTX <Ashley.C.Mort@…11756…>
Cc: geoserver-users@lists.sourceforge.net
Subject: [External] Re: [Geoserver-users] Error while creating store with raster data -image mosaic

Thanks for the detail, the information in screen snap (and the information when a raster cannot be accessed) is not always useful for troubleshooting.

I assume you updated to Java 11, is the com.sun.imageio.plugins.jpeg.JPEGImageReaderSpi still available in Java 11?

What happens if you take a few granules and place them in a folder to generate a new image mosaic? You could compare a newly generated datastore.properties to the one you have been using.

Jody

On Fri, Apr 21, 2023 at 8:41 PM Mort, Ashley C RTX via Geoserver-users <geoserver-users@lists.sourceforge.net> wrote:

I wanted to provide a little more detail on this question.

We’re working with an ImageMosaic Store that we’ve used successfully with every major GeoServer version from 2.17 - 2.21 but we are having the problems Ashish mentioned on GeoServer 2.22 or 2.23. We configure our Store to index the granules in Postgres using the ImageMosaic configuration files.

[datastore.properties]

StoreName=cite:ImageMosaicPostGIS

[BrowseImageMosaicStore.properties]

ExpandToRGB=false

TypeName=BrowseImageMosaicStore

SuggestedFormat=org.geotools.gce.image.WorldImageFormat

Name=BrowseImageMosaicStore

SuggestedSPI=com.sun.imageio.plugins.jpeg.JPEGImageReaderSpi

LevelsNum=1

PathType=ABSOLUTE

Heterogeneous=false

Caching=false

HeterogeneousCRS=false

LocationAttribute=location

Levels=1.2793554128E-4,1.0810019425E-4

CheckAuxiliaryMetadata=false

MosaicCRS=EPSG:404000

[indexer.properties]

Caching=false

AbsolutePath=true

Schema=*the_geom:Polygon,location:String

PropertyCollectors=

From: Jonchhen, Ashish RTX via Geoserver-users <geoserver-users@lists.sourceforge.net>
Sent: Friday, April 21, 2023 3:19 PM
To: geoserver-users@lists.sourceforge.net
Subject: [External] [Geoserver-users] Error while creating store with raster data -image mosaic

Hi,

So we are trying to switch to Geoserver 2.22.2 version, but running into following problem while trying to create a store with image mosaic. I tried to test on the 2.21 version, it works on that with our properties. However, on 2.22 and above, it’s the same error I am seeing. Attached are the logs and a screenshot of the UI. Please let me know if you have solution for this. I have seen online that users are having some error while upgrading to the 2.22 and above. Please let me know if you have any question for me. Thank you

Ashish Jonchhen

Software Engineer I

C : +1 620.757.3966

ashish.jonchhen@…11756…

He/Him/His

Raytheon Intelligence & Space

1717 E. Cityline Drive

Richardson, TX 75082

RTX.com | LinkedIn | Twitter | Instagram


Geoserver-users mailing list

Please make sure you read the following two resources before posting to this list:

If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer

Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

Jody Garnett

Jody Garnett