[Geoserver-users] Add data (granule) to an imagemosaic+time

Jean_Pommier · September 11, 2014, 7:44am

Hi list,

I’ve got an imagemosaic constituted of several NDVI geotiffs (using time dimension).
It works very well, but I’m stuck at the “add new data” step.
I’ve tried several ways : curl (http://docs.geoserver.org/2.5.x/en/user/rest/examples/curl.html), gsconfig.
Digging in the REST API, I guess it should have been
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @dv2011.zip "http://pigeo.fr/geoserver-prod/rest/workspaces/pigeo/coveragestores/NDVI/file.imagemosaic
(the zip file containing my new geotiff file)
Actually, it did add the data in the repository, but didn’t update the index. So it’s not really added, just copied in the folder.

I’ve even tried with several versions of geoserver (2.4.2, 2.5.2, 2.6.RC1) with no changes. So I guess I’m mistaken somewhere in my process.
My data are geotiffs, all the same. I’m running geoserver under tomcat7, java 7 oracle.
The mosaic index is stored in a postgis DB.
Any help, please ? Do I need to install some extension ?
Have a nice day,

Jean

···

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

ChristinaPSU1 · September 11, 2014, 9:33am

Hi Jean,

if I understand you correctly, you have an existing time enabled image mosaic and want to add new GeoTIFFs to it, yes?

As far as I am aware the Geoserver ImageMosaic module does not provide a mechanism to update the granule index when new granules are added to the ImageMosaic source directory. Instead you need to add the granules manually to your granule index in your PostGIS table. You can do this automated through a Script for example. This script would need to extract the respective parameters from the new granules (time dimension from filename, geographical extent from GeoTIFF, path) and write it to a new row in your granule index table (if the extent is the same for all of your granules you could also simply copy over the geometry from a previous granule).

Alternatively you would have to recreate the ImageMosaic every time you add new granules to it (which would require deleting the granule index table and deleting the supplementary files that GeoServer creates on ImageMosaic creation before recreating the ImageMosaic).

Regards,

Max

···

Von: Jean Pommier jean.pommier@anonymised.com
Gesendet: Donnerstag, 11. September 2014 08:44
An: geoserver-users@lists.sourceforge.net
Betreff: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi list,

I’ve got an imagemosaic constituted of several NDVI geotiffs (using time dimension).
It works very well, but I’m stuck at the “add new data” step.
I’ve tried several ways : curl (http://docs.geoserver.org/2.5.x/en/user/rest/examples/curl.html), gsconfig.
Digging in the REST API, I guess it should have been
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @dv2011.zip "http://pigeo.fr/geoserver-prod/rest/workspaces/pigeo/coveragestores/NDVI/file.imagemosaic
(the zip file containing my new geotiff file)
Actually, it did add the data in the repository, but didn’t update the index. So it’s not really added, just copied in the folder.

I’ve even tried with several versions of geoserver (2.4.2, 2.5.2, 2.6.RC1) with no changes. So I guess I’m mistaken somewhere in my process.
My data are geotiffs, all the same. I’m running geoserver under tomcat7, java 7 oracle.
The mosaic index is stored in a postgis DB.
Any help, please ? Do I need to install some extension ?
Have a nice day,

Jean

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

Jean_Pommier · September 11, 2014, 9:57am

Hi Max,

Yes, that’s what I meant.
Are you certain about this ‘manually edit the index’ stuff ?
I don’t get it : using REST DELETE command, you can remove a granule, and it is removed from the index too.
But when you add a granule, it is not inserted in the index ? Isn’t it the point of adding a granule to the mosaic ?
Seems weird, doesn’t it ?
Thanks,

Jean

···

Le 11/09/2014 11:33, Max Stephan a écrit :

Hi Jean,

if I understand you correctly, you have an existing time enabled image mosaic and want to add new GeoTIFFs to it, yes?

As far as I am aware the Geoserver ImageMosaic module does not provide a mechanism to update the granule index when new granules are added to the ImageMosaic source directory. Instead you need to add the granules manually to your granule index in your PostGIS table. You can do this automated through a Script for example. This script would need to extract the respective parameters from the new granules (time dimension from filename, geographical extent from GeoTIFF, path) and write it to a new row in your granule index table (if the extent is the same for all of your granules you could also simply copy over the geometry from a previous granule).

Alternatively you would have to recreate the ImageMosaic every time you add new granules to it (which would require deleting the granule index table and deleting the supplementary files that GeoServer creates on ImageMosaic creation before recreating the ImageMosaic).

Regards,

Max

Von: Jean Pommier jean.pommier@anonymised.com
Gesendet: Donnerstag, 11. September 2014 08:44
An: geoserver-users@anonymised.comet
Betreff: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi list,

I’ve got an imagemosaic constituted of several NDVI geotiffs (using time dimension).
It works very well, but I’m stuck at the “add new data” step.
I’ve tried several ways : curl (http://docs.geoserver.org/2.5.x/en/user/rest/examples/curl.html), gsconfig.
Digging in the REST API, I guess it should have been
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @dv2011.zip "http://pigeo.fr/geoserver-prod/rest/workspaces/pigeo/coveragestores/NDVI/file.imagemosaic
(the zip file containing my new geotiff file)
Actually, it did add the data in the repository, but didn’t update the index. So it’s not really added, just copied in the folder.

I’ve even tried with several versions of geoserver (2.4.2, 2.5.2, 2.6.RC1) with no changes. So I guess I’m mistaken somewhere in my process.
My data are geotiffs, all the same. I’m running geoserver under tomcat7, java 7 oracle.
The mosaic index is stored in a postgis DB.
Any help, please ? Do I need to install some extension ?
Have a nice day,

Jean

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

Stephen_Brooke · September 11, 2014, 4:30pm

Hi Jean,

I have this working on GeoServer 2.5.1. Here’s what I do to add a granule to my ImageMosaic+time and with a custom domain attribute:

Manually copy the granule, eg Granule_005_20140902_20.tif, to folder file:///D:/Data/Imagery which is where my ImageMosaic store points to
Make an HTTP POST to GeoServer REST API as follows (I’m not using Curl so you will need to adapt my solution but this should hopefully give you the key to getting it to work):

a. Method: POST

b. URL: http://localhost:/geoserver/rest/workspaces//coveragestores//external.imagemosaic?recalculate=nativebbox,latlonbbox

c. Body (plain/text format):

file:///D:/Data/imagery/Granule_005_20140902_20.tif

d. After I make this request the ImageMosaic spatial index is updated in my Postgresql/PostGIS database

This technique is captured in the GeoServer REST API documentation at:

http://docs.geoserver.org/stable/en/user/rest/api/coveragestores.html

“If the coverage store is a simple one (e.g. GeoTiff) it will return a 405, if the coverage store is a structured one (e.g., mosaic) it will harvest the specified files into it, which in turn will integrate the files into the store. Harvest meaning is store dependent, for mosaic the new files will be added as new granules of the mosaic, and existing files will get their attribute updated, other stores might have a different behavior.”

NOTE: The granule does not need to be in the folder that the ImageMosaic points to. In the HTTP request body you can specify a different location and the granule is just reference from that location in the spatial index relative to the folder pointed to by the ImageMosaic.

I would also be interested to hear if anybody has been able to add granules to an ImageMosaic by uploading them through the REST API, I was unable to get this to work

.

Hope that helps,

–Steve

···

From: Jean Pommier [mailto:jean.pommier@anonymised.com]
Sent: Thursday, September 11, 2014 2:58 AM
To: Max Stephan; geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi Max,

Yes, that’s what I meant.
Are you certain about this ‘manually edit the index’ stuff ?
I don’t get it : using REST DELETE command, you can remove a granule, and it is removed from the index too.
But when you add a granule, it is not inserted in the index ? Isn’t it the point of adding a granule to the mosaic ?
Seems weird, doesn’t it ?
Thanks,

Jean

Le 11/09/2014 11:33, Max Stephan a écrit :

Hi Jean,

if I understand you correctly, you have an existing time enabled image mosaic and want to add new GeoTIFFs to it, yes?

As far as I am aware the Geoserver ImageMosaic module does not provide a mechanism to update the granule index when new granules are added to the ImageMosaic source directory. Instead you need to add the granules manually to your granule index in your PostGIS table. You can do this automated through a Script for example. This script would need to extract the respective parameters from the new granules (time dimension from filename, geographical extent from GeoTIFF, path) and write it to a new row in your granule index table (if the extent is the same for all of your granules you could also simply copy over the geometry from a previous granule).

Alternatively you would have to recreate the ImageMosaic every time you add new granules to it (which would require deleting the granule index table and deleting the supplementary files that GeoServer creates on ImageMosaic creation before recreating the ImageMosaic).

Regards,

Max

Von: Jean Pommier jean.pommier@anonymised.com
Gesendet: Donnerstag, 11. September 2014 08:44
An: geoserver-users@lists.sourceforge.net
Betreff: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi list,

I’ve got an imagemosaic constituted of several NDVI geotiffs (using time dimension).
It works very well, but I’m stuck at the “add new data” step.
I’ve tried several ways : curl (http://docs.geoserver.org/2.5.x/en/user/rest/examples/curl.html), gsconfig.
Digging in the REST API, I guess it should have been
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @dv2011.zip "http://pigeo.fr/geoserver-prod/rest/workspaces/pigeo/coveragestores/NDVI/file.imagemosaic
(the zip file containing my new geotiff file)
Actually, it did add the data in the repository, but didn’t update the index. So it’s not really added, just copied in the folder.

I’ve even tried with several versions of geoserver (2.4.2, 2.5.2, 2.6.RC1) with no changes. So I guess I’m mistaken somewhere in my process.
My data are geotiffs, all the same. I’m running geoserver under tomcat7, java 7 oracle.
The mosaic index is stored in a postgis DB.
Any help, please ? Do I need to install some extension ?
Have a nice day,

Jean

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

Jean_Pommier · September 11, 2014, 6:58pm

Hi Steve,

Thanks for your answer, it gives me hope.
My issue is complementary with yours :

I manage to upload the granule using HTTP PUT. In cURL, it would read:
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @20130101_DV.tif.zip “http://localhost:8081/geoserver26RC1/rest/workspaces/pigeo/coveragestores/ndvi/file.imagemosaic”
Note that you have to put your files into a zip archive (saw it in the docs)
It does copy the file in my imagemosaic folder.

you would have expected it would update the index, but it doesn’t

It did try the same request as you outlined, but in cURL. It’s not much different, and indeed documented. But on my instance, it doesn’t do anything at all. I do get a 202, no error, but nothing happens
I’ll try tomorrow with other sets of data, maybe it my geotiff files having something weird inside (but the initial image mosaic generation is fine, so I would say the files are fine)

If you try the PUT request (upload), I’d be interested to know what it does on your instance. Will it recompute the index on the fly ?
Next episode tomorrow…

Jean

···

Le 11/09/2014 18:30, Stephen Brooke a écrit :

Hi Jean,

I have this working on GeoServer 2.5.1. Here’s what I do to add a granule to my ImageMosaic+time and with a custom domain attribute:

Manually copy the granule, eg Granule_005_20140902_20.tif, to folder file:///D:/Data/Imagery which is where my ImageMosaic store points to

Make an HTTP POST to GeoServer REST API as follows (I’m not using Curl so you will need to adapt my solution but this should hopefully give you the key to getting it to work):

a. Method: POST

b. URL: http://localhost:/geoserver/rest/workspaces//coveragestores//external.imagemosaic?recalculate=nativebbox,latlonbbox

c. Body (plain/text format):

file:///D:/Data/imagery/Granule_005_20140902_20.tif

d. After I make this request the ImageMosaic spatial index is updated in my Postgresql/PostGIS database

This technique is captured in the GeoServer REST API documentation at:

http://docs.geoserver.org/stable/en/user/rest/api/coveragestores.html

“If the coverage store is a simple one (e.g. GeoTiff) it will return a 405, if the coverage store is a structured one (e.g., mosaic) it will harvest the specified files into it, which in turn will integrate the files into the store. Harvest meaning is store dependent, for mosaic the new files will be added as new granules of the mosaic, and existing files will get their attribute updated, other stores might have a different behavior.”

NOTE: The granule does not need to be in the folder that the ImageMosaic points to. In the HTTP request body you can specify a different location and the granule is just reference from that location in the spatial index relative to the folder pointed to by the ImageMosaic.

I would also be interested to hear if anybody has been able to add granules to an ImageMosaic by uploading them through the REST API, I was unable to get this to work

.

Hope that helps,

–Steve

From: Jean Pommier [mailto:jean.pommier@anonymised.com]
Sent: Thursday, September 11, 2014 2:58 AM
To: Max Stephan; geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi Max,

Yes, that’s what I meant.
Are you certain about this ‘manually edit the index’ stuff ?
I don’t get it : using REST DELETE command, you can remove a granule, and it is removed from the index too.
But when you add a granule, it is not inserted in the index ? Isn’t it the point of adding a granule to the mosaic ?
Seems weird, doesn’t it ?
Thanks,

Jean

Le 11/09/2014 11:33, Max Stephan a écrit :

Hi Jean,

if I understand you correctly, you have an existing time enabled image mosaic and want to add new GeoTIFFs to it, yes?

As far as I am aware the Geoserver ImageMosaic module does not provide a mechanism to update the granule index when new granules are added to the ImageMosaic source directory. Instead you need to add the granules manually to your granule index in your PostGIS table. You can do this automated through a Script for example. This script would need to extract the respective parameters from the new granules (time dimension from filename, geographical extent from GeoTIFF, path) and write it to a new row in your granule index table (if the extent is the same for all of your granules you could also simply copy over the geometry from a previous granule).

Alternatively you would have to recreate the ImageMosaic every time you add new granules to it (which would require deleting the granule index table and deleting the supplementary files that GeoServer creates on ImageMosaic creation before recreating the ImageMosaic).

Regards,

Max

Von: Jean Pommier jean.pommier@anonymised.com
Gesendet: Donnerstag, 11. September 2014 08:44
An: geoserver-users@lists.sourceforge.net
Betreff: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi list,

I’ve got an imagemosaic constituted of several NDVI geotiffs (using time dimension).
It works very well, but I’m stuck at the “add new data” step.
I’ve tried several ways : curl (http://docs.geoserver.org/2.5.x/en/user/rest/examples/curl.html), gsconfig.
Digging in the REST API, I guess it should have been
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @dv2011.zip "http://pigeo.fr/geoserver-prod/rest/workspaces/pigeo/coveragestores/NDVI/file.imagemosaic
(the zip file containing my new geotiff file)
Actually, it did add the data in the repository, but didn’t update the index. So it’s not really added, just copied in the folder.

I’ve even tried with several versions of geoserver (2.4.2, 2.5.2, 2.6.RC1) with no changes. So I guess I’m mistaken somewhere in my process.
My data are geotiffs, all the same. I’m running geoserver under tomcat7, java 7 oracle.
The mosaic index is stored in a postgis DB.
Any help, please ? Do I need to install some extension ?
Have a nice day,

Jean

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

Stephen_Brooke · September 11, 2014, 8:32pm

Hi Jean,

I tried adding a granule the way you did it with “PUT” and Curl and it replaced my ImageMosaic completely (which is what a PUT request is supposed to do). After deleting and recreating the ImageMosaic I tried “POST” instead and it worked in the sense that it updated my spatial index for my existing ImageMosaic.

So I used your URL with the following alterations:

Used “POST” HTTP method instead of “PUT”
Added query parameter to the end of the URL: ?recalculate=nativebbox,latlonbbox

I think I know what is happening in your case. Instead of adding a granule you are effectively replacing the ImageMosaic with the contents of the zip file that you provide and the ImageMosaic takes that zip file’s base folder as its new working folder, that is, where the indexer, datastore, and regex property files are expected to be. So try “POST” instead and it should work for you too.

Hope that solves it for you and thanks, you helped me solve my problem too!

–Steve

···

From: Jean Pommier [mailto:jean.pommier@anonymised.com]
Sent: Thursday, September 11, 2014 11:59 AM
To: Stephen Brooke; Max Stephan; geoserver-users@anonymised.comet
Subject: Re: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi Steve,

Thanks for your answer, it gives me hope.
My issue is complementary with yours :

I manage to upload the granule using HTTP PUT. In cURL, it would read:
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @20130101_DV.tif.zip “http://localhost:8081/geoserver26RC1/rest/workspaces/pigeo/coveragestores/ndvi/file.imagemosaic”
Note that you have to put your files into a zip archive (saw it in the docs)
It does copy the file in my imagemosaic folder.

you would have expected it would update the index, but it doesn’t

It did try the same request as you outlined, but in cURL. It’s not much different, and indeed documented. But on my instance, it doesn’t do anything at all. I do get a 202, no error, but nothing happens
I’ll try tomorrow with other sets of data, maybe it my geotiff files having something weird inside (but the initial image mosaic generation is fine, so I would say the files are fine)

If you try the PUT request (upload), I’d be interested to know what it does on your instance. Will it recompute the index on the fly ?
Next episode tomorrow…

Jean

Le 11/09/2014 18:30, Stephen Brooke a écrit :

Hi Jean,

I have this working on GeoServer 2.5.1. Here’s what I do to add a granule to my ImageMosaic+time and with a custom domain attribute:

Manually copy the granule, eg Granule_005_20140902_20.tif, to folder file:///D:/Data/Imagery which is where my ImageMosaic store points to

Make an HTTP POST to GeoServer REST API as follows (I’m not using Curl so you will need to adapt my solution but this should hopefully give you the key to getting it to work):

a. Method: POST

b. URL: http://localhost:/geoserver/rest/workspaces//coveragestores//external.imagemosaic?recalculate=nativebbox,latlonbbox

c. Body (plain/text format):

file:///D:/Data/imagery/Granule_005_20140902_20.tif

d. After I make this request the ImageMosaic spatial index is updated in my Postgresql/PostGIS database

This technique is captured in the GeoServer REST API documentation at:

http://docs.geoserver.org/stable/en/user/rest/api/coveragestores.html

“If the coverage store is a simple one (e.g. GeoTiff) it will return a 405, if the coverage store is a structured one (e.g., mosaic) it will harvest the specified files into it, which in turn will integrate the files into the store. Harvest meaning is store dependent, for mosaic the new files will be added as new granules of the mosaic, and existing files will get their attribute updated, other stores might have a different behavior.”

NOTE: The granule does not need to be in the folder that the ImageMosaic points to. In the HTTP request body you can specify a different location and the granule is just reference from that location in the spatial index relative to the folder pointed to by the ImageMosaic.

I would also be interested to hear if anybody has been able to add granules to an ImageMosaic by uploading them through the REST API, I was unable to get this to work

.

Hope that helps,

–Steve

From: Jean Pommier [mailto:jean.pommier@anonymised.com]
Sent: Thursday, September 11, 2014 2:58 AM
To: Max Stephan; geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi Max,

Yes, that’s what I meant.
Are you certain about this ‘manually edit the index’ stuff ?
I don’t get it : using REST DELETE command, you can remove a granule, and it is removed from the index too.
But when you add a granule, it is not inserted in the index ? Isn’t it the point of adding a granule to the mosaic ?
Seems weird, doesn’t it ?
Thanks,

Jean

Le 11/09/2014 11:33, Max Stephan a écrit :

Hi Jean,

if I understand you correctly, you have an existing time enabled image mosaic and want to add new GeoTIFFs to it, yes?

As far as I am aware the Geoserver ImageMosaic module does not provide a mechanism to update the granule index when new granules are added to the ImageMosaic source directory. Instead you need to add the granules manually to your granule index in your PostGIS table. You can do this automated through a Script for example. This script would need to extract the respective parameters from the new granules (time dimension from filename, geographical extent from GeoTIFF, path) and write it to a new row in your granule index table (if the extent is the same for all of your granules you could also simply copy over the geometry from a previous granule).

Alternatively you would have to recreate the ImageMosaic every time you add new granules to it (which would require deleting the granule index table and deleting the supplementary files that GeoServer creates on ImageMosaic creation before recreating the ImageMosaic).

Regards,

Max

Von: Jean Pommier jean.pommier@anonymised.com
Gesendet: Donnerstag, 11. September 2014 08:44
An: geoserver-users@lists.sourceforge.net
Betreff: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi list,

I’ve got an imagemosaic constituted of several NDVI geotiffs (using time dimension).
It works very well, but I’m stuck at the “add new data” step.
I’ve tried several ways : curl (http://docs.geoserver.org/2.5.x/en/user/rest/examples/curl.html), gsconfig.
Digging in the REST API, I guess it should have been
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @dv2011.zip "http://pigeo.fr/geoserver-prod/rest/workspaces/pigeo/coveragestores/NDVI/file.imagemosaic
(the zip file containing my new geotiff file)
Actually, it did add the data in the repository, but didn’t update the index. So it’s not really added, just copied in the folder.

I’ve even tried with several versions of geoserver (2.4.2, 2.5.2, 2.6.RC1) with no changes. So I guess I’m mistaken somewhere in my process.
My data are geotiffs, all the same. I’m running geoserver under tomcat7, java 7 oracle.
The mosaic index is stored in a postgis DB.
Any help, please ? Do I need to install some extension ?
Have a nice day,

Jean

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

Jean_Pommier · September 11, 2014, 9:44pm

Hi again,

Nope, apparently that’s not it…
Good try, however

···

Le 11/09/2014 22:32, Stephen Brooke a écrit :

Hi Jean,

I tried adding a granule the way you did it with “PUT” and Curl and it replaced my ImageMosaic completely (which is what a PUT request is supposed to do). After deleting and recreating the ImageMosaic I tried “POST” instead and it worked in the sense that it updated my spatial index for my existing ImageMosaic.

So I used your URL with the following alterations:

Used “POST” HTTP method instead of “PUT”

Added query parameter to the end of the URL: ?recalculate=nativebbox,latlonbbox

I think I know what is happening in your case. Instead of adding a granule you are effectively replacing the ImageMosaic with the contents of the zip file that you provide and the ImageMosaic takes that zip file’s base folder as its new working folder, that is, where the indexer, datastore, and regex property files are expected to be. So try “POST” instead and it should work for you too.

Hope that solves it for you and thanks, you helped me solve my problem too!

–Steve

From: Jean Pommier [mailto:jean.pommier@anonymised.com]
Sent: Thursday, September 11, 2014 11:59 AM
To: Stephen Brooke; Max Stephan; geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi Steve,

Thanks for your answer, it gives me hope.
My issue is complementary with yours :

I manage to upload the granule using HTTP PUT. In cURL, it would read:
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @20130101_DV.tif.zip “http://localhost:8081/geoserver26RC1/rest/workspaces/pigeo/coveragestores/ndvi/file.imagemosaic”
Note that you have to put your files into a zip archive (saw it in the docs)
It does copy the file in my imagemosaic folder.

you would have expected it would update the index, but it doesn’t

It did try the same request as you outlined, but in cURL. It’s not much different, and indeed documented. But on my instance, it doesn’t do anything at all. I do get a 202, no error, but nothing happens
I’ll try tomorrow with other sets of data, maybe it my geotiff files having something weird inside (but the initial image mosaic generation is fine, so I would say the files are fine)

If you try the PUT request (upload), I’d be interested to know what it does on your instance. Will it recompute the index on the fly ?
Next episode tomorrow…

Jean

Le 11/09/2014 18:30, Stephen Brooke a écrit :

Hi Jean,

I have this working on GeoServer 2.5.1. Here’s what I do to add a granule to my ImageMosaic+time and with a custom domain attribute:

Manually copy the granule, eg Granule_005_20140902_20.tif, to folder file:///D:/Data/Imagery which is where my ImageMosaic store points to

Make an HTTP POST to GeoServer REST API as follows (I’m not using Curl so you will need to adapt my solution but this should hopefully give you the key to getting it to work):

a. Method: POST

b. URL: http://localhost:/geoserver/rest/workspaces//coveragestores//external.imagemosaic?recalculate=nativebbox,latlonbbox

c. Body (plain/text format):

file:///D:/Data/imagery/Granule_005_20140902_20.tif

d. After I make this request the ImageMosaic spatial index is updated in my Postgresql/PostGIS database

This technique is captured in the GeoServer REST API documentation at:

http://docs.geoserver.org/stable/en/user/rest/api/coveragestores.html

“If the coverage store is a simple one (e.g. GeoTiff) it will return a 405, if the coverage store is a structured one (e.g., mosaic) it will harvest the specified files into it, which in turn will integrate the files into the store. Harvest meaning is store dependent, for mosaic the new files will be added as new granules of the mosaic, and existing files will get their attribute updated, other stores might have a different behavior.”

NOTE: The granule does not need to be in the folder that the ImageMosaic points to. In the HTTP request body you can specify a different location and the granule is just reference from that location in the spatial index relative to the folder pointed to by the ImageMosaic.

I would also be interested to hear if anybody has been able to add granules to an ImageMosaic by uploading them through the REST API, I was unable to get this to work

.

Hope that helps,

–Steve

From: Jean Pommier [mailto:jean.pommier@anonymised.com]
Sent: Thursday, September 11, 2014 2:58 AM
To: Max Stephan; geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi Max,

Yes, that’s what I meant.
Are you certain about this ‘manually edit the index’ stuff ?
I don’t get it : using REST DELETE command, you can remove a granule, and it is removed from the index too.
But when you add a granule, it is not inserted in the index ? Isn’t it the point of adding a granule to the mosaic ?
Seems weird, doesn’t it ?
Thanks,

Jean

Le 11/09/2014 11:33, Max Stephan a écrit :

Hi Jean,

if I understand you correctly, you have an existing time enabled image mosaic and want to add new GeoTIFFs to it, yes?

As far as I am aware the Geoserver ImageMosaic module does not provide a mechanism to update the granule index when new granules are added to the ImageMosaic source directory. Instead you need to add the granules manually to your granule index in your PostGIS table. You can do this automated through a Script for example. This script would need to extract the respective parameters from the new granules (time dimension from filename, geographical extent from GeoTIFF, path) and write it to a new row in your granule index table (if the extent is the same for all of your granules you could also simply copy over the geometry from a previous granule).

Alternatively you would have to recreate the ImageMosaic every time you add new granules to it (which would require deleting the granule index table and deleting the supplementary files that GeoServer creates on ImageMosaic creation before recreating the ImageMosaic).

Regards,

Max

Von: Jean Pommier jean.pommier@anonymised.com
Gesendet: Donnerstag, 11. September 2014 08:44
An: geoserver-users@lists.sourceforge.net
Betreff: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi list,

I’ve got an imagemosaic constituted of several NDVI geotiffs (using time dimension).
It works very well, but I’m stuck at the “add new data” step.
I’ve tried several ways : curl (http://docs.geoserver.org/2.5.x/en/user/rest/examples/curl.html), gsconfig.
Digging in the REST API, I guess it should have been
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @dv2011.zip "http://pigeo.fr/geoserver-prod/rest/workspaces/pigeo/coveragestores/NDVI/file.imagemosaic
(the zip file containing my new geotiff file)
Actually, it did add the data in the repository, but didn’t update the index. So it’s not really added, just copied in the folder.

I’ve even tried with several versions of geoserver (2.4.2, 2.5.2, 2.6.RC1) with no changes. So I guess I’m mistaken somewhere in my process.
My data are geotiffs, all the same. I’m running geoserver under tomcat7, java 7 oracle.
The mosaic index is stored in a postgis DB.
Any help, please ? Do I need to install some extension ?
Have a nice day,

Jean

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

Jean_Pommier · September 11, 2014, 9:48pm

Mmmh, I switched the log level to Geotools Developer Logging level, and I get somehting weird matching with my external.imagemosaic?recalculate=nativebbox,latlonbbox request. I guess I’ll have to ask to the devs…
It says :

2014-09-11 21:43:09,723 DEBUG [org.geotools.gce.arcgrid] - ArcGridFormatFactory is availaible.
2014-09-11 21:43:09,723 DEBUG [org.geotools.gce.arcgrid] - Creating a new ArcGriFormat.
2014-09-11 21:43:09,724 INFO [org.geoserver.catalog.rest] - PUT file, mimetype: text/plain
2014-09-11 21:43:09,728 DEBUG [org.geotools.util] - CRSConverterFactory can be applied from Strings to CRS only.
2014-09-11 21:43:09,728 DEBUG [org.geotools.util] - InterpolationConverterFactory can be applied from Strings to Interpolation only.
2014-09-11 21:43:09,728 DEBUG [org.geotools.util] - CRSConverterFactory can be applied from Strings to CRS only.
2014-09-11 21:43:09,728 DEBUG [org.geotools.util] - InterpolationConverterFactory can be applied from Strings to Interpolation only.
2014-09-11 21:43:09,742 DEBUG [org.geotools.jdbc] - CLOSE CONNECTION
2014-09-11 21:43:09,745 INFO [org.geotools.gce.imagemosaic] - Now indexing file 20130101_DV.tif
2014-09-11 21:43:09,748 DEBUG [org.geotools.gce.arcgrid] - ArcGridFormatFactory is availaible.
2014-09-11 21:43:09,750 TRACE [org.geotools.factory] - ENTRY (GridCoverageFactory)
2014-09-11 21:43:09,750 TRACE [org.geotools.factory] - RETURN (GridCoverageFactory): found implementation GridCoverageFactory.
2014-09-11 21:43:09,751 DEBUG [org.geotools.gce.gtopo30] - Unrecognized file (file extension doesn’t match)
java.io.IOException: Unrecognized file (file extension doesn’t match)
at org.geotools.gce.gtopo30.GTopo30Reader.(GTopo30Reader.java:231)
at org.geotools.gce.gtopo30.GTopo30Reader.(GTopo30Reader.java:161)
at org.geotools.gce.gtopo30.GTopo30Format.accepts(GTopo30Format.java:177)
at org.geotools.coverage.grid.io.GridFormatFinder.findFormats(GridFormatFinder.java:188)
at org.geotools.coverage.grid.io.GridFormatFinder.findFormat(GridFormatFinder.java:241)
at org.geotools.gce.imagemosaic.ImageMosaicWalker.handleFile(ImageMosaicWalker.java:148)
at org.geotools.gce.imagemosaic.ImageMosaicDirectoryWalker$MosaicDirectoryWalker.handleFile(ImageMosaicDirectoryWalker.java:98)
at org.apache.commons.io.DirectoryWalker.walk(DirectoryWalker.java:367)
at org.apache.commons.io.DirectoryWalker.walk(DirectoryWalker.java:335)
at org.geotools.gce.imagemosaic.ImageMosaicDirectoryWalker$MosaicDirectoryWalker.(ImageMosaicDirectoryWalker.java:114)
at org.geotools.gce.imagemosaic.ImageMosaicDirectoryWalker.run(ImageMosaicDirectoryWalker.java:196)
at org.geotools.gce.imagemosaic.ImageMosaicReader$HarvestedResource.harvestCalculation(ImageMosaicReader.java:405)
at org.geotools.gce.imagemosaic.ImageMosaicReader$HarvestedResource.access$100(ImageMosaicReader.java:187)
at org.geotools.gce.imagemosaic.ImageMosaicReader$HarvestedResource$1.harvest(ImageMosaicReader.java:202)
at org.geotools.gce.imagemosaic.ImageMosaicReader.harvest(ImageMosaicReader.java:1250)
at org.geoserver.catalog.CoverageDimensionCustomizerReader$CoverageDimensionCustomizerStructuredReader.harvest(CoverageDimensionCustomizerReader.java:121)
at org.geoserver.catalog.rest.CoverageStoreFileResource.handlePost(CoverageStoreFileResource.java:89)
at org.restlet.Finder.handle(Finder.java:296)
at org.geoserver.rest.BeanDelegatingRestlet.handle(BeanDelegatingRestlet.java:37)
at org.restlet.Filter.doHandle(Filter.java:105)
at org.restlet.Filter.handle(Filter.java:134)
at org.restlet.Router.handle(Router.java:444)
at org.geoserver.rest.RESTDispatcher$1.handle(RESTDispatcher.java:204)
at com.noelios.restlet.ext.servlet.ServletConverter.service(ServletConverter.java:129)
at org.geoserver.rest.RESTDispatcher.handleRequestInternal(RESTDispatcher.java:86)
at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.geoserver.filters.ThreadLocalsCleanupFilter.doFilter(ThreadLocalsCleanupFilter.java:27)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:74)
at org.geoserver.wms.animate.AnimatorFilter.doFilter(AnimatorFilter.java:70)
at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:70)
at org.geoserver.filters.SpringDelegatingFilter.doFilter(SpringDelegatingFilter.java:45)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.geoserver.platform.AdvancedDispatchFilter.doFilter(AdvancedDispatchFilter.java:49)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerAnonymousAuthenticationFilter.doFilter(GeoServerAnonymousAuthenticationFilter.java:53)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:201)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.geoserver.security.filter.GeoServerBasicAuthenticationFilter.doFilter(GeoServerBasicAuthenticationFilter.java:82)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:68)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.geoserver.security.filter.GeoServerSecurityContextPersistenceFilter$1.doFilter(GeoServerSecurityContextPersistenceFilter.java:52)
at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:72)
at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
at org.geoserver.security.GeoServerSecurityFilterChainProxy.doFilter(GeoServerSecurityFilterChainProxy.java:134)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:75)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:48)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.geoserver.filters.SessionDebugFilter.doFilter(SessionDebugFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.geoserver.filters.FlushSafeFilter.doFilter(FlushSafeFilter.java:43)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.vfny.geoserver.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:109)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-09-11 21:43:09,752 DEBUG [org.geotools.gce.arcgrid] - Creating a new ArcGriFormat.
2014-09-11 21:43:09,753 TRACE [org.geotools.factory] - ENTRY (GridCoverageFactory)
2014-09-11 21:43:09,753 TRACE [org.geotools.factory] - RETURN (GridCoverageFactory): found implementation GridCoverageFactory.
2014-09-11 21:43:09,755 TRACE [org.geotools.factory] - ENTRY (DatumFactory, DATUM_FACTORY)
2014-09-11 21:43:09,756 TRACE [org.geotools.factory] - RETURN (DatumFactory, DATUM_FACTORY): found implementation DatumAliases.
2014-09-11 21:43:09,756 TRACE [org.geotools.factory] - ENTRY (MathTransformFactory, MATH_TRANSFORM_FACTORY)
2014-09-11 21:43:09,756 TRACE [org.geotools.factory] - RETURN (MathTransformFactory, MATH_TRANSFORM_FACTORY): found implementation DefaultMathTransformFactory.
2014-09-11 21:43:09,757 TRACE [org.geotools.factory] - ENTRY (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY)
2014-09-11 21:43:09,757 TRACE [org.geotools.factory] - CHECK (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): user provided a Class.
2014-09-11 21:43:09,757 TRACE [org.geotools.factory] - CHECK (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): consider hint[last] AbstractEpsgMediator.
2014-09-11 21:43:09,757 TRACE [org.geotools.factory] - THROW (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): could not find implementation.
2014-09-11 21:43:09,758 TRACE [org.geotools.factory] - ENTRY (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY)
2014-09-11 21:43:09,758 TRACE [org.geotools.factory] - CHECK (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): user provided a Class.
2014-09-11 21:43:09,758 TRACE [org.geotools.factory] - CHECK (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): consider hint[last] AbstractEpsgMediator.
2014-09-11 21:43:09,758 TRACE [org.geotools.factory] - THROW (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): could not find implementation.
2014-09-11 21:43:09,759 TRACE [org.geotools.factory] - ENTRY (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY)
2014-09-11 21:43:09,759 TRACE [org.geotools.factory] - CHECK (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): user provided a Class.
2014-09-11 21:43:09,759 TRACE [org.geotools.factory] - CHECK (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): consider hint[last] AbstractEpsgMediator.
2014-09-11 21:43:09,759 TRACE [org.geotools.factory] - THROW (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): could not find implementation.
2014-09-11 21:43:09,760 TRACE [org.geotools.factory] - ENTRY (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY)
2014-09-11 21:43:09,760 TRACE [org.geotools.factory] - CHECK (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): user provided a Class.
2014-09-11 21:43:09,760 TRACE [org.geotools.factory] - CHECK (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): consider hint[last] AbstractEpsgMediator.
2014-09-11 21:43:09,760 TRACE [org.geotools.factory] - THROW (CRSAuthorityFactory, CRS_AUTHORITY_FACTORY): could not find implementation.
2014-09-11 21:43:09,761 DEBUG [org.geotools.referencing.factory] - Échec dans la fabrique primaire: Aucun code “EPSG:4326” de l’autorité “European Petroleum Survey Group” n’a été trouvé pour un objet de type “IdentifiedObject”. La fabrique secondaire sera interrogée.
2014-09-11 21:43:09,761 DEBUG [org.geotools.coverage.grid.io.imageio.geotiff] - Logging tiePoints:
Tie point
Raster Space point (0.0, 0.0, 0.0)
Model Space point (-12.242570219767783, 25.00004829999998, 0.0)
Logging pixScales:
Pixel Scale
scaleX=0.008928571400000005 is set? true
scaleY=0.008928571400000003 is set? true
scaleZ=1.0 is set? true

2014-09-11 21:43:09,762 DEBUG [org.geotools.gce.imagemosaic] - Nothing to process!!!

···

Le 11/09/2014 22:32, Stephen Brooke a écrit :

Hi Jean,

I tried adding a granule the way you did it with “PUT” and Curl and it replaced my ImageMosaic completely (which is what a PUT request is supposed to do). After deleting and recreating the ImageMosaic I tried “POST” instead and it worked in the sense that it updated my spatial index for my existing ImageMosaic.

So I used your URL with the following alterations:

Used “POST” HTTP method instead of “PUT”

Added query parameter to the end of the URL: ?recalculate=nativebbox,latlonbbox

I think I know what is happening in your case. Instead of adding a granule you are effectively replacing the ImageMosaic with the contents of the zip file that you provide and the ImageMosaic takes that zip file’s base folder as its new working folder, that is, where the indexer, datastore, and regex property files are expected to be. So try “POST” instead and it should work for you too.

Hope that solves it for you and thanks, you helped me solve my problem too!

–Steve

From: Jean Pommier [mailto:jean.pommier@anonymised.com]
Sent: Thursday, September 11, 2014 11:59 AM
To: Stephen Brooke; Max Stephan; geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi Steve,

Thanks for your answer, it gives me hope.
My issue is complementary with yours :

I manage to upload the granule using HTTP PUT. In cURL, it would read:
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @20130101_DV.tif.zip “http://localhost:8081/geoserver26RC1/rest/workspaces/pigeo/coveragestores/ndvi/file.imagemosaic”
Note that you have to put your files into a zip archive (saw it in the docs)
It does copy the file in my imagemosaic folder.

you would have expected it would update the index, but it doesn’t

It did try the same request as you outlined, but in cURL. It’s not much different, and indeed documented. But on my instance, it doesn’t do anything at all. I do get a 202, no error, but nothing happens
I’ll try tomorrow with other sets of data, maybe it my geotiff files having something weird inside (but the initial image mosaic generation is fine, so I would say the files are fine)

If you try the PUT request (upload), I’d be interested to know what it does on your instance. Will it recompute the index on the fly ?
Next episode tomorrow…

Jean

Le 11/09/2014 18:30, Stephen Brooke a écrit :

Hi Jean,

I have this working on GeoServer 2.5.1. Here’s what I do to add a granule to my ImageMosaic+time and with a custom domain attribute:

Manually copy the granule, eg Granule_005_20140902_20.tif, to folder file:///D:/Data/Imagery which is where my ImageMosaic store points to

Make an HTTP POST to GeoServer REST API as follows (I’m not using Curl so you will need to adapt my solution but this should hopefully give you the key to getting it to work):

a. Method: POST

b. URL: http://localhost:/geoserver/rest/workspaces//coveragestores//external.imagemosaic?recalculate=nativebbox,latlonbbox

c. Body (plain/text format):

file:///D:/Data/imagery/Granule_005_20140902_20.tif

d. After I make this request the ImageMosaic spatial index is updated in my Postgresql/PostGIS database

This technique is captured in the GeoServer REST API documentation at:

http://docs.geoserver.org/stable/en/user/rest/api/coveragestores.html

“If the coverage store is a simple one (e.g. GeoTiff) it will return a 405, if the coverage store is a structured one (e.g., mosaic) it will harvest the specified files into it, which in turn will integrate the files into the store. Harvest meaning is store dependent, for mosaic the new files will be added as new granules of the mosaic, and existing files will get their attribute updated, other stores might have a different behavior.”

NOTE: The granule does not need to be in the folder that the ImageMosaic points to. In the HTTP request body you can specify a different location and the granule is just reference from that location in the spatial index relative to the folder pointed to by the ImageMosaic.

I would also be interested to hear if anybody has been able to add granules to an ImageMosaic by uploading them through the REST API, I was unable to get this to work

.

Hope that helps,

–Steve

From: Jean Pommier [mailto:jean.pommier@anonymised.com]
Sent: Thursday, September 11, 2014 2:58 AM
To: Max Stephan; geoserver-users@lists.sourceforge.net
Subject: Re: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi Max,

Yes, that’s what I meant.
Are you certain about this ‘manually edit the index’ stuff ?
I don’t get it : using REST DELETE command, you can remove a granule, and it is removed from the index too.
But when you add a granule, it is not inserted in the index ? Isn’t it the point of adding a granule to the mosaic ?
Seems weird, doesn’t it ?
Thanks,

Jean

Le 11/09/2014 11:33, Max Stephan a écrit :

Hi Jean,

if I understand you correctly, you have an existing time enabled image mosaic and want to add new GeoTIFFs to it, yes?

As far as I am aware the Geoserver ImageMosaic module does not provide a mechanism to update the granule index when new granules are added to the ImageMosaic source directory. Instead you need to add the granules manually to your granule index in your PostGIS table. You can do this automated through a Script for example. This script would need to extract the respective parameters from the new granules (time dimension from filename, geographical extent from GeoTIFF, path) and write it to a new row in your granule index table (if the extent is the same for all of your granules you could also simply copy over the geometry from a previous granule).

Alternatively you would have to recreate the ImageMosaic every time you add new granules to it (which would require deleting the granule index table and deleting the supplementary files that GeoServer creates on ImageMosaic creation before recreating the ImageMosaic).

Regards,

Max

Von: Jean Pommier jean.pommier@anonymised.com
Gesendet: Donnerstag, 11. September 2014 08:44
An: geoserver-users@lists.sourceforge.net
Betreff: [Geoserver-users] Add data (granule) to an imagemosaic+time

Hi list,

I’ve got an imagemosaic constituted of several NDVI geotiffs (using time dimension).
It works very well, but I’m stuck at the “add new data” step.
I’ve tried several ways : curl (http://docs.geoserver.org/2.5.x/en/user/rest/examples/curl.html), gsconfig.
Digging in the REST API, I guess it should have been
curl -v -u admin:geoserver -XPUT -H “Content-type: application/zip” --data-binary @dv2011.zip "http://pigeo.fr/geoserver-prod/rest/workspaces/pigeo/coveragestores/NDVI/file.imagemosaic
(the zip file containing my new geotiff file)
Actually, it did add the data in the repository, but didn’t update the index. So it’s not really added, just copied in the folder.

I’ve even tried with several versions of geoserver (2.4.2, 2.5.2, 2.6.RC1) with no changes. So I guess I’m mistaken somewhere in my process.
My data are geotiffs, all the same. I’m running geoserver under tomcat7, java 7 oracle.
The mosaic index is stored in a postgis DB.
Any help, please ? Do I need to install some extension ?
Have a nice day,

Jean

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr

–

Jean Pommier – pi-Geosolutions

Ingénieur, consultant indépendant

Tél. : (+33) 6 09 23 21 36
E-mail : jp@anonymised.com
Web : www.pi-geosolutions.fr