[GeoNetwork-devel] Tomcat8 and | in query parameters

Hi,

I think I’ve hit an issue with Geonetwork 3.0.5 where requests with | in the query parameters fail on tomcat8 because of the behaviour described in this stack exchange post: http://stackoverflow.com/questions/41053653/tomcat-8-is-not-able-to-handle-get-request-with-in-query-parameters

The particular issue I’m seeing is that associated records don’t display in default view, the debug console in the browser shows that the following sorts of request https://URL/geonetwork/srv/eng/md.relations?_content_type=json&uuid=UUID&type=parent|children|service|dataset return a 400 error with no errors in the geonetwork log files. If the | are substituted for , then the request works.

I’m using tomcat 8.0.32.0, by the way.

Is this something that should be submitted as a github issue for 3.0.x, eg to use a different separator, and/or is it fixed in 3.2.x? Is there a patch I can do to on current install since it’s a production instance and I might not have the option to try multiple different tomcat versions to see which triggers the bug and which does not?

Many thanks

Jo

···

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Hi Developers,

I’ve not had a response to my question about pipes in query parameters causing an issue with tomcat8, so I’ve dug a little deeper on my own and this commit seems to be the culprit: https://github.com/geonetwork/core-geonetwork/commit/1695e14d7a606f073021c32195a9fcbfe89b56cd. It looks to me as if this is only a problem with geonetwork 3.0.x and not with 3.2.x and later. Can someone confirm this for me? If so, could I revert this line: https://github.com/geonetwork/core-geonetwork/blob/3.0.5/services/src/main/java/org/fao/geonet/guiservices/metadata/GetRelated.java#L322 so that it uses commas rather than pipes, which seems to be what it did before that commit?

I’m happy to submit an issue for this, but I don’t have enough understanding of whether the fix has implications elsewhere in the code, or even if it’s going to allow my associated resources to display, given the comment with that commit about angularjs using pipes rather than commas.

Can someone give me some advice here please?

Thanks

Jo

···

On Mon, Mar 13, 2017 at 1:34 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi,

I think I’ve hit an issue with Geonetwork 3.0.5 where requests with | in the query parameters fail on tomcat8 because of the behaviour described in this stack exchange post: http://stackoverflow.com/questions/41053653/tomcat-8-is-not-able-to-handle-get-request-with-in-query-parameters

The particular issue I’m seeing is that associated records don’t display in default view, the debug console in the browser shows that the following sorts of request https://URL/geonetwork/srv/eng/md.relations?_content_type=json&uuid=UUID&type=parent|children|service|dataset return a 400 error with no errors in the geonetwork log files. If the | are substituted for , then the request works.

I’m using tomcat 8.0.32.0, by the way.

Is this something that should be submitted as a github issue for 3.0.x, eg to use a different separator, and/or is it fixed in 3.2.x? Is there a patch I can do to on current install since it’s a production instance and I might not have the option to try multiple different tomcat versions to see which triggers the bug and which does not?

Many thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Hi Jo

I don’t know the reason for that change. In 3.2.x the service has change and should be used the end-point /api/records/UUID/related like:

https://SERVER/geonetwork/srv/api/records/UUID/related?type=parent&type=children&type=services&type=datasets

At least in the UI seem sending the values separated.

Not sure if will solve your case, but have you check if Tomcat has the URI encoding to UTF-8, I got some issues with accents, etc. if not set.

Although the pipe character should not cause that … take look in server.xml to the Connector:

<``Connector` `port``=``"8080"` `maxHttpHeaderSize``=``"8192"` maxThreads=“150” minSpareThreads=“25” maxSpareThreads=“75” ```enableLookups``=``"false" redirectPort``=``"8443" acceptCount``=``"100"
connectionTimeout``=``"20000"` `disableUploadTimeout``=``"true"` URIEncoding=“UTF-8” ```/>

And make sure that has the URIEncoding=“UTF-8” attribute.

Regards,
Jose García

···

On Fri, Mar 17, 2017 at 12:15 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’ve not had a response to my question about pipes in query parameters causing an issue with tomcat8, so I’ve dug a little deeper on my own and this commit seems to be the culprit: https://github.com/geonetwork/core-geonetwork/commit/1695e14d7a606f073021c32195a9fcbfe89b56cd. It looks to me as if this is only a problem with geonetwork 3.0.x and not with 3.2.x and later. Can someone confirm this for me? If so, could I revert this line: https://github.com/geonetwork/core-geonetwork/blob/3.0.5/services/src/main/java/org/fao/geonet/guiservices/metadata/GetRelated.java#L322 so that it uses commas rather than pipes, which seems to be what it did before that commit?

I’m happy to submit an issue for this, but I don’t have enough understanding of whether the fix has implications elsewhere in the code, or even if it’s going to allow my associated resources to display, given the comment with that commit about angularjs using pipes rather than commas.

Can someone give me some advice here please?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@…537…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

On Mon, Mar 13, 2017 at 1:34 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi,

I think I’ve hit an issue with Geonetwork 3.0.5 where requests with | in the query parameters fail on tomcat8 because of the behaviour described in this stack exchange post: http://stackoverflow.com/questions/41053653/tomcat-8-is-not-able-to-handle-get-request-with-in-query-parameters

The particular issue I’m seeing is that associated records don’t display in default view, the debug console in the browser shows that the following sorts of request https://URL/geonetwork/srv/eng/md.relations?_content_type=json&uuid=UUID&type=parent|children|service|dataset return a 400 error with no errors in the geonetwork log files. If the | are substituted for , then the request works.

I’m using tomcat 8.0.32.0, by the way.

Is this something that should be submitted as a github issue for 3.0.x, eg to use a different separator, and/or is it fixed in 3.2.x? Is there a patch I can do to on current install since it’s a production instance and I might not have the option to try multiple different tomcat versions to see which triggers the bug and which does not?

Many thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Hi Jose,

I don’t have any statements about URIEncoding in server.xml but will add that in and see if it makes any difference.

Thanks

Jo

···

On Fri, Mar 24, 2017 at 12:45 PM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I don’t know the reason for that change. In 3.2.x the service has change and should be used the end-point /api/records/UUID/related like:

https://SERVER/geonetwork/srv/api/records/UUID/related?type=parent&type=children&type=services&type=datasets

At least in the UI seem sending the values separated.

Not sure if will solve your case, but have you check if Tomcat has the URI encoding to UTF-8, I got some issues with accents, etc. if not set.

Although the pipe character should not cause that … take look in server.xml to the Connector:

<``Connector` `port``=``"8080"` `maxHttpHeaderSize``=``"8192"` maxThreads=“150” minSpareThreads=“25” maxSpareThreads=“75” ```enableLookups``=``"false" redirectPort``=``"8443" acceptCount``=``"100"
connectionTimeout``=``"20000"` `disableUploadTimeout``=``"true"` URIEncoding=“UTF-8” ```/>

And make sure that has the URIEncoding=“UTF-8” attribute.

Regards,
Jose García

On Fri, Mar 17, 2017 at 12:15 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’ve not had a response to my question about pipes in query parameters causing an issue with tomcat8, so I’ve dug a little deeper on my own and this commit seems to be the culprit: https://github.com/geonetwork/core-geonetwork/commit/1695e14d7a606f073021c32195a9fcbfe89b56cd. It looks to me as if this is only a problem with geonetwork 3.0.x and not with 3.2.x and later. Can someone confirm this for me? If so, could I revert this line: https://github.com/geonetwork/core-geonetwork/blob/3.0.5/services/src/main/java/org/fao/geonet/guiservices/metadata/GetRelated.java#L322 so that it uses commas rather than pipes, which seems to be what it did before that commit?

I’m happy to submit an issue for this, but I don’t have enough understanding of whether the fix has implications elsewhere in the code, or even if it’s going to allow my associated resources to display, given the comment with that commit about angularjs using pipes rather than commas.

Can someone give me some advice here please?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Mon, Mar 13, 2017 at 1:34 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi,

I think I’ve hit an issue with Geonetwork 3.0.5 where requests with | in the query parameters fail on tomcat8 because of the behaviour described in this stack exchange post: http://stackoverflow.com/questions/41053653/tomcat-8-is-not-able-to-handle-get-request-with-in-query-parameters

The particular issue I’m seeing is that associated records don’t display in default view, the debug console in the browser shows that the following sorts of request https://URL/geonetwork/srv/eng/md.relations?_content_type=json&uuid=UUID&type=parent|children|service|dataset return a 400 error with no errors in the geonetwork log files. If the | are substituted for , then the request works.

I’m using tomcat 8.0.32.0, by the way.

Is this something that should be submitted as a github issue for 3.0.x, eg to use a different separator, and/or is it fixed in 3.2.x? Is there a patch I can do to on current install since it’s a production instance and I might not have the option to try multiple different tomcat versions to see which triggers the bug and which does not?

Many thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Hi Jose, Developers,

I have just had the opportunity to try adding the URIEncoding element to tomcat8’s server.xml and I can report that it makes no difference to the display (or not) of associated records in geonetwork 3.0. Is there another fix for this?

Thanks

Jo

···

On Wed, Mar 29, 2017 at 4:10 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Jose,

I don’t have any statements about URIEncoding in server.xml but will add that in and see if it makes any difference.

Thanks

Jo

On Fri, Mar 24, 2017 at 12:45 PM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I don’t know the reason for that change. In 3.2.x the service has change and should be used the end-point /api/records/UUID/related like:

https://SERVER/geonetwork/srv/api/records/UUID/related?type=parent&type=children&type=services&type=datasets

At least in the UI seem sending the values separated.

Not sure if will solve your case, but have you check if Tomcat has the URI encoding to UTF-8, I got some issues with accents, etc. if not set.

Although the pipe character should not cause that … take look in server.xml to the Connector:

<``Connector` `port``=``"8080"` `maxHttpHeaderSize``=``"8192"` maxThreads=“150” minSpareThreads=“25” maxSpareThreads=“75” ```enableLookups``=``"false" redirectPort``=``"8443" acceptCount``=``"100"
connectionTimeout``=``"20000"` `disableUploadTimeout``=``"true"` URIEncoding=“UTF-8” ```/>

And make sure that has the URIEncoding=“UTF-8” attribute.

Regards,
Jose García

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Fri, Mar 17, 2017 at 12:15 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’ve not had a response to my question about pipes in query parameters causing an issue with tomcat8, so I’ve dug a little deeper on my own and this commit seems to be the culprit: https://github.com/geonetwork/core-geonetwork/commit/1695e14d7a606f073021c32195a9fcbfe89b56cd. It looks to me as if this is only a problem with geonetwork 3.0.x and not with 3.2.x and later. Can someone confirm this for me? If so, could I revert this line: https://github.com/geonetwork/core-geonetwork/blob/3.0.5/services/src/main/java/org/fao/geonet/guiservices/metadata/GetRelated.java#L322 so that it uses commas rather than pipes, which seems to be what it did before that commit?

I’m happy to submit an issue for this, but I don’t have enough understanding of whether the fix has implications elsewhere in the code, or even if it’s going to allow my associated resources to display, given the comment with that commit about angularjs using pipes rather than commas.

Can someone give me some advice here please?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Mon, Mar 13, 2017 at 1:34 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi,

I think I’ve hit an issue with Geonetwork 3.0.5 where requests with | in the query parameters fail on tomcat8 because of the behaviour described in this stack exchange post: http://stackoverflow.com/questions/41053653/tomcat-8-is-not-able-to-handle-get-request-with-in-query-parameters

The particular issue I’m seeing is that associated records don’t display in default view, the debug console in the browser shows that the following sorts of request https://URL/geonetwork/srv/eng/md.relations?_content_type=json&uuid=UUID&type=parent|children|service|dataset return a 400 error with no errors in the geonetwork log files. If the | are substituted for , then the request works.

I’m using tomcat 8.0.32.0, by the way.

Is this something that should be submitted as a github issue for 3.0.x, eg to use a different separator, and/or is it fixed in 3.2.x? Is there a patch I can do to on current install since it’s a production instance and I might not have the option to try multiple different tomcat versions to see which triggers the bug and which does not?

Many thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Hi Jo

I need to try it in tomcat 8, will check during today.

Maybe we can define other separator in the service to avoid the issue. Seem the service accepts also a comma, seem was changed to use a pipe due to Angular UI using that, but I guess that can be changed.

Regards,
Jose García

···

On Tue, Apr 25, 2017 at 6:59 PM, Jo Cook <jocook@anonymised.com036…> wrote:

Hi Jose, Developers,

I have just had the opportunity to try adding the URIEncoding element to tomcat8’s server.xml and I can report that it makes no difference to the display (or not) of associated records in geonetwork 3.0. Is there another fix for this?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

Register here: https://www.eventbrite.co.uk/e/astun-user-conference-2017-tickets-31457635635?aff=es2
Astun User Conference web page: https://astuntechnology.com/astun-user-conference/

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.

On Wed, Mar 29, 2017 at 4:10 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Jose,

I don’t have any statements about URIEncoding in server.xml but will add that in and see if it makes any difference.

Thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Fri, Mar 24, 2017 at 12:45 PM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I don’t know the reason for that change. In 3.2.x the service has change and should be used the end-point /api/records/UUID/related like:

https://SERVER/geonetwork/srv/api/records/UUID/related?type=parent&type=children&type=services&type=datasets

At least in the UI seem sending the values separated.

Not sure if will solve your case, but have you check if Tomcat has the URI encoding to UTF-8, I got some issues with accents, etc. if not set.

Although the pipe character should not cause that … take look in server.xml to the Connector:

<``Connector` `port``=``"8080"` `maxHttpHeaderSize``=``"8192"` maxThreads=“150” minSpareThreads=“25” maxSpareThreads=“75” ```enableLookups``=``"false" redirectPort``=``"8443" acceptCount``=``"100"
connectionTimeout``=``"20000"` `disableUploadTimeout``=``"true"` URIEncoding=“UTF-8” ```/>

And make sure that has the URIEncoding=“UTF-8” attribute.

Regards,
Jose García

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Fri, Mar 17, 2017 at 12:15 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’ve not had a response to my question about pipes in query parameters causing an issue with tomcat8, so I’ve dug a little deeper on my own and this commit seems to be the culprit: https://github.com/geonetwork/core-geonetwork/commit/1695e14d7a606f073021c32195a9fcbfe89b56cd. It looks to me as if this is only a problem with geonetwork 3.0.x and not with 3.2.x and later. Can someone confirm this for me? If so, could I revert this line: https://github.com/geonetwork/core-geonetwork/blob/3.0.5/services/src/main/java/org/fao/geonet/guiservices/metadata/GetRelated.java#L322 so that it uses commas rather than pipes, which seems to be what it did before that commit?

I’m happy to submit an issue for this, but I don’t have enough understanding of whether the fix has implications elsewhere in the code, or even if it’s going to allow my associated resources to display, given the comment with that commit about angularjs using pipes rather than commas.

Can someone give me some advice here please?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Mon, Mar 13, 2017 at 1:34 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi,

I think I’ve hit an issue with Geonetwork 3.0.5 where requests with | in the query parameters fail on tomcat8 because of the behaviour described in this stack exchange post: http://stackoverflow.com/questions/41053653/tomcat-8-is-not-able-to-handle-get-request-with-in-query-parameters

The particular issue I’m seeing is that associated records don’t display in default view, the debug console in the browser shows that the following sorts of request https://URL/geonetwork/srv/eng/md.relations?_content_type=json&uuid=UUID&type=parent|children|service|dataset return a 400 error with no errors in the geonetwork log files. If the | are substituted for , then the request works.

I’m using tomcat 8.0.32.0, by the way.

Is this something that should be submitted as a github issue for 3.0.x, eg to use a different separator, and/or is it fixed in 3.2.x? Is there a patch I can do to on current install since it’s a production instance and I might not have the option to try multiple different tomcat versions to see which triggers the bug and which does not?

Many thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Hi Jo

Seem the no support of the pipe in the url is due to a fix for a security issue, seem related to this: https://bugzilla.redhat.com/show_bug.cgi?id=1397484

I see 2 options, in the Angular code:

  • Encode in the request the pipe to %7C
  • Change the request to use a comma instead of a pipe

I’ll check to commit a fix with one of these options.

Regards,
Jose García

···

On Wed, Apr 26, 2017 at 8:30 AM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I need to try it in tomcat 8, will check during today.

Maybe we can define other separator in the service to avoid the issue. Seem the service accepts also a comma, seem was changed to use a pipe due to Angular UI using that, but I guess that can be changed.

Regards,
Jose García

On Tue, Apr 25, 2017 at 6:59 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Jose, Developers,

I have just had the opportunity to try adding the URIEncoding element to tomcat8’s server.xml and I can report that it makes no difference to the display (or not) of associated records in geonetwork 3.0. Is there another fix for this?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

Register here: https://www.eventbrite.co.uk/e/astun-user-conference-2017-tickets-31457635635?aff=es2
Astun User Conference web page: https://astuntechnology.com/astun-user-conference/

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Wed, Mar 29, 2017 at 4:10 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Jose,

I don’t have any statements about URIEncoding in server.xml but will add that in and see if it makes any difference.

Thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Fri, Mar 24, 2017 at 12:45 PM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I don’t know the reason for that change. In 3.2.x the service has change and should be used the end-point /api/records/UUID/related like:

https://SERVER/geonetwork/srv/api/records/UUID/related?type=parent&type=children&type=services&type=datasets

At least in the UI seem sending the values separated.

Not sure if will solve your case, but have you check if Tomcat has the URI encoding to UTF-8, I got some issues with accents, etc. if not set.

Although the pipe character should not cause that … take look in server.xml to the Connector:

<``Connector` `port``=``"8080"` `maxHttpHeaderSize``=``"8192"` maxThreads=“150” minSpareThreads=“25” maxSpareThreads=“75” ```enableLookups``=``"false" redirectPort``=``"8443" acceptCount``=``"100"
connectionTimeout``=``"20000"` `disableUploadTimeout``=``"true"` URIEncoding=“UTF-8” ```/>

And make sure that has the URIEncoding=“UTF-8” attribute.

Regards,
Jose García

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Fri, Mar 17, 2017 at 12:15 PM, Jo Cook <jocook@…1036…> wrote:

Hi Developers,

I’ve not had a response to my question about pipes in query parameters causing an issue with tomcat8, so I’ve dug a little deeper on my own and this commit seems to be the culprit: https://github.com/geonetwork/core-geonetwork/commit/1695e14d7a606f073021c32195a9fcbfe89b56cd. It looks to me as if this is only a problem with geonetwork 3.0.x and not with 3.2.x and later. Can someone confirm this for me? If so, could I revert this line: https://github.com/geonetwork/core-geonetwork/blob/3.0.5/services/src/main/java/org/fao/geonet/guiservices/metadata/GetRelated.java#L322 so that it uses commas rather than pipes, which seems to be what it did before that commit?

I’m happy to submit an issue for this, but I don’t have enough understanding of whether the fix has implications elsewhere in the code, or even if it’s going to allow my associated resources to display, given the comment with that commit about angularjs using pipes rather than commas.

Can someone give me some advice here please?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Mon, Mar 13, 2017 at 1:34 PM, Jo Cook <jocook@…1036…> wrote:

Hi,

I think I’ve hit an issue with Geonetwork 3.0.5 where requests with | in the query parameters fail on tomcat8 because of the behaviour described in this stack exchange post: http://stackoverflow.com/questions/41053653/tomcat-8-is-not-able-to-handle-get-request-with-in-query-parameters

The particular issue I’m seeing is that associated records don’t display in default view, the debug console in the browser shows that the following sorts of request https://URL/geonetwork/srv/eng/md.relations?_content_type=json&uuid=UUID&type=parent|children|service|dataset return a 400 error with no errors in the geonetwork log files. If the | are substituted for , then the request works.

I’m using tomcat 8.0.32.0, by the way.

Is this something that should be submitted as a github issue for 3.0.x, eg to use a different separator, and/or is it fixed in 3.2.x? Is there a patch I can do to on current install since it’s a production instance and I might not have the option to try multiple different tomcat versions to see which triggers the bug and which does not?

Many thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Hi, we could also allow multiple type parameters type=children&type=sources to be more consistent with other services in the API ?

Cheers

Francois

···

2017-04-26 8:48 GMT+02:00 Jose Garcia <jose.garcia@anonymised.com>:

Hi Jo

Seem the no support of the pipe in the url is due to a fix for a security issue, seem related to this: https://bugzilla.redhat.com/show_bug.cgi?id=1397484

I see 2 options, in the Angular code:

  • Encode in the request the pipe to %7C
  • Change the request to use a comma instead of a pipe

I’ll check to commit a fix with one of these options.

Regards,
Jose García


Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@…537…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

On Wed, Apr 26, 2017 at 8:30 AM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I need to try it in tomcat 8, will check during today.

Maybe we can define other separator in the service to avoid the issue. Seem the service accepts also a comma, seem was changed to use a pipe due to Angular UI using that, but I guess that can be changed.

Regards,
Jose García

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Tue, Apr 25, 2017 at 6:59 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Jose, Developers,

I have just had the opportunity to try adding the URIEncoding element to tomcat8’s server.xml and I can report that it makes no difference to the display (or not) of associated records in geonetwork 3.0. Is there another fix for this?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

Register here: https://www.eventbrite.co.uk/e/astun-user-conference-2017-tickets-31457635635?aff=es2
Astun User Conference web page: https://astuntechnology.com/astun-user-conference/

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Wed, Mar 29, 2017 at 4:10 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Jose,

I don’t have any statements about URIEncoding in server.xml but will add that in and see if it makes any difference.

Thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Fri, Mar 24, 2017 at 12:45 PM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I don’t know the reason for that change. In 3.2.x the service has change and should be used the end-point /api/records/UUID/related like:

https://SERVER/geonetwork/srv/api/records/UUID/related?type=parent&type=children&type=services&type=datasets

At least in the UI seem sending the values separated.

Not sure if will solve your case, but have you check if Tomcat has the URI encoding to UTF-8, I got some issues with accents, etc. if not set.

Although the pipe character should not cause that … take look in server.xml to the Connector:

<``Connector` `port``=``"8080"` `maxHttpHeaderSize``=``"8192"` maxThreads=“150” minSpareThreads=“25” maxSpareThreads=“75” ```enableLookups``=``"false" redirectPort``=``"8443" acceptCount``=``"100"
connectionTimeout``=``"20000"` `disableUploadTimeout``=``"true"` URIEncoding=“UTF-8” ```/>

And make sure that has the URIEncoding=“UTF-8” attribute.

Regards,
Jose García

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Fri, Mar 17, 2017 at 12:15 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’ve not had a response to my question about pipes in query parameters causing an issue with tomcat8, so I’ve dug a little deeper on my own and this commit seems to be the culprit: https://github.com/geonetwork/core-geonetwork/commit/1695e14d7a606f073021c32195a9fcbfe89b56cd. It looks to me as if this is only a problem with geonetwork 3.0.x and not with 3.2.x and later. Can someone confirm this for me? If so, could I revert this line: https://github.com/geonetwork/core-geonetwork/blob/3.0.5/services/src/main/java/org/fao/geonet/guiservices/metadata/GetRelated.java#L322 so that it uses commas rather than pipes, which seems to be what it did before that commit?

I’m happy to submit an issue for this, but I don’t have enough understanding of whether the fix has implications elsewhere in the code, or even if it’s going to allow my associated resources to display, given the comment with that commit about angularjs using pipes rather than commas.

Can someone give me some advice here please?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Mon, Mar 13, 2017 at 1:34 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi,

I think I’ve hit an issue with Geonetwork 3.0.5 where requests with | in the query parameters fail on tomcat8 because of the behaviour described in this stack exchange post: http://stackoverflow.com/questions/41053653/tomcat-8-is-not-able-to-handle-get-request-with-in-query-parameters

The particular issue I’m seeing is that associated records don’t display in default view, the debug console in the browser shows that the following sorts of request https://URL/geonetwork/srv/eng/md.relations?_content_type=json&uuid=UUID&type=parent|children|service|dataset return a 400 error with no errors in the geonetwork log files. If the | are substituted for , then the request works.

I’m using tomcat 8.0.32.0, by the way.

Is this something that should be submitted as a github issue for 3.0.x, eg to use a different separator, and/or is it fixed in 3.2.x? Is there a patch I can do to on current install since it’s a production instance and I might not have the option to try multiple different tomcat versions to see which triggers the bug and which does not?

Many thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Hi Jo

I have test and committed a fix for this:

https://github.com/geonetwork/core-geonetwork/commit/ea774a16b8456fbb1f629ced53ab5ade6e7d04ea

Regards,
Jose García

···

On Wed, Apr 26, 2017 at 8:48 AM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

Seem the no support of the pipe in the url is due to a fix for a security issue, seem related to this: https://bugzilla.redhat.com/show_bug.cgi?id=1397484

I see 2 options, in the Angular code:

  • Encode in the request the pipe to %7C
  • Change the request to use a comma instead of a pipe

I’ll check to commit a fix with one of these options.

Regards,
Jose García

On Wed, Apr 26, 2017 at 8:30 AM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I need to try it in tomcat 8, will check during today.

Maybe we can define other separator in the service to avoid the issue. Seem the service accepts also a comma, seem was changed to use a pipe due to Angular UI using that, but I guess that can be changed.

Regards,
Jose García

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Tue, Apr 25, 2017 at 6:59 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Jose, Developers,

I have just had the opportunity to try adding the URIEncoding element to tomcat8’s server.xml and I can report that it makes no difference to the display (or not) of associated records in geonetwork 3.0. Is there another fix for this?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

Register here: https://www.eventbrite.co.uk/e/astun-user-conference-2017-tickets-31457635635?aff=es2
Astun User Conference web page: https://astuntechnology.com/astun-user-conference/

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Wed, Mar 29, 2017 at 4:10 PM, Jo Cook <jocook@anonymised.com1036…> wrote:

Hi Jose,

I don’t have any statements about URIEncoding in server.xml but will add that in and see if it makes any difference.

Thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Fri, Mar 24, 2017 at 12:45 PM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I don’t know the reason for that change. In 3.2.x the service has change and should be used the end-point /api/records/UUID/related like:

https://SERVER/geonetwork/srv/api/records/UUID/related?type=parent&type=children&type=services&type=datasets

At least in the UI seem sending the values separated.

Not sure if will solve your case, but have you check if Tomcat has the URI encoding to UTF-8, I got some issues with accents, etc. if not set.

Although the pipe character should not cause that … take look in server.xml to the Connector:

<``Connector` `port``=``"8080"` `maxHttpHeaderSize``=``"8192"` maxThreads=“150” minSpareThreads=“25” maxSpareThreads=“75” ```enableLookups``=``"false" redirectPort``=``"8443" acceptCount``=``"100"
connectionTimeout``=``"20000"` `disableUploadTimeout``=``"true"` URIEncoding=“UTF-8” ```/>

And make sure that has the URIEncoding=“UTF-8” attribute.

Regards,
Jose García

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Fri, Mar 17, 2017 at 12:15 PM, Jo Cook <jocook@anonymised.com.> wrote:

Hi Developers,

I’ve not had a response to my question about pipes in query parameters causing an issue with tomcat8, so I’ve dug a little deeper on my own and this commit seems to be the culprit: https://github.com/geonetwork/core-geonetwork/commit/1695e14d7a606f073021c32195a9fcbfe89b56cd. It looks to me as if this is only a problem with geonetwork 3.0.x and not with 3.2.x and later. Can someone confirm this for me? If so, could I revert this line: https://github.com/geonetwork/core-geonetwork/blob/3.0.5/services/src/main/java/org/fao/geonet/guiservices/metadata/GetRelated.java#L322 so that it uses commas rather than pipes, which seems to be what it did before that commit?

I’m happy to submit an issue for this, but I don’t have enough understanding of whether the fix has implications elsewhere in the code, or even if it’s going to allow my associated resources to display, given the comment with that commit about angularjs using pipes rather than commas.

Can someone give me some advice here please?

Thanks

Jo


Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.


Check out the vibrant tech community on one of the world’s most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Mon, Mar 13, 2017 at 1:34 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi,

I think I’ve hit an issue with Geonetwork 3.0.5 where requests with | in the query parameters fail on tomcat8 because of the behaviour described in this stack exchange post: http://stackoverflow.com/questions/41053653/tomcat-8-is-not-able-to-handle-get-request-with-in-query-parameters

The particular issue I’m seeing is that associated records don’t display in default view, the debug console in the browser shows that the following sorts of request https://URL/geonetwork/srv/eng/md.relations?_content_type=json&uuid=UUID&type=parent|children|service|dataset return a 400 error with no errors in the geonetwork log files. If the | are substituted for , then the request works.

I’m using tomcat 8.0.32.0, by the way.

Is this something that should be submitted as a github issue for 3.0.x, eg to use a different separator, and/or is it fixed in 3.2.x? Is there a patch I can do to on current install since it’s a production instance and I might not have the option to try multiple different tomcat versions to see which triggers the bug and which does not?

Many thanks

Jo

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Vriendelijke groeten / Kind regards,

Jose García


Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.