[GeoNetwork-devel] Strange problem with Geonetwork in multinode mode

Jo_Cook2 · December 21, 2016, 5:02pm

Hi Developers,

I’m running Geonetwork 3.2.x in multinode mode with postgresql as the back end database and have noticed a strange issue when editing records on the non-default node. If I edit a record, for example changing the title or the abstract, and save and close it, then this change does not show in the contribute menu or the display mode. The change does show when the record is edited. Rebuilding the index also makes the change show. I can’t find any errors in the log files and can confirm that the same problem does not occur in the default node.

Step by step procedure for testing:

Create new record from template
Make minimal changes, for example to the title
Save and close (should take you back to contribute menu)
Record shows title as it was in the original template
Go to tools and rebuild index
Return to contribute menu, title displays correctly

I was initially seeing errors in the log files about not having enough database connections, so I’ve increased those in [mynode].xml. I am not seeing any other errors that look to be related to this problem.

Can anyone replicate this problem for me, or suggest a fix?

Thanks

Jo

···

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jose_Garcia1 · December 22, 2016, 7:38am

Hi Jo

I need to test this, but seems like if the Lucene reader is not refreshed after the changes. I remember similar stuff time ago, but was non using the multinode mode, so depending how you got a metadata, the information returned was not the last one.

I can not provide right now a fix as need to test to confirm if it’s related to previous stuff or if it’s something else, like maybe not indexing in the proper node.

To be sure, does this not happen in the default node only in the other nodes?

Regards,
Jose García

···

On Wed, Dec 21, 2016 at 6:02 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’m running Geonetwork 3.2.x in multinode mode with postgresql as the back end database and have noticed a strange issue when editing records on the non-default node. If I edit a record, for example changing the title or the abstract, and save and close it, then this change does not show in the contribute menu or the display mode. The change does show when the record is edited. Rebuilding the index also makes the change show. I can’t find any errors in the log files and can confirm that the same problem does not occur in the default node.

Step by step procedure for testing:

Create new record from template

Make minimal changes, for example to the title

Save and close (should take you back to contribute menu)

Record shows title as it was in the original template

Go to tools and rebuild index

Return to contribute menu, title displays correctly

I was initially seeing errors in the log files about not having enough database connections, so I’ve increased those in [mynode].xml. I am not seeing any other errors that look to be related to this problem.

Can anyone replicate this problem for me, or suggest a fix?

Thanks

Jo

–

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

–
Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

^{Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.}

Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel

GeoNetwork-devel mailing list
GeoNetwork-devel@…537…sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Jo_Cook2 · December 22, 2016, 10:21am

Hi Jose,

I can confirm that this problem does not happen on the default node.

Thanks

Jo

···

On Thu, Dec 22, 2016 at 7:38 AM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I need to test this, but seems like if the Lucene reader is not refreshed after the changes. I remember similar stuff time ago, but was non using the multinode mode, so depending how you got a metadata, the information returned was not the last one.

I can not provide right now a fix as need to test to confirm if it’s related to previous stuff or if it’s something else, like maybe not indexing in the proper node.

To be sure, does this not happen in the default node only in the other nodes?

Regards,
Jose García

–

On Wed, Dec 21, 2016 at 6:02 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’m running Geonetwork 3.2.x in multinode mode with postgresql as the back end database and have noticed a strange issue when editing records on the non-default node. If I edit a record, for example changing the title or the abstract, and save and close it, then this change does not show in the contribute menu or the display mode. The change does show when the record is edited. Rebuilding the index also makes the change show. I can’t find any errors in the log files and can confirm that the same problem does not occur in the default node.

Step by step procedure for testing:

Create new record from template

Make minimal changes, for example to the title

Save and close (should take you back to contribute menu)

Record shows title as it was in the original template

Go to tools and rebuild index

Return to contribute menu, title displays correctly

I was initially seeing errors in the log files about not having enough database connections, so I’ve increased those in [mynode].xml. I am not seeing any other errors that look to be related to this problem.

Can anyone replicate this problem for me, or suggest a fix?

Thanks

Jo

–

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

–
Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

^{Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.}

Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel

GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

Jose_Garcia1 · January 9, 2017, 11:09am

Hi

I can reproduce the issue and found it’s not related to the Lucene readers not updated, but to this code in the indexing, that gets an old copy of the metadata:

https://github.com/geonetwork/core-geonetwork/blob/3.2.x/core/src/main/java/org/fao/geonet/kernel/DataManager.java#L597

Element md = getXmlSerializer().selectNoXLinkResolver(metadataId, true);

What is strange is that few lines below, it’s done this: https://github.com/geonetwork/core-geonetwork/blob/3.2.x/core/src/main/java/org/fao/geonet/kernel/DataManager.java#L616

fullMd = getMetadataRepository().findOne(id$);

And there it’s retrieved the xml and it’s fine containing the latest version.

Apart that seems a waste of resources to retrieve twice the metadata, I’m checking why the first call retrieves the old metadata instead of the new one. As soon as have clear will check to commit a fix for this.

Regards,
Jose García

···

On Thu, Dec 22, 2016 at 11:21 AM, Jo Cook <jocook@anonymised.com> wrote:

Hi Jose,

I can confirm that this problem does not happen on the default node.

Thanks

Jo

–
Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

^{Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.}

–

On Thu, Dec 22, 2016 at 7:38 AM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I need to test this, but seems like if the Lucene reader is not refreshed after the changes. I remember similar stuff time ago, but was non using the multinode mode, so depending how you got a metadata, the information returned was not the last one.

I can not provide right now a fix as need to test to confirm if it’s related to previous stuff or if it’s something else, like maybe not indexing in the proper node.

To be sure, does this not happen in the default node only in the other nodes?

Regards,
Jose García

–

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Wed, Dec 21, 2016 at 6:02 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’m running Geonetwork 3.2.x in multinode mode with postgresql as the back end database and have noticed a strange issue when editing records on the non-default node. If I edit a record, for example changing the title or the abstract, and save and close it, then this change does not show in the contribute menu or the display mode. The change does show when the record is edited. Rebuilding the index also makes the change show. I can’t find any errors in the log files and can confirm that the same problem does not occur in the default node.

Step by step procedure for testing:

Create new record from template

Make minimal changes, for example to the title

Save and close (should take you back to contribute menu)

Record shows title as it was in the original template

Go to tools and rebuild index

Return to contribute menu, title displays correctly

I was initially seeing errors in the log files about not having enough database connections, so I’ve increased those in [mynode].xml. I am not seeing any other errors that look to be related to this problem.

Can anyone replicate this problem for me, or suggest a fix?

Thanks

Jo

–

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

–
Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

^{Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.}

Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel

GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

Jose_Garcia1 · January 9, 2017, 11:51am

Hi

The problem seem this code: https://github.com/geonetwork/core-geonetwork/blob/3.2.x/domain/src/main/java/org/fao/geonet/repository/MetadataRepositoryImpl.java#L81-L87

It’s kind of syntactic sugar to accept a metadata id as an string:

@Override
public Metadata findOne(String id) {
try {
return _entityManager.find(Metadata.class, Integer.parseInt(id));
} catch (NumberFormatException e) {
throw new IllegalArgumentException(“id parameter of findByIdString must be parsable to an integer. It was '” + id + “'”);
}
}

Supposedly, should do the same as using findOne with an integer value (that works), but seem not the case.

I can fix the actual code in XmlSerializer to use findOne with integer parameter, but I suspect the actual code with string parameter doesn’t work due to _entityManager not being in the transaction. But would be good to hear from others, as if thats the case, the code using _entityManager seem not reliable.

Regards,
Jose García

···

On Mon, Jan 9, 2017 at 12:09 PM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi

I can reproduce the issue and found it’s not related to the Lucene readers not updated, but to this code in the indexing, that gets an old copy of the metadata:

https://github.com/geonetwork/core-geonetwork/blob/3.2.x/core/src/main/java/org/fao/geonet/kernel/DataManager.java#L597

Element md = getXmlSerializer().selectNoXLinkResolver(metadataId, true);

What is strange is that few lines below, it’s done this: https://github.com/geonetwork/core-geonetwork/blob/3.2.x/core/src/main/java/org/fao/geonet/kernel/DataManager.java#L616

fullMd = getMetadataRepository().findOne(id$);

And there it’s retrieved the xml and it’s fine containing the latest version.

Apart that seems a waste of resources to retrieve twice the metadata, I’m checking why the first call retrieves the old metadata instead of the new one. As soon as have clear will check to commit a fix for this.

Regards,
Jose García

–

On Thu, Dec 22, 2016 at 11:21 AM, Jo Cook <jocook@anonymised.com> wrote:

Hi Jose,

I can confirm that this problem does not happen on the default node.

Thanks

Jo

–
Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

^{Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.}

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.

On Thu, Dec 22, 2016 at 7:38 AM, Jose Garcia <jose.garcia@anonymised.com> wrote:

Hi Jo

I need to test this, but seems like if the Lucene reader is not refreshed after the changes. I remember similar stuff time ago, but was non using the multinode mode, so depending how you got a metadata, the information returned was not the last one.

I can not provide right now a fix as need to test to confirm if it’s related to previous stuff or if it’s something else, like maybe not indexing in the proper node.

To be sure, does this not happen in the default node only in the other nodes?

Regards,
Jose García

–

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

On Wed, Dec 21, 2016 at 6:02 PM, Jo Cook <jocook@anonymised.com> wrote:

Hi Developers,

I’m running Geonetwork 3.2.x in multinode mode with postgresql as the back end database and have noticed a strange issue when editing records on the non-default node. If I edit a record, for example changing the title or the abstract, and save and close it, then this change does not show in the contribute menu or the display mode. The change does show when the record is edited. Rebuilding the index also makes the change show. I can’t find any errors in the log files and can confirm that the same problem does not occur in the default node.

Step by step procedure for testing:

Create new record from template

Make minimal changes, for example to the title

Save and close (should take you back to contribute menu)

Record shows title as it was in the original template

Go to tools and rebuild index

Return to contribute menu, title displays correctly

I was initially seeing errors in the log files about not having enough database connections, so I’ve increased those in [mynode].xml. I am not seeing any other errors that look to be related to this problem.

Can anyone replicate this problem for me, or suggest a fix?

Thanks

Jo

–

Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek

–
Astun Technology Ltd, The Coach House, 17 West Street, Epsom, Surrey, KT18 7RL, UK
t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech

iShare - enterprise geographic intelligence platform
GeoServer, PostGIS and QGIS training
Helpdesk and customer portal

^{Company registration no. 5410695. Registered in England and Wales. Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no. 864201149.}

Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/intel

GeoNetwork-devel mailing list
GeoNetwork-devel@anonymised.comorge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

–

Vriendelijke groeten / Kind regards,

Jose García

Veenderweg 13
6721 WD Bennekom
The Netherlands
T: +31 (0)318 416664

Please consider the environment before printing this email.