[GeoNetwork-users] publishing workflow and Java errors

Hi: Using 2.6.4, we are operating a GN instance, we have a publishing
workflow in support of our WMO requirements.

Server:
- total RAM: 1GB
- RAM allocated to Tomcat: 256MB

Process (in Python):
- login using XML services API
- search for record identifier using XML services API
- if exists:
- get record version using metadata.edit, scrape HTML to get hidden
form value for version
- update using XML services API
- else:
- insert using XML services API
- logout using XML services API

When we run the initial inserts on this process (~60 records), all
records insert with no issues.

In subsequent runs, where we need to update records per above,
performance gets progressively slower until ~ the 40th record update, at
which point the following exception is raised:

type Exception report
message
description The server encountered an internal error () that prevented
it from fulfilling this request.
exception
javax.servlet.ServletException: Servlet execution threw an exception
root cause
java.lang.OutOfMemoryError: Java heap space
...

I realize that the documentation suggests at least 1GB RAM and 2GB is
optimal; is this required on the system as a whole, or to be totally
dedicated to the GN instance? Is the issue that we don't have enough
RAM dedicated?

Having said this, any thoughts on working around this issue? Are we
doing something in our workflow that is causing this issue in
particular?

Thanks

..Tom

Hi Tom

The recommendation of memory is for Tomcat in your case. 256 Mb is quite
low memory for GeoNetwork. Also check how the inserts are done to make sure
you reuse the session after the login. Usually the process should be like
(possibly is your case already):

Login
Grab the JSESSIONID from the cookies
Send insert request with previous JSESSIONID
Send insert request with previous JSESSIONID
...
Logout

This way you guarantee using 1 session, otherwise can end with quite
sessions that take the memory.

Regards,
Jose García

On Wed, May 8, 2013 at 11:00 PM, Kralidis,Tom [Ontario] <
Tom.Kralidis@anonymised.com> wrote:

Hi: Using 2.6.4, we are operating a GN instance, we have a publishing
workflow in support of our WMO requirements.

Server:
- total RAM: 1GB
- RAM allocated to Tomcat: 256MB

Process (in Python):
- login using XML services API
- search for record identifier using XML services API
- if exists:
- get record version using metadata.edit, scrape HTML to get hidden
form value for version
- update using XML services API
- else:
- insert using XML services API
- logout using XML services API

When we run the initial inserts on this process (~60 records), all
records insert with no issues.

In subsequent runs, where we need to update records per above,
performance gets progressively slower until ~ the 40th record update, at
which point the following exception is raised:

type Exception report
message
description The server encountered an internal error () that prevented
it from fulfilling this request.
exception
javax.servlet.ServletException: Servlet execution threw an exception
root cause
java.lang.OutOfMemoryError: Java heap space
...

I realize that the documentation suggests at least 1GB RAM and 2GB is
optimal; is this required on the system as a whole, or to be totally
dedicated to the GN instance? Is the issue that we don't have enough
RAM dedicated?

Having said this, any thoughts on working around this issue? Are we
doing something in our workflow that is causing this issue in
particular?

Thanks

..Tom

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and
their applications. This 200-page book is written by three acclaimed
leaders in the field. The early access version is available now.
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

--
*
GeoCat Bridge for ArcGIS allows instant publishing of data and metadata on
GeoServer and GeoNetwork. Visit http://geocat.net for details.
_________________________
Jose García
GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
http://GeoCat.net/&gt;

*

Hi Jose: thanks for the info.

We are getting the JSESSIONID from the Cookie HTTP response header on login, then we are passing the exact Cookie value to subsequent requests via the Set-Cookie HTTP request header. Is this the desired interaction or should we pass JSESSIONID= as part of the URL instead?

From: Jose Garcia [mailto:jose.garcia@…444…]
Sent: Thursday, May 09, 2013 02:02
To: Kralidis,Tom [Ontario]
Cc: geonetwork-users@lists.sourceforge.net
Subject: Re: [GeoNetwork-users] publishing workflow and Java errors

Hi Tom

The recommendation of memory is for Tomcat in your case. 256 Mb is quite low memory for GeoNetwork. Also check how the inserts are done to make sure you reuse the session after the login. Usually the process should be like (possibly is your case already):

Login
Grab the JSESSIONID from the cookies
Send insert request with previous JSESSIONID
Send insert request with previous JSESSIONID
...
Logout

This way you guarantee using 1 session, otherwise can end with quite sessions that take the memory.

Regards,
Jose García

On Wed, May 8, 2013 at 11:00 PM, Kralidis,Tom [Ontario] <Tom.Kralidis@...43...> wrote:
Hi: Using 2.6.4, we are operating a GN instance, we have a publishing
workflow in support of our WMO requirements.

Server:
- total RAM: 1GB
- RAM allocated to Tomcat: 256MB

Process (in Python):
- login using XML services API
- search for record identifier using XML services API
- if exists:
- get record version using metadata.edit, scrape HTML to get hidden
form value for version
- update using XML services API
- else:
- insert using XML services API
- logout using XML services API

When we run the initial inserts on this process (~60 records), all
records insert with no issues.

In subsequent runs, where we need to update records per above,
performance gets progressively slower until ~ the 40th record update, at
which point the following exception is raised:

type Exception report
message
description The server encountered an internal error () that prevented
it from fulfilling this request.
exception
javax.servlet.ServletException: Servlet execution threw an exception
root cause
java.lang.OutOfMemoryError: Java heap space
...

I realize that the documentation suggests at least 1GB RAM and 2GB is
optimal; is this required on the system as a whole, or to be totally
dedicated to the GN instance? Is the issue that we don't have enough
RAM dedicated?

Having said this, any thoughts on working around this issue? Are we
doing something in our workflow that is causing this issue in
particular?

Thanks

..Tom

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and
their applications. This 200-page book is written by three acclaimed
leaders in the field. The early access version is available now.
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

--
GeoCat Bridge for ArcGIS allows instant publishing of data and metadata on GeoServer and GeoNetwork. Visit http://geocat.net for details.
_________________________
Jose García
GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
http://GeoCat.net

Hi Tom

I think is correct the way you're doing it. To verify, check in the
responses that the same JSESSIONID is returned.

Regards,
Jose García

On Thu, May 9, 2013 at 2:42 PM, Kralidis,Tom [Ontario] <
Tom.Kralidis@anonymised.com> wrote:

Hi Jose: thanks for the info.

We are getting the JSESSIONID from the Cookie HTTP response header on
login, then we are passing the exact Cookie value to subsequent requests
via the Set-Cookie HTTP request header. Is this the desired interaction or
should we pass JSESSIONID= as part of the URL instead?

From: Jose Garcia [mailto:jose.garcia@anonymised.com]
Sent: Thursday, May 09, 2013 02:02
To: Kralidis,Tom [Ontario]
Cc: geonetwork-users@lists.sourceforge.net
Subject: Re: [GeoNetwork-users] publishing workflow and Java errors

Hi Tom

The recommendation of memory is for Tomcat in your case. 256 Mb is quite
low memory for GeoNetwork. Also check how the inserts are done to make sure
you reuse the session after the login. Usually the process should be like
(possibly is your case already):

Login
Grab the JSESSIONID from the cookies
Send insert request with previous JSESSIONID
Send insert request with previous JSESSIONID
...
Logout

This way you guarantee using 1 session, otherwise can end with quite
sessions that take the memory.

Regards,
Jose García

On Wed, May 8, 2013 at 11:00 PM, Kralidis,Tom [Ontario] <
Tom.Kralidis@anonymised.com> wrote:
Hi: Using 2.6.4, we are operating a GN instance, we have a publishing
workflow in support of our WMO requirements.

Server:
- total RAM: 1GB
- RAM allocated to Tomcat: 256MB

Process (in Python):
- login using XML services API
- search for record identifier using XML services API
- if exists:
- get record version using metadata.edit, scrape HTML to get hidden
form value for version
- update using XML services API
- else:
- insert using XML services API
- logout using XML services API

When we run the initial inserts on this process (~60 records), all
records insert with no issues.

In subsequent runs, where we need to update records per above,
performance gets progressively slower until ~ the 40th record update, at
which point the following exception is raised:

type Exception report
message
description The server encountered an internal error () that prevented
it from fulfilling this request.
exception
javax.servlet.ServletException: Servlet execution threw an exception
root cause
java.lang.OutOfMemoryError: Java heap space
...

I realize that the documentation suggests at least 1GB RAM and 2GB is
optimal; is this required on the system as a whole, or to be totally
dedicated to the GN instance? Is the issue that we don't have enough
RAM dedicated?

Having said this, any thoughts on working around this issue? Are we
doing something in our workflow that is causing this issue in
particular?

Thanks

..Tom

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and
their applications. This 200-page book is written by three acclaimed
leaders in the field. The early access version is available now.
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

--
GeoCat Bridge for ArcGIS allows instant publishing of data and metadata on
GeoServer and GeoNetwork. Visit http://geocat.net for details.
_________________________
Jose García
GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
http://GeoCat.net

--
*
GeoCat Bridge for ArcGIS allows instant publishing of data and metadata on
GeoServer and GeoNetwork. Visit http://geocat.net for details.
_________________________
Jose García
GeoCat bv
Veenderweg 13
6721 WD Bennekom
The Netherlands
http://GeoCat.net/&gt;

*