Hi list,
I am new in the list and I am facing a problem which I will try to
summarize:
We are running on a Linux server a GN version 2.0.2 that has been working
pretty fine during the last months.
Recently, the following events have started to happen:
- Almost at the end of each day GN is having an I-O Exception when
dispatching to output with the stylesheet "search-results.xsl". To be more
precise, it's not this xsl that raises the problem but one of those
following this one in the process of showing a request result
(metadata-iso19115.xsl, metadata-fgdc-std.xsl , main.xsl or metadata.xsl,
etc (see Example 1 below, sorry logs are in French ).
- Usually, once it happens for the 1st time of the day it is
quickly followed by other errors having the same symptoms. As a result GN
his having, after a while, another kind of error:
"java.io.FileNotFoundException:
/home/www/www.sandre.eaufrance.fr.geonetwork/web/loc/fr/xml/strings.xml (Too
many open files)" (see Example 2 below).
- And, finally, we've got a "java.net.SocketException: Too many
open files" that causes our Tomcat server to freeze (see Example 3 below),
- We restart the server and everything works fine during the whole
day until the next I-O Exception !!
I have looked through the previous forum threads and haven't found any
solution.
Does someone have an idea that could help us go forward . ?
Here are the pieces of reasoning I have made:
- The request sent to GN by our users and causing the failure seems not to
be the same. I have identified no specific pattern except that they only use
'any' and 'selregion' parameters) but they often work fine during the day ->
problem due to system settings ?
- We have not changed majors settings on our server (nor on GN),
- Things are not always happening at the same hour -> not related to a cron
task.
As a result I have several questions:
- How can we trigger a request only having 'any' and 'selregion'
parameters sent to GN? Even in simple search, I've got other search
parameters appearing. It could help me trying to reproduce the error?
- The I-O Exception causes this cascade of problems (cause =
Concurrent access to an xsl ? or .?) but apparently xml files are not closed
when this I-O Exception happens which causes that "Too many open files"
error -> is that normal ?
- What can cause this I-O Exception (of course)?
*************
Example 1 :
**************
2008-03-09 07:24:31 | HTML Request (GET) : /geonetwork/srv/fr/main.search
2008-03-09 07:24:31 | Session created for client :192.168.1.105
2008-03-09 07:24:31 | Dispatching : main.search
2008-03-09 07:24:31 | -> parameters are :
2008-03-09 07:24:31 | any = var
2008-03-09 07:24:31 | selregion =
9999;-61.79611;-61.18708;15.87;16.51292
2008-03-09 07:24:31 | [main.search] Creating default search data
2008-03-09 07:24:31 | [main.search] Creating searchers
2008-03-09 07:24:31 | [main.search] Getting summary
2008-03-09 07:24:31 | -> dispatching to output for : main.search
2008-03-09 07:24:31 | -> forwarding to : main.present
2008-03-09 07:24:31 | Dispatching : main.present
2008-03-09 07:24:31 | -> parameters are :
2008-03-09 07:24:31 | summary =
2008-03-09 07:24:31 | forward =
2008-03-09 07:24:32 | -> dispatching to output for : main.present
2008-03-09 07:24:32 | -> transforming with stylesheet :
/home/www/www.sandre.eaufrance.fr.geonetwork/web/xsl/search-results.xsl
file:///home/www/xxx/web/xsl/search-results.xsl; Ligne #6; Colonne #32;
Exception d'E-S avec le fichier de la feuille de style : main.xsl
file:///home/www/www.sandre.eaufrance.fr.geonetwork/web/xsl/search-results.x
sl; Ligne #7; Colonne #36; Exception d'E-S avec le fichier de la feuille de
style : metadata.xsl
file:///home/www/www.sandre.eaufrance.fr.geonetwork/web/xsl/search-results.x
sl; Ligne #25; Colonne #40; javax.xml.transform.TransformerException: Erreur
de ElemTemplateElement : formTitle
*************
Example 2 :
**************
2008-03-09 07:25:33 | HTML Request (GET) : /geonetwork/srv/fr/main.search
2008-03-09 07:25:33 | Session created for client :192.168.1.105
2008-03-09 07:25:33 | Dispatching : main.search
2008-03-09 07:25:33 | -> parameters are :
2008-03-09 07:25:33 | any = bassins
2008-03-09 07:25:33 | selregion =
68;-5.79028;9.56222;41.36493;51.09111
2008-03-09 07:25:33 | [main.search] Creating default search data
2008-03-09 07:25:33 | [main.search] Creating searchers
reconnecting: 3628448>=3600000 ms since last connection
2008-03-09 07:25:33 | [main.search] Getting summary
2008-03-09 07:25:33 | -> dispatching to output for : main.search
2008-03-09 07:25:33 | -> forwarding to : main.present
2008-03-09 07:25:33 | Dispatching : main.present
2008-03-09 07:25:33 | -> parameters are :
2008-03-09 07:25:33 | summary =
2008-03-09 07:25:33 | forward =
2008-03-09 07:25:33 | -> dispatching to output for : main.present
2008-03-09 07:25:33 | [main.present] Exception executing gui service :
java.io.FileNotFoundException: /home/www/www.sandre.ea
ufrance.fr.geonetwork/web/loc/fr/xml/strings.xml (Too many open files)
2008-03-09 07:25:33 | [main.present] (C) Stack trace is :
java.io.FileNotFoundException:
/home/www/www.sandre.eaufrance.fr.geonetwork/web/loc/fr/xml/strings.xml (Too
many open files)
*************
Example 3 :
*************
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
at java.net.ServerSocket.implAccept(ServerSocket.java:450)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at
org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultSe
rverSocketFactory.java:60)
at
org.apache.tomcat.util.net.PoolTcpEndpoint.acceptSocket(PoolTcpEndpoint.java
:407)
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWo
rkerThread.java:70)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.jav
a:684)
at java.lang.Thread.run(Thread.java:595)
Thank you in advance for your help
Sylvain