Hi all, after some more tests with Sylvain on that
"java.io.FileNotFoundException" we could reproduce the problem on both 2.0.3
and trunk when stressing GeoNetwork (on linux, tomcat or jetty at least).
Problem occurs when too many search are made by lucene (you could try to use
apachebench "ab" to stress the app). Lucene opens index files but not close
thoses files in some cases. When the number of files opened by the java process
reach the system limit (ulimit -a), IOException occurs.
As said by Renaud, on option could be to increase the ulimit (see Lucene FAQ
also)
http://wiki.apache.org/lucene-java/LuceneFAQ#head-48921635adf2c968f7936dc07d51dfb40d638b82
Another (and better) option is to close the IndexSearcher/Reader in
LuceneSearcher once search ended.
A comment on the close method in LuceneSearcher said "RGFIX: check this" and
close only the reader. It looks like it's better to close both searcher and
reader once the search is done. Then the session use only the hits element.
Any comments on that?
Applying the patch attached will solve the problem. If that sounds good to
everyone, it could be applied for 2.3
as not so many users seem to face the problem for the time being ?
Ciao. Francois
PS: Thanks a lot to Sylvain for his help on tracking/solving that bug ...
(attachments)
GeoNetwork_tooManyOpenFiles.patch (1.32 KB)
+1 from me to fix that in 2.3. We can consider also applying it to 2.2.1 if it works well.
Ciao,
Jeroen
On Mar 26, 2008, at 6:10 PM, Francois-Xavier Prunayre wrote:
Hi all, after some more tests with Sylvain on that
"java.io.FileNotFoundException" we could reproduce the problem on both 2.0.3
and trunk when stressing GeoNetwork (on linux, tomcat or jetty at least).
Problem occurs when too many search are made by lucene (you could try to use
apachebench "ab" to stress the app). Lucene opens index files but not close
thoses files in some cases. When the number of files opened by the java process
reach the system limit (ulimit -a), IOException occurs.
As said by Renaud, on option could be to increase the ulimit (see Lucene FAQ
also)
Log In - Apache Software Foundation
Another (and better) option is to close the IndexSearcher/Reader in
LuceneSearcher once search ended.
A comment on the close method in LuceneSearcher said "RGFIX: check this" and
close only the reader. It looks like it's better to close both searcher and
reader once the search is done. Then the session use only the hits element.
Any comments on that?
Applying the patch attached will solve the problem. If that sounds good to
everyone, it could be applied for 2.3
as not so many users seem to face the problem for the time being ?
Ciao. Francois
PS: Thanks a lot to Sylvain for his help on tracking/solving that bug ...
<GeoNetwork_tooManyOpenFiles.patch>-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
geonetwork-users List Signup and Options
GeoNetwork OpenSource is maintained at GeoNetwork - Geographic Metadata Catalog download | SourceForge.net
Hi,
As Geonetwork is soon to have an ebRIM interface, are there any
publically available test ebXML documents, in particular for using with
the SubmitObjects ebRS request?
It doesn't matter if the actual data URLs don't resolve, I am just
looking to load up a server and can't find the test data used in the ESA
HMA testbed for the EO Product package.
Thanks,
Norman
hi,
we've had the "Too many open files" error on NGR. Though I'm not sure what
part of the code caused it, after discussing with François I've applied the
following fix to the code, in trunk and GN2.4 branches:
try {
computeQuery(srvContext, request, config);
performQuery(request, srvContext!=null?true:false);
initSearchRange(srvContext);
}
finally {
_hits = null;
_searcher.close();
_searcher = null;
setValid(false);
}
This forces the clean up of the resources after search, even if some
exception occurred during it. Now I don't know if this fixes the problem at
NGR, but at least it doesn't harm.
Kind regards
Heikki Doeleman
Jeroen Ticheler-3 wrote:
+1 from me to fix that in 2.3. We can consider also applying it to
2.2.1 if it works well.
Ciao,
Jeroen
On Mar 26, 2008, at 6:10 PM, Francois-Xavier Prunayre wrote:
Hi all, after some more tests with Sylvain on that
"java.io.FileNotFoundException" we could reproduce the problem on
both 2.0.3
and trunk when stressing GeoNetwork (on linux, tomcat or jetty at
least).
Problem occurs when too many search are made by lucene (you could
try to use
apachebench "ab" to stress the app). Lucene opens index files but
not close
thoses files in some cases. When the number of files opened by the
java process
reach the system limit (ulimit -a), IOException occurs.
As said by Renaud, on option could be to increase the ulimit (see
Lucene FAQ
also)
http://wiki.apache.org/lucene-java/LuceneFAQ#head-48921635adf2c968f7936dc07d51dfb40d638b82
Another (and better) option is to close the IndexSearcher/Reader in
LuceneSearcher once search ended.
A comment on the close method in LuceneSearcher said "RGFIX: check
this" and
close only the reader. It looks like it's better to close both
searcher and
reader once the search is done. Then the session use only the hits
element.
Any comments on that?
Applying the patch attached will solve the problem. If that sounds
good to
everyone, it could be applied for 2.3
as not so many users seem to face the problem for the time being ?
Ciao. Francois
PS: Thanks a lot to Sylvain for his help on tracking/solving that
bug ...
<
GeoNetwork_tooManyOpenFiles
.patch
>
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork
--
View this message in context: http://n2.nabble.com/Re-GeoNetwork-users-I-O-Exception-Too-many-open-files-tp2019782p3944336.html
Sent from the GeoNetwork developer mailing list archive at Nabble.com.