[Geoserver-devel] Hudson build bot: too many open files, restart needed?

GeoServer GeoServer Developer
1

Hi,
Hudson builds are failing because of "too many open files" issues,
both in Geotools and GeoServer and, it seems, also right out of
the svn update.

I guess a restart is in order? Thought it would be good to first
grab a list of what files are kept open for debugging purposes

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

2

So it is not just Hudson, and you want a full reboot?
If it is Linux, as root, record the output of:

ls -l /proc/*/fd
ls -efwww

See the per-process open file descriptors limit:
ulimit -n

The limit includes children, so exec'ed svn subprocesses will be affected by an fd leak in the parent process.

I think it unlikely that this is system level issue, as since 2.2 or 2.4 (?) this is dynamic and large. for example:
cat /proc/sys/fs/file-max
389262
Handles in use:
cat /proc/sys/fs/file-nr
6208 0 389262

Unless the system is near its soft limit, you should be able to resolve the issue by restarting tomcat.

Kind regards,
Ben.

On 13/06/11 04:25, Andrea Aime wrote:

Hi,
Hudson builds are failing because of "too many open files" issues,
both in Geotools and GeoServer and, it seems, also right out of
the svn update.

I guess a restart is in order? Thought it would be good to first
grab a list of what files are kept open for debugging purposes

Cheers
Andrea

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

3

Server restarted. Most of the files were log files… so yeah probably a hudson bug. Time for an upgrade probably.

On Sun, Jun 12, 2011 at 7:34 PM, Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com wrote:

So it is not just Hudson, and you want a full reboot?
If it is Linux, as root, record the output of:

ls -l /proc/*/fd
ls -efwww

See the per-process open file descriptors limit:
ulimit -n

The limit includes children, so exec’ed svn subprocesses will be
affected by an fd leak in the parent process.

I think it unlikely that this is system level issue, as since 2.2 or 2.4
(?) this is dynamic and large. for example:
cat /proc/sys/fs/file-max
389262
Handles in use:
cat /proc/sys/fs/file-nr
6208 0 389262

Unless the system is near its soft limit, you should be able to resolve
the issue by restarting tomcat.

Kind regards,
Ben.

On 13/06/11 04:25, Andrea Aime wrote:

Hi,
Hudson builds are failing because of “too many open files” issues,
both in Geotools and GeoServer and, it seems, also right out of
the svn update.

I guess a restart is in order? Thought it would be good to first
grab a list of what files are kept open for debugging purposes

Cheers
Andrea

–

Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com254…>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

EditLive Enterprise is the world’s most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev

Geotools-devel mailing list
Geotools-devel@anonymised.coms.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.