[GRASS-user] Maximum number of GRASS files within a mapset?

GRASS GRASS user
1

Hi,

I seem to be running into a GRASS limitation relating to the maximum number of rasters available within a single mapset location, which appears to be exactly 32000. Attempting to create additional rasters generates the message:

ERROR: can't make mapset element cell_misc/test00

which seems to be coming from lib/gis/mapset_msc.c:
   if (access (path, 0) != 0)
   G_fatal_error (_("can't make mapset element %s (%s)"), p_element, path);

Is there any way of circumventing this limitation, or is it a hardcoded C limitation?

thanks for the help,
Shaun

--
Shaun C. Walbridge
GIS Analyst / Information Manager
National Center for Ecological Analysis and Synthesis (NCEAS)
p: 805.570.8714

2

On Fri, 23 Mar 2007, Shaun Walbridge wrote:

Hi,

I seem to be running into a GRASS limitation relating to the maximum number of rasters available within a single mapset location, which appears to be exactly 32000. Attempting to create additional rasters generates the message:

ERROR: can't make mapset element cell_misc/test00

which seems to be coming from lib/gis/mapset_msc.c:
if (access (path, 0) != 0)
G_fatal_error (_("can't make mapset element %s (%s)"), p_element, path);

Is there any way of circumventing this limitation, or is it a hardcoded C limitation?

I fear it may be a Unix limitation - the maximum number of files in one directory. Maybe reading documentation to see if there was anyway to increase this would be the best course of action.

Paul

3

Paul Kelly wrote:

> I seem to be running into a GRASS limitation relating to the maximum number
> of rasters available within a single mapset location, which appears to be
> exactly 32000. Attempting to create additional rasters generates the
> message:
>
> ERROR: can't make mapset element cell_misc/test00

                                     ^^^^^^^^^

Note: cell_misc contains subdirectories, not files ...

> which seems to be coming from lib/gis/mapset_msc.c:
> if (access (path, 0) != 0)
> G_fatal_error (_("can't make mapset element %s (%s)"), p_element, path);
>
> Is there any way of circumventing this limitation, or is it a hardcoded C
> limitation?

I fear it may be a Unix limitation - the maximum number of files in one
directory.

Or possibly the maximum link count; cell_misc has a subdirectory for
each map, and each subdirectory adds to its parent's link count due to
the ".." entry.

I note that <linux/ext2_fs.h> has this:

  /*
   * Maximal count of links to a file
   */
  #define EXT2_LINK_MAX 32000

So I'm fairly sure that's the culprit.

Changing the directory layout is a proposal for 7.x; until then, the
only thing that I can suggest is to try a different filesystem, e.g.
ReiserFS (ext3 has the same 32000-link limit).

--
Glynn Clements <glynn@gclements.plus.com>

4

Hmm hmm, I was going to say it is a ‘subdirectory’ issue most likely…

wow, that’s a massive amount of seperate rasters! I suppose unfortunately the only answer I can offer would be to create another mapset, or if there was anyway possible to merge some of your raster content together…

Is there someway to link externally available?

On 23-Mar-07, at 4:38 PM, Glynn Clements wrote:

Paul Kelly wrote:

I seem to be running into a GRASS limitation relating to the maximum number
of rasters available within a single mapset location, which appears to be
exactly 32000. Attempting to create additional rasters generates the
message:

ERROR: can’t make mapset element cell_misc/test00

^^^^^^^^^

Note: cell_misc contains subdirectories, not files …

which seems to be coming from lib/gis/mapset_msc.c:
if (access (path, 0) != 0)
G_fatal_error (_(“can’t make mapset element %s (%s)”), p_element, path);

Is there any way of circumventing this limitation, or is it a hardcoded C
limitation?

I fear it may be a Unix limitation - the maximum number of files in one
directory.

Or possibly the maximum link count; cell_misc has a subdirectory for
each map, and each subdirectory adds to its parent’s link count due to
the “…” entry.

I note that <linux/ext2_fs.h> has this:

/*

  • Maximal count of links to a file
    */
    #define EXT2_LINK_MAX 32000

So I’m fairly sure that’s the culprit.

Changing the directory layout is a proposal for 7.x; until then, the
only thing that I can suggest is to try a different filesystem, e.g.
ReiserFS (ext3 has the same 32000-link limit).

–
Glynn Clements <glynn@gclements.plus.com>

grassuser mailing list
grassuser@grass.itc.it
http://grass.itc.it/mailman/listinfo/grassuser

Amaginary Studios
amagine@telus.net
Phone 250 845 7999

Mars Sjoden
3320 Mount Davis Way
Houston BC
V0J 1Z2

READ CAREFULLY.
By reading this email, you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (“BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

5

Shaun wrote:

I seem to be running into a GRASS limitation relating to the maximum
number of rasters available within a single mapset location, which
appears to be exactly 32000.

Paul:

I fear it may be a Unix limitation - the maximum number of files
in one directory.

Amaginary Studios wrote:

wow, that's a massive amount of seperate rasters! I suppose
unfortunately the only answer I can offer would be to create another
mapset,

yeah, try creating a new mapset to put new maps in, then add the
previous mapset(s) into the map search path with the g.mapsets module
(with an "s").

even if a mapset is not in the search path, you can still access it with
"mapname@othermapset".

Hamish