[Geoserver-users] Symlinks for data sub-directories?

Hi

We have been struggling to get GeoServer to follow data stored in a
symlinked directory (symlinked from GeoServer's actual data directory). All
permissions have been set and the owner/group is set to Tomcat7. The ls
output shows:

lrwxrwxrwx 1 tomcat7 tomcat7 21 Aug 4 13:03 spatial -> /mnt/sdi/spatial/

and this link and file system can be navigated from the command line.

If any of the data is copied over from the symlinked directory to the
actual data directory (in our case, at /var/geoserver/data/), then it can be
loaded as expected. If not, then we get an error message:

Error creating data store, check the parameters. Error message: Could not
acquire data access 'XYZ'

The log show the same message, along with the usual NullPointerException.

Is there an additional Tomcat or GeoServer configuration that is required to
access symlinked directories?

Thanks!
Derek

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Symlinks-for-data-sub-directories-tp5155119.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

Derek,

symlinks should work, but note that access depends on the permissions of the symlink target, not of the symlink itself.

What is the output of this?:

ls -al /mnt/sdi/spatial

If permissions are the problem, you can change the ownership and permissions of /mnt/sdi/spatial (*recursively*) with these commands, but note that this may not be what you want so use with caution:

chown -R tomcat7:tomcat7 /mnt/sdi/spatial
chmod -R ug+rX /mnt/sdi/spatial

Kind regards,
Ben.

On 05/08/14 21:14, Derek wrote:

Hi

We have been struggling to get GeoServer to follow data stored in a
symlinked directory (symlinked from GeoServer's actual data directory). All
permissions have been set and the owner/group is set to Tomcat7. The ls
output shows:

lrwxrwxrwx 1 tomcat7 tomcat7 21 Aug 4 13:03 spatial -> /mnt/sdi/spatial/

and this link and file system can be navigated from the command line.

If any of the data is copied over from the symlinked directory to the
actual data directory (in our case, at /var/geoserver/data/), then it can be
loaded as expected. If not, then we get an error message:

Error creating data store, check the parameters. Error message: Could not
acquire data access 'XYZ'

The log show the same message, along with the usual NullPointerException.

Is there an additional Tomcat or GeoServer configuration that is required to
access symlinked directories?

Thanks!
Derek

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Symlinks-for-data-sub-directories-tp5155119.html
Sent from the GeoServer - User mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk
_______________________________________________
Geoserver-users mailing list
Geoserver-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Mineral Resources Flagship
Australian Resources Research Centre

Hi Ben

Thanks for the reply. Yes, the permissions look OK. Using your command, for example, ls -al /mnt/sdi/spatial/city/planning, I get this listing for a directory containing a shapefile that I am trying to access:

drwxrwxrwx 2 tomcat7 tomcat7 4096 Feb 26 09:37 .
drwxrwxrwx 7 tomcat7 tomcat7 4096 Aug 5 10:57 …

-rw-rwxrwx 1 tomcat7 tomcat7 47966 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.dbf
-rw-rwxrwx 1 tomcat7 tomcat7 145 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.prj
-rw-rwxrwx 1 tomcat7 tomcat7 1556 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.sbn
-rw-rwxrwx 1 tomcat7 tomcat7 196 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.sbx
-rw-rwxrwx 1 tomcat7 tomcat7 108300 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.shp
-rw-rwxrwx 1 tomcat7 tomcat7 1212 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.shx

Is there something else that needs to be changed?

Regards
Derek

···

On 6 August 2014 04:55, Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com> wrote:

Derek,

symlinks should work, but note that access depends on the permissions of the symlink target, not of the symlink itself.

What is the output of this?:

ls -al /mnt/sdi/spatial

If permissions are the problem, you can change the ownership and permissions of /mnt/sdi/spatial (recursively) with these commands, but note that this may not be what you want so use with caution:

chown -R tomcat7:tomcat7 /mnt/sdi/spatial
chmod -R ug+rX /mnt/sdi/spatial

Kind regards,
Ben.

On 05/08/14 21:14, Derek wrote:

Hi

We have been struggling to get GeoServer to follow data stored in a
symlinked directory (symlinked from GeoServer’s actual data directory). All
permissions have been set and the owner/group is set to Tomcat7. The ls
output shows:

lrwxrwxrwx 1 tomcat7 tomcat7 21 Aug 4 13:03 spatial → /mnt/sdi/spatial/

and this link and file system can be navigated from the command line.

If any of the data is copied over from the symlinked directory to the
actual data directory (in our case, at /var/geoserver/data/), then it can be
loaded as expected. If not, then we get an error message:

Error creating data store, check the parameters. Error message: Could not
acquire data access ‘XYZ’

The log show the same message, along with the usual NullPointerException.

Is there an additional Tomcat or GeoServer configuration that is required to
access symlinked directories?

Thanks!
Derek


View this message in context: http://osgeo-org.1560.x6.nabble.com/Symlinks-for-data-sub-directories-tp5155119.html
Sent from the GeoServer - User mailing list archive at Nabble.com.


Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.
http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk


Geoserver-users mailing list
Geoserver-users@anonymised.comsourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-users


Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com
Software Engineer
CSIRO Mineral Resources Flagship
Australian Resources Research Centre

Derek,

can you show us the contents of the datastore.xml file in your data directory that defines this data store?

What platform are you using? CentOS / RedHat often have selinux enforcing; this additional security mechanism may prevent tomcat7 from accessing some files. If you have selinux security contexts, you can see them with:

ls -alZ /mnt/sdi/spatial
ls -alZ /mnt/sdi/spatial/city/planning

Is this a network mount? What sort of filesystem? ext4, VFAT, or NTFS? Try:

mount | grep /mnt/sdi

It would also be helpful if you have a stack trace from the geoserver logs when this error occurs. This may give us more detail about the underlying error.

Kind regards,
Ben.

On 06/08/14 15:26, Derek wrote:

Hi Ben

Thanks for the reply. Yes, the permissions look OK. Using your
command, for example, ls -al /mnt/sdi/spatial/city/planning, I get this
listing for a directory containing a shapefile that I am trying to access:

drwxrwxrwx 2 tomcat7 tomcat7 4096 Feb 26 09:37 .
drwxrwxrwx 7 tomcat7 tomcat7 4096 Aug 5 10:57 ..
-rw-rwxrwx 1 tomcat7 tomcat7 47966 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.dbf
-rw-rwxrwx 1 tomcat7 tomcat7 145 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.prj
-rw-rwxrwx 1 tomcat7 tomcat7 1556 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.sbn
-rw-rwxrwx 1 tomcat7 tomcat7 196 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.sbx
-rw-rwxrwx 1 tomcat7 tomcat7 108300 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.shp
-rw-rwxrwx 1 tomcat7 tomcat7 1212 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.shx

Is there something else that needs to be changed?

Regards
Derek

On 6 August 2014 04:55, Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com
<mailto:Ben.Caradoc-Davies@anonymised.com>> wrote:

    Derek,

    symlinks should work, but note that access depends on the
    permissions of the symlink target, not of the symlink itself.

    What is the output of this?:

    ls -al /mnt/sdi/spatial

    If permissions are the problem, you can change the ownership and
    permissions of /mnt/sdi/spatial (*recursively*) with these commands,
    but note that this may not be what you want so use with caution:

    chown -R tomcat7:tomcat7 /mnt/sdi/spatial
    chmod -R ug+rX /mnt/sdi/spatial

    Kind regards,
    Ben.

    On 05/08/14 21:14, Derek wrote:

        Hi

        We have been struggling to get GeoServer to follow data stored in a
        symlinked directory (symlinked from GeoServer's actual data
        directory). All
        permissions have been set and the owner/group is set to Tomcat7.
        The ls
        output shows:

        lrwxrwxrwx 1 tomcat7 tomcat7 21 Aug 4 13:03 spatial ->
        /mnt/sdi/spatial/

        and this link and file system can be navigated from the command
        line.

        If any of the data is copied over from the symlinked directory
        to the
        actual data directory (in our case, at /var/geoserver/data/),
        then it can be
        loaded as expected. If not, then we get an error message:

        Error creating data store, check the parameters. Error message:
        Could not
        acquire data access 'XYZ'

        The log show the same message, along with the usual
        NullPointerException.

        Is there an additional Tomcat or GeoServer configuration that is
        required to
        access symlinked directories?

        Thanks!
        Derek

        --
        View this message in context:
        http://osgeo-org.1560.x6.__nabble.com/Symlinks-for-data-__sub-directories-tp5155119.html
        <http://osgeo-org.1560.x6.nabble.com/Symlinks-for-data-sub-directories-tp5155119.html&gt;
        Sent from the GeoServer - User mailing list archive at Nabble.com.

        ------------------------------__------------------------------__------------------
        Infragistics Professional
        Build stunning WinForms apps today!
        Reboot your WinForms applications with our WinForms controls.
        Build a bridge from your legacy apps to the future.
        http://pubads.g.doubleclick.__net/gampad/clk?id=153845071&__iu=/4140/ostg.clktrk
        <http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk&gt;
        _________________________________________________
        Geoserver-users mailing list
        Geoserver-users@anonymised.com
        <mailto:Geoserver-users@lists.sourceforge.net>
        https://lists.sourceforge.net/__lists/listinfo/geoserver-users
        <https://lists.sourceforge.net/lists/listinfo/geoserver-users&gt;

    --
    Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
    Software Engineer
    CSIRO Mineral Resources Flagship
    Australian Resources Research Centre

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Mineral Resources Flagship
Australian Resources Research Centre

Hi Ben

The mount command looks like:

/dev/mapper/san on /mnt/sdi type ext4 (rw,noatime,stripe=2560,data=ordered)

(The network admin tells me this a SAN drive mounted directly to the server.)

The response from the “Z” parameter - we are running Ubuntu - looks like:

-rw-rwxrwx 1 tomcat7 tomcat7 ? 47966 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.dbf
-rw-rwxrwx 1 tomcat7 tomcat7 ? 145 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.prj
-rw-rwxrwx 1 tomcat7 tomcat7 ? 1556 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.sbn
-rw-rwxrwx 1 tomcat7 tomcat7 ? 196 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.sbx
-rw-rwxrwx 1 tomcat7 tomcat7 ? 108300 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.shp
-rw-rwxrwx 1 tomcat7 tomcat7 ? 1212 Feb 26 09:37 CoT_master_sbc_2010_poly_wgs84_1.shx

No datastore.xml has been created for this shapefile. Here’s an example from another one (but this from data already on the host, and not on the symlinked drive).

DataStoreInfoImpl-63f945a9:147a4d4b8b2:-7fff Pop Pop Shapefile true WorkspaceInfoImpl-10073beb:147a14d6b3c:-8000 ISO-8859-1 shapefile true false Etc/UTC true sdi true file:ne_10m_admin_0_countries/ne_10m_admin_0_countries.shp shape <__default>false

I attach the relevant portion of the error trace from catalina.out.

Thanks
Derek

catalina.txt (20.5 KB)

···

On 6 August 2014 09:52, Ben Caradoc-Davies <Ben.Caradoc-Davies@…367…> wrote:

Derek,

can you show us the contents of the datastore.xml file in your data directory that defines this data store?

What platform are you using? CentOS / RedHat often have selinux enforcing; this additional security mechanism may prevent tomcat7 from accessing some files. If you have selinux security contexts, you can see them with:

ls -alZ /mnt/sdi/spatial
ls -alZ /mnt/sdi/spatial/city/planning

Is this a network mount? What sort of filesystem? ext4, VFAT, or NTFS? Try:

mount | grep /mnt/sdi

It would also be helpful if you have a stack trace from the geoserver logs when this error occurs. This may give us more detail about the underlying error.

Kind regards,
Ben.

On 06/08/14 15:26, Derek wrote:

Hi Ben

Thanks for the reply. Yes, the permissions look OK. Using your
command, for example, ls -al /mnt/sdi/spatial/city/planning, I get this
listing for a directory containing a shapefile that I am trying to access:

drwxrwxrwx 2 tomcat7 tomcat7 4096 Feb 26 09:37 .
drwxrwxrwx 7 tomcat7 tomcat7 4096 Aug 5 10:57 …
-rw-rwxrwx 1 tomcat7 tomcat7 47966 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.dbf
-rw-rwxrwx 1 tomcat7 tomcat7 145 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.prj
-rw-rwxrwx 1 tomcat7 tomcat7 1556 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.sbn
-rw-rwxrwx 1 tomcat7 tomcat7 196 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.sbx
-rw-rwxrwx 1 tomcat7 tomcat7 108300 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.shp
-rw-rwxrwx 1 tomcat7 tomcat7 1212 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.shx

Is there something else that needs to be changed?

Regards
Derek

On 6 August 2014 04:55, Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com

mailto:[Ben.Caradoc-Davies@csiro.au](mailto:Ben.Caradoc-Davies@anonymised.com)> wrote:

Derek,

symlinks should work, but note that access depends on the
permissions of the symlink target, not of the symlink itself.

What is the output of this?:

ls -al /mnt/sdi/spatial

If permissions are the problem, you can change the ownership and
permissions of /mnt/sdi/spatial (recursively) with these commands,
but note that this may not be what you want so use with caution:

chown -R tomcat7:tomcat7 /mnt/sdi/spatial
chmod -R ug+rX /mnt/sdi/spatial

Kind regards,
Ben.

On 05/08/14 21:14, Derek wrote:

Hi

We have been struggling to get GeoServer to follow data stored in a
symlinked directory (symlinked from GeoServer’s actual data
directory). All
permissions have been set and the owner/group is set to Tomcat7.
The ls
output shows:

lrwxrwxrwx 1 tomcat7 tomcat7 21 Aug 4 13:03 spatial →
/mnt/sdi/spatial/

and this link and file system can be navigated from the command
line.

If any of the data is copied over from the symlinked directory
to the
actual data directory (in our case, at /var/geoserver/data/),
then it can be
loaded as expected. If not, then we get an error message:

Error creating data store, check the parameters. Error message:
Could not
acquire data access ‘XYZ’

The log show the same message, along with the usual
NullPointerException.

Is there an additional Tomcat or GeoServer configuration that is
required to
access symlinked directories?

Thanks!
Derek


View this message in context:

http://osgeo-org.1560.x6.__nabble.com/Symlinks-for-data-__sub-directories-tp5155119.html

<http://osgeo-org.1560.x6.nabble.com/Symlinks-for-data-sub-directories-tp5155119.html>
Sent from the GeoServer - User mailing list archive at Nabble.com.

------------------------------------------------------------------------------

Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms controls.
Build a bridge from your legacy apps to the future.

http://pubads.g.doubleclick.__net/gampad/clk?id=153845071&__iu=/4140/ostg.clktrk
<http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk>


Geoserver-users mailing list
Geoserver-users@anonymised.comsourceforge.net
mailto:[Geoserver-users@anonymised.comsourceforge.net](mailto:Geoserver-users@anonymised.comsts.sourceforge.net)
https://lists.sourceforge.net/__lists/listinfo/geoserver-users

<https://lists.sourceforge.net/lists/listinfo/geoserver-users>


Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com
Software Engineer
CSIRO Mineral Resources Flagship
Australian Resources Research Centre


Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com
Software Engineer
CSIRO Mineral Resources Flagship
Australian Resources Research Centre

Derek,

selinux is not present and is not your problem.

I tested adding a shapefile using GeoServer 2.5.2, then moving the directory away, and substituting a symlink. The shapefile continued to work. As far as I can tell, you are doing everything right.

The user interface is a bit broken in my browser, but that is a separate issue. It was equally broken with and without the symlink. I had to manually edit the url.

Kind regards,
Ben.

On 06/08/14 19:37, Derek wrote:

Hi Ben

The mount command looks like:

/dev/mapper/san on /mnt/sdi type ext4 (rw,noatime,stripe=2560,data=ordered)

(The network admin tells me this a SAN drive mounted directly to the
server.)

The response from the "Z" parameter - we are running Ubuntu - looks like:

-rw-rwxrwx 1 tomcat7 tomcat7 ? 47966 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.dbf
-rw-rwxrwx 1 tomcat7 tomcat7 ? 145 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.prj
-rw-rwxrwx 1 tomcat7 tomcat7 ? 1556 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.sbn
-rw-rwxrwx 1 tomcat7 tomcat7 ? 196 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.sbx
-rw-rwxrwx 1 tomcat7 tomcat7 ? 108300 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.shp
-rw-rwxrwx 1 tomcat7 tomcat7 ? 1212 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.shx

No datastore.xml has been created for this shapefile. Here's an example
from another one (but this from data already on the host, and not on the
symlinked drive).

<dataStore>
   <id>DataStoreInfoImpl-63f945a9:147a4d4b8b2:-7fff</id>
   <name>Pop</name>
   <description>Pop</description>
   <type>Shapefile</type>
   <enabled>true</enabled>
   <workspace>
     <id>WorkspaceInfoImpl-10073beb:147a14d6b3c:-8000</id>
   </workspace>
   <connectionParameters>
     <entry key="charset">ISO-8859-1</entry>
     <entry key="filetype">shapefile</entry>
     <entry key="create spatial index">true</entry>
     <entry key="memory mapped buffer">false</entry>
     <entry key="timezone">Etc/UTC</entry>
     <entry key="enable spatial index">true</entry>
     <entry key="namespace">sdi</entry>
     <entry key="cache and reuse memory maps">true</entry>
     <entry
key="url">file:ne_10m_admin_0_countries/ne_10m_admin_0_countries.shp</entry>
     <entry key="fstype">shape</entry>
   </connectionParameters>
   <__default>false</__default>
</dataStore>

I attach the relevant portion of the error trace from catalina.out.

Thanks
Derek

On 6 August 2014 09:52, Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com
<mailto:Ben.Caradoc-Davies@anonymised.com>> wrote:

    Derek,

    can you show us the contents of the datastore.xml file in your data
    directory that defines this data store?

    What platform are you using? CentOS / RedHat often have selinux
    enforcing; this additional security mechanism may prevent tomcat7
    from accessing some files. If you have selinux security contexts,
    you can see them with:

    ls -alZ /mnt/sdi/spatial
    ls -alZ /mnt/sdi/spatial/city/planning

    Is this a network mount? What sort of filesystem? ext4, VFAT, or
    NTFS? Try:

    mount | grep /mnt/sdi

    It would also be helpful if you have a stack trace from the
    geoserver logs when this error occurs. This may give us more detail
    about the underlying error.

    Kind regards,
    Ben.

    On 06/08/14 15:26, Derek wrote:

        Hi Ben

        Thanks for the reply. Yes, the permissions look OK. Using your
        command, for example, ls -al /mnt/sdi/spatial/city/__planning, I
        get this
        listing for a directory containing a shapefile that I am trying
        to access:

        drwxrwxrwx 2 tomcat7 tomcat7 4096 Feb 26 09:37 .
        drwxrwxrwx 7 tomcat7 tomcat7 4096 Aug 5 10:57 ..
        -rw-rwxrwx 1 tomcat7 tomcat7 47966 Feb 26 09:37
        CoT_master_sbc_2010_poly___wgs84_1.dbf
        -rw-rwxrwx 1 tomcat7 tomcat7 145 Feb 26 09:37
        CoT_master_sbc_2010_poly___wgs84_1.prj
        -rw-rwxrwx 1 tomcat7 tomcat7 1556 Feb 26 09:37
        CoT_master_sbc_2010_poly___wgs84_1.sbn
        -rw-rwxrwx 1 tomcat7 tomcat7 196 Feb 26 09:37
        CoT_master_sbc_2010_poly___wgs84_1.sbx
        -rw-rwxrwx 1 tomcat7 tomcat7 108300 Feb 26 09:37
        CoT_master_sbc_2010_poly___wgs84_1.shp
        -rw-rwxrwx 1 tomcat7 tomcat7 1212 Feb 26 09:37
        CoT_master_sbc_2010_poly___wgs84_1.shx

        Is there something else that needs to be changed?

        Regards
        Derek

        On 6 August 2014 04:55, Ben Caradoc-Davies
        <Ben.Caradoc-Davies@anonymised.com
        <mailto:Ben.Caradoc-Davies@anonymised.com
        <mailto:Ben.Caradoc-Davies@anonymised.com>>> wrote:

             Derek,

             symlinks should work, but note that access depends on the
             permissions of the symlink target, not of the symlink itself.

             What is the output of this?:

             ls -al /mnt/sdi/spatial

             If permissions are the problem, you can change the
        ownership and
             permissions of /mnt/sdi/spatial (*recursively*) with these
        commands,
             but note that this may not be what you want so use with
        caution:

             chown -R tomcat7:tomcat7 /mnt/sdi/spatial
             chmod -R ug+rX /mnt/sdi/spatial

             Kind regards,
             Ben.

             On 05/08/14 21:14, Derek wrote:

                 Hi

                 We have been struggling to get GeoServer to follow data
        stored in a
                 symlinked directory (symlinked from GeoServer's actual data
                 directory). All
                 permissions have been set and the owner/group is set to
        Tomcat7.
                 The ls
                 output shows:

                 lrwxrwxrwx 1 tomcat7 tomcat7 21 Aug 4 13:03 spatial ->
                 /mnt/sdi/spatial/

                 and this link and file system can be navigated from the
        command
                 line.

                 If any of the data is copied over from the symlinked
        directory
                 to the
                 actual data directory (in our case, at
        /var/geoserver/data/),
                 then it can be
                 loaded as expected. If not, then we get an error message:

                 Error creating data store, check the parameters. Error
        message:
                 Could not
                 acquire data access 'XYZ'

                 The log show the same message, along with the usual
                 NullPointerException.

                 Is there an additional Tomcat or GeoServer
        configuration that is
                 required to
                 access symlinked directories?

                 Thanks!
                 Derek

                 --
                 View this message in context:
        http://osgeo-org.1560.x6.__nab__ble.com/Symlinks-for-data-____sub-directories-tp5155119.html
        <http://nabble.com/Symlinks-for-data-__sub-directories-tp5155119.html&gt;

        <http://osgeo-org.1560.x6.__nabble.com/Symlinks-for-data-__sub-directories-tp5155119.html
        <http://osgeo-org.1560.x6.nabble.com/Symlinks-for-data-sub-directories-tp5155119.html&gt;\_\_&gt;
                 Sent from the GeoServer - User mailing list archive at
        Nabble.com.

        ------------------------------____----------------------------__--__------------------

                 Infragistics Professional
                 Build stunning WinForms apps today!
                 Reboot your WinForms applications with our WinForms
        controls.
                 Build a bridge from your legacy apps to the future.
        http://pubads.g.doubleclick.____net/gampad/clk?id=153845071&____iu=/4140/ostg.clktrk

        <http://pubads.g.doubleclick.__net/gampad/clk?id=153845071&__iu=/4140/ostg.clktrk
        <http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk&gt;&gt;
                 ___________________________________________________
                 Geoserver-users mailing list
                 Geoserver-users@anonymised.com
        <http://sourceforge.net>
                 <mailto:Geoserver-users@anonymised.com
        <mailto:Geoserver-users@lists.sourceforge.net>>
        https://lists.sourceforge.net/____lists/listinfo/geoserver-__users
        <https://lists.sourceforge.net/__lists/listinfo/geoserver-users&gt;

        <https://lists.sourceforge.__net/lists/listinfo/geoserver-__users <https://lists.sourceforge.net/lists/listinfo/geoserver\-users&gt;&gt;

             --
             Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
             Software Engineer
             CSIRO Mineral Resources Flagship
             Australian Resources Research Centre

    --
    Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
    Software Engineer
    CSIRO Mineral Resources Flagship
    Australian Resources Research Centre

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer
CSIRO Mineral Resources Flagship
Australian Resources Research Centre

Hi Ben

Thanks for testing on your side as well. I am a bit stuck here, as moving a large volume of data files across to a machine not meant to store them does not seem like a good option. It also means that we cannot use one GeoServer instance to “point” to multiple locations unless we can get symlinks to work.

Any other ideas or tests I can run?

Thanks
Derek

···

On 7 August 2014 08:19, Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com> wrote:

Derek,

selinux is not present and is not your problem.

I tested adding a shapefile using GeoServer 2.5.2, then moving the directory away, and substituting a symlink. The shapefile continued to work. As far as I can tell, you are doing everything right.

The user interface is a bit broken in my browser, but that is a separate issue. It was equally broken with and without the symlink. I had to manually edit the url.

Kind regards,
Ben.

On 06/08/14 19:37, Derek wrote:

Hi Ben

The mount command looks like:

/dev/mapper/san on /mnt/sdi type ext4 (rw,noatime,stripe=2560,data=ordered)

(The network admin tells me this a SAN drive mounted directly to the
server.)

The response from the “Z” parameter - we are running Ubuntu - looks like:

-rw-rwxrwx 1 tomcat7 tomcat7 ? 47966 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.dbf
-rw-rwxrwx 1 tomcat7 tomcat7 ? 145 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.prj
-rw-rwxrwx 1 tomcat7 tomcat7 ? 1556 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.sbn
-rw-rwxrwx 1 tomcat7 tomcat7 ? 196 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.sbx
-rw-rwxrwx 1 tomcat7 tomcat7 ? 108300 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.shp
-rw-rwxrwx 1 tomcat7 tomcat7 ? 1212 Feb 26 09:37
CoT_master_sbc_2010_poly_wgs84_1.shx

No datastore.xml has been created for this shapefile. Here’s an example
from another one (but this from data already on the host, and not on the
symlinked drive).

DataStoreInfoImpl-63f945a9:147a4d4b8b2:-7fff Pop Pop Shapefile true WorkspaceInfoImpl-10073beb:147a14d6b3c:-8000 ISO-8859-1 shapefile true false Etc/UTC true sdi true file:ne_10m_admin_0_countries/ne_10m_admin_0_countries.shp shape <__default>false

I attach the relevant portion of the error trace from catalina.out.

Thanks
Derek

On 6 August 2014 09:52, Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com

mailto:[Ben.Caradoc-Davies@csiro.au](mailto:Ben.Caradoc-Davies@anonymised.com)> wrote:

Derek,

can you show us the contents of the datastore.xml file in your data
directory that defines this data store?

What platform are you using? CentOS / RedHat often have selinux
enforcing; this additional security mechanism may prevent tomcat7
from accessing some files. If you have selinux security contexts,
you can see them with:

ls -alZ /mnt/sdi/spatial
ls -alZ /mnt/sdi/spatial/city/planning

Is this a network mount? What sort of filesystem? ext4, VFAT, or
NTFS? Try:

mount | grep /mnt/sdi

It would also be helpful if you have a stack trace from the
geoserver logs when this error occurs. This may give us more detail
about the underlying error.

Kind regards,
Ben.

On 06/08/14 15:26, Derek wrote:

Hi Ben

Thanks for the reply. Yes, the permissions look OK. Using your

command, for example, ls -al /mnt/sdi/spatial/city/__planning, I

get this
listing for a directory containing a shapefile that I am trying
to access:

drwxrwxrwx 2 tomcat7 tomcat7 4096 Feb 26 09:37 .
drwxrwxrwx 7 tomcat7 tomcat7 4096 Aug 5 10:57 …
-rw-rwxrwx 1 tomcat7 tomcat7 47966 Feb 26 09:37

CoT_master_sbc_2010_poly___wgs84_1.dbf

-rw-rwxrwx 1 tomcat7 tomcat7 145 Feb 26 09:37

CoT_master_sbc_2010_poly___wgs84_1.prj

-rw-rwxrwx 1 tomcat7 tomcat7 1556 Feb 26 09:37

CoT_master_sbc_2010_poly___wgs84_1.sbn

-rw-rwxrwx 1 tomcat7 tomcat7 196 Feb 26 09:37

CoT_master_sbc_2010_poly___wgs84_1.sbx

-rw-rwxrwx 1 tomcat7 tomcat7 108300 Feb 26 09:37

CoT_master_sbc_2010_poly___wgs84_1.shp

-rw-rwxrwx 1 tomcat7 tomcat7 1212 Feb 26 09:37

CoT_master_sbc_2010_poly___wgs84_1.shx

Is there something else that needs to be changed?

Regards
Derek

On 6 August 2014 04:55, Ben Caradoc-Davies
<Ben.Caradoc-Davies@anonymised.com

<mailto:Ben.Caradoc-Davies@__csiro.au

mailto:[Ben.Caradoc-Davies@csiro.au](mailto:Ben.Caradoc-Davies@anonymised.com)>> wrote:

Derek,

symlinks should work, but note that access depends on the
permissions of the symlink target, not of the symlink itself.

What is the output of this?:

ls -al /mnt/sdi/spatial

If permissions are the problem, you can change the
ownership and
permissions of /mnt/sdi/spatial (recursively) with these
commands,
but note that this may not be what you want so use with
caution:

chown -R tomcat7:tomcat7 /mnt/sdi/spatial
chmod -R ug+rX /mnt/sdi/spatial

Kind regards,
Ben.

On 05/08/14 21:14, Derek wrote:

Hi

We have been struggling to get GeoServer to follow data
stored in a
symlinked directory (symlinked from GeoServer’s actual data
directory). All
permissions have been set and the owner/group is set to
Tomcat7.
The ls
output shows:

lrwxrwxrwx 1 tomcat7 tomcat7 21 Aug 4 13:03 spatial →
/mnt/sdi/spatial/

and this link and file system can be navigated from the
command
line.

If any of the data is copied over from the symlinked
directory
to the
actual data directory (in our case, at
/var/geoserver/data/),
then it can be
loaded as expected. If not, then we get an error message:

Error creating data store, check the parameters. Error
message:
Could not
acquire data access ‘XYZ’

The log show the same message, along with the usual
NullPointerException.

Is there an additional Tomcat or GeoServer
configuration that is
required to
access symlinked directories?

Thanks!
Derek


View this message in context:

http://osgeo-org.1560.x6.__nab__ble.com/Symlinks-for-data-____sub-directories-tp5155119.html
<http://nabble.com/Symlinks-for-data-__sub-directories-tp5155119.html>

<http://osgeo-org.1560.x6.nabble.com/Symlinks-for-data-__sub-directories-tp5155119.html
<http://osgeo-org.1560.x6.nabble.com/Symlinks-for-data-sub-directories-tp5155119.html>
>

Sent from the GeoServer - User mailing list archive at
Nabble.com.

------------------------------____----------------------------------------------

Infragistics Professional
Build stunning WinForms apps today!
Reboot your WinForms applications with our WinForms
controls.
Build a bridge from your legacy apps to the future.

http://pubads.g.doubleclick.____net/gampad/clk?id=153845071&____iu=/4140/ostg.clktrk

<http://pubads.g.doubleclick.__net/gampad/clk?id=153845071&__iu=/4140/ostg.clktrk
<http://pubads.g.doubleclick.net/gampad/clk?id=153845071&iu=/4140/ostg.clktrk>>


Geoserver-users mailing list
Geoserver-users@anonymised.comsource__forge.net
<http://sourceforge.net>
<mailto:Geoserver-users@anonymised.com__sourceforge.net
mailto:[Geoserver-users@anonymised.comsourceforge.net](mailto:Geoserver-users@anonymised.comsts.sourceforge.net)>
https://lists.sourceforge.net/____lists/listinfo/geoserver-__users
<https://lists.sourceforge.net/__lists/listinfo/geoserver-users>

<https://lists.sourceforge.__net/lists/listinfo/geoserver-__users <https://lists.sourceforge.net/lists/listinfo/geoserver-users>>


Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com
Software Engineer
CSIRO Mineral Resources Flagship
Australian Resources Research Centre


Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com
Software Engineer
CSIRO Mineral Resources Flagship
Australian Resources Research Centre


Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com
Software Engineer
CSIRO Mineral Resources Flagship
Australian Resources Research Centre

On Thu, Aug 7, 2014 at 8:36 AM, Derek <gamesbook@anonymised.com> wrote:

Hi Ben

Thanks for testing on your side as well. I am a bit stuck here, as moving
a large volume of data files across to a machine not meant to store them
does not seem like a good option. It also means that we cannot use one
GeoServer instance to "point" to multiple locations unless we can get
symlinks to work.

Any other ideas or tests I can run?

Have you tried avoid using the symliks? GeoServer can access any full path
you're giving to it, what if you point directly to the files?

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

Thanks Andrea

That works! Its less user-friendly, as the user now has to navigate “by hand” to the correct place, but it does work and so we will use this way for now.

I am still curious as to why the symlink method does not work; given that the target data and directories are accessible by this route.

Derek

···

On 7 August 2014 09:23, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, Aug 7, 2014 at 8:36 AM, Derek <gamesbook@anonymised.com> wrote:

Hi Ben

Thanks for testing on your side as well. I am a bit stuck here, as moving a large volume of data files across to a machine not meant to store them does not seem like a good option. It also means that we cannot use one GeoServer instance to “point” to multiple locations unless we can get symlinks to work.

Any other ideas or tests I can run?

Have you tried avoid using the symliks? GeoServer can access any full path
you’re giving to it, what if you point directly to the files?

Cheers
Andrea

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it


On Fri, Aug 8, 2014 at 8:25 AM, Derek <gamesbook@anonymised.com> wrote:

Thanks Andrea

That works! Its less user-friendly, as the user now has to navigate "by
hand" to the correct place, but it does work and so we will use this way
for now.

Good to hear

I am still curious as to why the symlink method does not work; given that
the target data and directories are accessible by this route.

Hum... don't know, Ben tried on his machine and it worked. Must be
something specific to your environment, which means we'd
need some developer debugging though the code (possibly the JDK own code)
to figure it out

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------