[SAC] [OSGeo] #607: Maven Deploy fails with 405 Error

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------
GeoTools uses an automated build system (ie maven) which during the
release process deploys new artifacts to
http://download.osgeo.org/webdav/geotools/

We are treating this webdav directory as a maven repository maintains both
the geotools library for downloads; but also little xml files which tells
the build system how to download it safely and get a consistent set of
jars that work together. Think something similar to Perl CPAN.

Each developer configures maven with their osgeoid and credentials and
those credentials are used during the deploy process.

Here is the failure:

[INFO] Error deploying artifact: Failed to transfer file:
http://download.osgeo.org/webdav/geotools/org/geotools/geotools/2.7-M2/geotools-2.7-M2.pom.
Return code is: 405 Method Not Allowed

This issue is preventing the release of GeoTools 2.7-M2.

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by warmerdam):

Jody,

The download site was compromised, and we have had to redirect
download.osgeo.org to a backup server. Special site configuration, such
as webdav, was lost in this move. We are hoping to have the download
blade scrubbed, reinstalled and setup in the next few days at which time
we will also need to setup the webdav support again - potentially with
some help. I'll leave this ticket open to address that point.

Are you aware of a previous ticket or other documentation on how to setup
the WebDAV support?

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:1&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by jive):

The only other issue that mentions this is #344

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:2&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: normal | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by wildintellect):

Webdav can be entirely handled by Apache (in fact svn setups commonly use
webdav)
http://httpd.apache.org/docs/2.2/mod/mod_dav.html

Do we have a copy of the apache config from the prior download server that
we might be able to use as a template?

On a debian based system to enable the modules (likely already on the
system, easy to apt-get if not)
Add the appropriate section like the example in the docs then:
sudo a2enmod dav
sudo a2enmod dav_fs
and apache reload

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:3&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------
Changes (by bencaradocdavies):

  * priority: normal => blocker

Comment:

Any progress on this issue? I am also unable to deploy to
download.osgeo.org via webdav. I get the same error message:
[INFO] Error deploying artifact: Failed to transfer file:
http://download.osgeo.org/webdav/geotools/org/earthresourceml/earthresourceml-1.1/1.1.0-1/earthresourceml-1.1-1.1.0-1.jar.
Return code is: 405 Method Not Allowed

This now blocks development, so I have set priority to blocker.

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:4&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by wildintellect):

So I did:
{{{
sudo a2enmod dav
sudo a2enmod dav_fs
sudo a2enmod ldap
}}}
* Copied /etc/apache2/conf.d/ldap_auth_url.inc from the svn server
* Created /etc/apache2/conf.d/webdav/geotools.conf
* Edited /etc/apache2/sites-available/download.osgeo.org.conf to include
the conf.d/webdav/*.conf

I have not restart apache2 for these to take effect yet, can someone
please double check my edits so we can move to testing.

Thanks - Alex

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:5&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by wildintellect):

Moved the config files to include.d since conf.d was being autoloaded in
the wrong order. Appears ready for testing. Please try it and let us know
if it works.

Would like to consider moving all webdav to https. Will maven have an
issue with https?

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/607#comment:6&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by bencaradocdavies):

OK, the 405 looks fixed. I am now getting an access denied error, so
perhaps my credentials have not been enabled.

INFO] Error deploying artifact: Authorization failed: Access denied to:
http://download.osgeo.org/webdav/geotools/org/earthresourceml/earthresourceml-1.1/1.1.0-1/earthresourceml-1.1-1.1.0-1.jar

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:7&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by jive):

Maven can support a range of things FTP, SCP, SFTP (more details here
http://maven.apache.org/wagon/). I don't see https on their list. Looks
like we are using webdav from here (http://maven.apache.org/wagon/wagon-
providers/wagon-webdav/).
Reading further(
http://docs.codehaus.org/display/MAVENUSER/Deploying+3rd+Party+Jars+With+WebDAV)
I could see no examples using https.

Okay finally found an example:
http://docs.codehaus.org/display/HAUSMATES/Maven+Guide

Uses:

    <site>
       <id>codehaus.org</id>
       <url>dav:https://dav.codehaus.org/xfire/&lt;/url&gt;
     </site>

And has interesting steps required to install a certificate.

It would take us some time to sort this out as we would need to update 30
pom.xml files; but yeah it looks like https would work. If SFTP or one of
the other protocols is better suited I think it would be more suited for
our needs (as it is a tested wagon implementation).

I think we only did webdav as it was something quick for Paul Ramsey to
setup at refractions.

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:8&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by bencaradocdavies):

Still getting authorization failed errors. Was apache restarted? Are LDAP
queries being appearing on the LDAP server?

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:9&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by wildintellect):

Replying to [comment:9 bencaradocdavies]:
> Still getting authorization failed errors. Was apache restarted? Are
LDAP queries being appearing on the LDAP server?

The apache server error log shows "user oliveros not found". Is this user
in the geotools group of the LDAP?

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:10&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by bencaradocdavies):

oliveros is not a member of our group.

You should see some failures from bencaradocdavies and jive, who have both
seen authorization failures while deploying from maven.

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:11&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by wildintellect):

Replying to [comment:11 bencaradocdavies]:
> oliveros is not a member of our group.
>
> You should see some failures from bencaradocdavies and jive, who have
both seen authorization failures while deploying from maven.

Ok I see those in the Access log, not in the error log. Looks like it's
returning a 403. My best guess is that apache doesn't have write
permission on the geotools folder on the server. Should we change the
owner or group and to what?

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/607#comment:12&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by aaime):

Quick note here. I did not read all of the above, but I can login into the
dav properly (did not check writing though). The thing is, everybody seems
to have to login to access the DAV, even normal maven users that just want
to download libraries from the repo.
The repo should be setup wide open for read only access instead.

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:13&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by bencaradocdavies):

I concur with aaime. The geotools maven repo (
http://download.osgeo.org/webdav/geotools/ ) must be world-readable. If I
follow this link in a browser (i.e. no authentication) I get a "401
Authorization Required" response. This will cause all nonprivileged users
to be unable to use the repository.

Uploads are still failing as well.

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:14&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by mcoudert):

As aaime and bencaradocdavies reported few minutes ago, it causes errors
when building maven projects relying on geotools repository.

{{{
[WARNING] Unable to get resource 'org.geotools:gt-epsg-
extension:jar:2.6.2' from
  repository geotools (http://download.osgeo.org/webdav/geotools/):
Authorization
  failed: Access denied to:
http://download.osgeo.org/webdav/geotools/org/geotool
s/gt-epsg-extension/2.6.2/gt-epsg-extension-2.6.2.jar
}}}

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:15&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: new
Priority: blocker | Milestone:
Component: Systems Admin | Keywords:
---------------------------+------------------------------------------------

Comment(by wildintellect):

Replying to [comment:14 bencaradocdavies]:
> I concur with aaime. The geotools maven repo (
http://download.osgeo.org/webdav/geotools/ ) must be world-readable. If I
follow this link in a browser (i.e. no authentication) I get a "401
Authorization Required" response. This will cause all nonprivileged users
to be unable to use the repository.
>
> Uploads are still failing as well.

It is now world readable using a LIMITEXCEPT statement in apache
{{{
<LimitExcept GET PROPFIND OPTIONS REPORT>
}}}
Some tweaks may be required. I still do not have a solution to the write
problem, though I suspect it's system permission related not an LDAP
issue.

--
Ticket URL: <https://trac.osgeo.org/osgeo/ticket/607#comment:16&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: closed
Priority: blocker | Milestone:
Component: Systems Admin | Resolution: fixed
Keywords: |
---------------------------+------------------------------------------------
Changes (by wildintellect):

  * status: new => closed
  * resolution: => fixed

Comment:

Replying to [comment:16 wildintellect]:
> Replying to [comment:14 bencaradocdavies]:
> > I concur with aaime. The geotools maven repo (
http://download.osgeo.org/webdav/geotools/ ) must be world-readable. If I
follow this link in a browser (i.e. no authentication) I get a "401
Authorization Required" response. This will cause all nonprivileged users
to be unable to use the repository.
> >
> > Uploads are still failing as well.
>
> It is now world readable using a LIMITEXCEPT statement in apache
> {{{
> <LimitExcept GET PROPFIND OPTIONS REPORT>
> }}}
> Some tweaks may be required. I still do not have a solution to the write
problem, though I suspect it's system permission related not an LDAP
issue.

Changed ownership of the folder to www-data (the apache user) and aaime
was able to write. Need some others to test, and to make sure people who
don't have permission can not write.

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:17&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: closed
Priority: blocker | Milestone:
Component: Systems Admin | Resolution: fixed
Keywords: |
---------------------------+------------------------------------------------

Comment(by jive):

woot - I can confirm that geotools can be released again. Deploying 2.7-M2
now.

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:18&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.

#607: Maven Deploy fails with 405 Error
---------------------------+------------------------------------------------
Reporter: jive | Owner: sac@…
     Type: defect | Status: closed
Priority: blocker | Milestone:
Component: Systems Admin | Resolution: fixed
Keywords: |
---------------------------+------------------------------------------------

Comment(by bencaradocdavies):

I can confirm that I am able to successfully upload (deploy) a new maven
artifact:

[INFO] [deploy:deploy-file]
Uploading:
http://download.osgeo.org/webdav/geotools//org/earthresourceml/earthresourceml-1.1/1.1.0-1/earthresourceml-1.1-1.1.0-1.jar
[INFO] Uploading project information for earthresourceml-1.1 1.1.0-1
[INFO] Retrieving previous metadata from osgeo
[INFO] repository metadata for: 'artifact
org.earthresourceml:earthresourceml-1.1' could not be found on repository:
osgeo, so will be created
[INFO] Uploading repository metadata for: 'artifact
org.earthresourceml:earthresourceml-1.1'

Furthermore, if I remove my maven credentials for this repo and attempt to
redeploy, the operation fails with a 401. This is a success as it
demonstrates that credentials are required to write:

[INFO] Error deploying artifact: Failed to transfer file:
http://download.osgeo.org/webdav/geotools/org/earthresourceml/earthresourceml-1.1/1.1.0-1/earthresourceml-1.1-1.1.0-1.jar.
Return code is: 401 Unauthorized

Thanks a lot! Looks like it is all working.

--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/607#comment:19&gt;
OSGeo <http://www.osgeo.org/&gt;
OSGeo committee and general foundation issue tracker.