[Geoserver-devel] Datestamped GeoTools artifacts from opengeo repo while building GeoServer?

Ben_Caradoc-Davies · March 25, 2009, 5:29am

I build my own local GeoTools trunk and then build GeoServer trunk. Recently I noticed that, while building GeoServer, maven downloads what look like datestamped GeoTools snapshot artifacts, even though, as far as I know, I do not need them to build GeoServer trunk, which depends on GeoTools artifacts 2.6-SNAPSHOT. For example:

Downloading: http://repo.opengeo.org//org/geotools/gt-main/2.6-SNAPSHOT/gt-main-2.6-20090325.063156-27.jar
1159K downloaded

Because I am building my own GeoTools trunk artifacts (2.6-SNAPSHOT) and have installed these fresh jars in my local repository, I do not need nor did I expect to download GeoTools artifacts from the maven repositories. All my Eclipse dependencies are on non-datestamped artifacts, for example:
M2_REPO/org/geotools/gt-main/2.6-SNAPSHOT/gt-main-2.6-SNAPSHOT.jar

Could someone please explain to me:
(1) Why is maven getting GeoTools artifacts that are not a dependency of the GeoServer projects that I am building? Is it because they are installed in the 2.6-SNAPSHOT directory on the remote repo?
(2) Who is installing these artifacts, and why?
(3) Is this a correct use of the maven repository structure?

Downloading these unnecessary artifacts slows my build process, and wastes network and disk resources. I got one batch for the 24th, now one for the 25th. Where does it end?

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Justin_Deoliveira · March 25, 2009, 5:58am

The joys of maven :). Trust me, I feel your pain.

That said, something seems a miss. If you build geotools locally and then build geoserver there should be no external downloads... there should only be a check of the timestamps, in which it should find your local copy newer and prevent the download.

I do this more or less daily to try avoid the downloads like you are doing, and it always work. I actually build offline (-o) just to make sure i don't get any downloads. So something must be off.

I am curious, when does the geotools main dependency get dragged in and downloaded? At which module?

Some additional comments inline.

Ben Caradoc-Davies wrote:

I build my own local GeoTools trunk and then build GeoServer trunk. Recently I noticed that, while building GeoServer, maven downloads what look like datestamped GeoTools snapshot artifacts, even though, as far as I know, I do not need them to build GeoServer trunk, which depends on GeoTools artifacts 2.6-SNAPSHOT. For example:

Downloading: http://repo.opengeo.org//org/geotools/gt-main/2.6-SNAPSHOT/gt-main-2.6-20090325.063156-27.jar
1159K downloaded

Because I am building my own GeoTools trunk artifacts (2.6-SNAPSHOT) and have installed these fresh jars in my local repository, I do not need nor did I expect to download GeoTools artifacts from the maven repositories. All my Eclipse dependencies are on non-datestamped artifacts, for example:
M2_REPO/org/geotools/gt-main/2.6-SNAPSHOT/gt-main-2.6-SNAPSHOT.jar

Could someone please explain to me:
(1) Why is maven getting GeoTools artifacts that are not a dependency of the GeoServer projects that I am building? Is it because they are installed in the 2.6-SNAPSHOT directory on the remote repo?
(2) Who is installing these artifacts, and why?

We have a nightly script that does this so that people don't have to check out and build geotools to build geoserver. One thing you can do to avoid downloading is just comment out the opengeo repository from the root geoserver pom of your checkout. Or use settings.xml to override it.

(3) Is this a correct use of the maven repository structure?

Yup, afaik. WE just run mvn deploy:deploy and it handles creating the proper repository structure.

Downloading these unnecessary artifacts slows my build process, and wastes network and disk resources. I got one batch for the 24th, now one for the 25th. Where does it end?

Kind regards,

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Ben_Caradoc-Davies · March 25, 2009, 6:14am

Justin Deoliveira wrote:

The joys of maven :). Trust me, I feel your pain.
That said, something seems a miss. If you build geotools locally and then build geoserver there should be no external downloads... there should only be a check of the timestamps, in which it should find your local copy newer and prevent the download.

That is my understanding. Until today.

I do this more or less daily to try avoid the downloads like you are doing, and it always work. I actually build offline (-o) just to make sure i don't get any downloads.

That is a pain. I want to make sure I have up-to-date dependencies.

So something must be off.

Looks like it.

If those are meant to be UTC timestamps after the date, they are in the future. Would maven honour an artifact that claimed to be from the future, and thus authoritative over locally built and correctly timestamped artifacts? My local GeoServer build was at around 05:15 UTC, and the artifacts downloaded at this time appear be stamped after 06:30 UTC.

The metadata suggests that the trailing digits are HHMMSS. Are they UTC?
http://repo.opengeo.org//org/geotools/gt-main/2.6-SNAPSHOT/maven-metadata.xml

Do we have a rogue build box that is claiming to be from the future?

I am curious, when does the geotools main dependency get dragged in and downloaded? At which module?

All of them. I get a snapshot downloaded for every GeoTools artifact. And I just realised that the downloaded jar is copied over my local gt-whatever-2.6-SNAPSHOT.jar, so I lose all my local changes.

Who manages the machine that builds and deploys?

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Ben_Caradoc-Davies · March 26, 2009, 2:51am

Ben Caradoc-Davies wrote:

I build my own local GeoTools trunk and then build GeoServer trunk. Recently I noticed that, while building GeoServer, maven downloads what look like datestamped GeoTools snapshot artifacts, even though, as far as I know, I do not need them to build GeoServer trunk, which depends on GeoTools artifacts 2.6-SNAPSHOT. For example:
Downloading: http://repo.opengeo.org//org/geotools/gt-main/2.6-SNAPSHOT/gt-main-2.6-20090325.063156-27.jar
1159K downloaded

Today my GeoServer build worked fine and honoured my locally built GeoTools artifacts. It did not download any of the daily snapshots. Whatever was causing my problem has gone away ...

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Ben_Caradoc-Davies · March 26, 2009, 2:54am

Ben Caradoc-Davies wrote:

Ben Caradoc-Davies wrote:

I build my own local GeoTools trunk and then build GeoServer trunk. Recently I noticed that, while building GeoServer, maven downloads what look like datestamped GeoTools snapshot artifacts, even though, as far as I know, I do not need them to build GeoServer trunk, which depends on GeoTools artifacts 2.6-SNAPSHOT. For example:
Downloading: http://repo.opengeo.org//org/geotools/gt-main/2.6-SNAPSHOT/gt-main-2.6-20090325.063156-27.jar
1159K downloaded

Today my GeoServer build worked fine and honoured my locally built GeoTools artifacts. It did not download any of the daily snapshots. Whatever was causing my problem has gone away ...

Perhaps it is working because no GeoTools snapshot was built overnight?

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Ben_Caradoc-Davies · March 27, 2009, 4:52am

Ben Caradoc-Davies wrote:

Ben Caradoc-Davies wrote:

Ben Caradoc-Davies wrote:

I build my own local GeoTools trunk and then build GeoServer trunk. Recently I noticed that, while building GeoServer, maven downloads what look like datestamped GeoTools snapshot artifacts, even though, as far as I know, I do not need them to build GeoServer trunk, which depends on GeoTools artifacts 2.6-SNAPSHOT. For example:
Downloading: http://repo.opengeo.org//org/geotools/gt-main/2.6-SNAPSHOT/gt-main-2.6-20090325.063156-27.jar
1159K downloaded

Today my GeoServer build worked fine and honoured my locally built GeoTools artifacts. It did not download any of the daily snapshots. Whatever was causing my problem has gone away ...

Perhaps it is working because no GeoTools snapshot was built overnight?

This problem still exists. I think the timestamps are wrong. These problems occur after noon my time.

The snapshots have a local timestamp of 11pm. 11pm EDT = 3am UTC (assuming this machine honours daylight saving.) However the maven metadata marks these as being updated at 6:30am UTC, 3 and a half hours in the future. The files are uploaded at about noon my time, so if at any time in the next three and a half hours I make a local GeoTools build, when I run GeoServer the older snapshots on the remote repo will claim to be newer.

- Could I be right?

- Who is responsible for timestamping these snapshots?

- Where do I lodge a Jira issue?

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

jive · March 27, 2009, 5:38am

Lodge to geotools; sounds like a server is configured incorrectly.
Boy maven does bring home the complications of international development.

Jody

On Fri, Mar 27, 2009 at 3:52 PM, Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com wrote:

Ben Caradoc-Davies wrote:

Ben Caradoc-Davies wrote:

Ben Caradoc-Davies wrote:

I build my own local GeoTools trunk and then build GeoServer trunk.
Recently I noticed that, while building GeoServer, maven downloads what
look like datestamped GeoTools snapshot artifacts, even though, as far
as I know, I do not need them to build GeoServer trunk, which depends on
GeoTools artifacts 2.6-SNAPSHOT. For example:
Downloading:
http://repo.opengeo.org//org/geotools/gt-main/2.6-SNAPSHOT/gt-main-2.6-20090325.063156-27.jar
1159K downloaded
Today my GeoServer build worked fine and honoured my locally built
GeoTools artifacts. It did not download any of the daily snapshots.
Whatever was causing my problem has gone away …

Perhaps it is working because no GeoTools snapshot was built overnight?

This problem still exists. I think the timestamps are wrong. These
problems occur after noon my time.

The snapshots have a local timestamp of 11pm. 11pm EDT = 3am UTC
(assuming this machine honours daylight saving.) However the maven
metadata marks these as being updated at 6:30am UTC, 3 and a half hours
in the future. The files are uploaded at about noon my time, so if at
any time in the next three and a half hours I make a local GeoTools
build, when I run GeoServer the older snapshots on the remote repo will
claim to be newer.

Could I be right?

Who is responsible for timestamping these snapshots?

Where do I lodge a Jira issue?

Kind regards,

–
Ben Caradoc-Davies Ben.Caradoc-Davies@anonymised.com
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia

Geotools-devel mailing list
Geotools-devel@anonymised.coms.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

Andrea_Aime3 · March 27, 2009, 7:38am

Ben Caradoc-Davies ha scritto:

This problem still exists. I think the timestamps are wrong. These problems occur after noon my time.

I just logged in our Hudson/nightly build box and I can confirm
there is a problem. Now in Italy it's 8:36am, meaning the NY time
is 3.36am, yet gridlock.openplans.org reports:
[aaime@anonymised.com ~]$ date --utc
Fri Mar 27 11:17:18 UTC 2009
[aaime@anonymised.com ~]$ date
Fri Mar 27 07:17:23 EDT 2009

So it seems to be roughly 4 hours in the future.
I don't know how the machine is setup, if the time is manually
set or if there is a connection with a rogue NTP server somewhere,
so I'll just pass down the info to the admins in NY when they
get around.

I'll let you know.
Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Justin_Deoliveira · March 27, 2009, 4:05pm

Arne, did you fix this? I just logged into gridlock and the time seems ok to me.

Andrea Aime wrote:

Ben Caradoc-Davies ha scritto:

This problem still exists. I think the timestamps are wrong. These problems occur after noon my time.

I just logged in our Hudson/nightly build box and I can confirm
there is a problem. Now in Italy it's 8:36am, meaning the NY time
is 3.36am, yet gridlock.openplans.org reports:
[aaime@anonymised.com ~]$ date --utc
Fri Mar 27 11:17:18 UTC 2009
[aaime@anonymised.com ~]$ date
Fri Mar 27 07:17:23 EDT 2009

So it seems to be roughly 4 hours in the future.
I don't know how the machine is setup, if the time is manually
set or if there is a connection with a rogue NTP server somewhere,
so I'll just pass down the info to the admins in NY when they
get around.

I'll let you know.
Cheers
Andrea

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

Andrea_Aime3 · March 27, 2009, 4:14pm

Justin Deoliveira ha scritto:

Arne, did you fix this? I just logged into gridlock and the time seems ok to me.

Yeah, Arne fixed it. I wanted to reply to this thread but I thought
to let some hours pass to make sure the clock did not go
bye bye again. As you just confirmed, it didn't

Cheers
Andrea

--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.

Ben_Caradoc-Davies · March 30, 2009, 2:43am

Andrea Aime wrote:

Justin Deoliveira ha scritto:

Arne, did you fix this? I just logged into gridlock and the time seems ok to me.

Yeah, Arne fixed it. I wanted to reply to this thread but I thought
to let some hours pass to make sure the clock did not go
bye bye again. As you just confirmed, it didn't

Thanks!

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineer, CSIRO Exploration and Mining
Australian Resources Research Centre
26 Dick Perry Ave, Kensington WA 6151, Australia