Hello,
2009/11/19 Craig Jones <jonescc@anonymised.com>:
> Hi Michael,
>
> Not sure why the internal error occurs. Doesn't occur in my instance
> when I get the mysql unique constraint error. I had assumed it was
> caused by the unique constraint error, but perhaps not. Perhaps someone
> else can comment.
>
> Checking the source for the harvester, I notice that the uuid for the
> service metadata is derived from the url for the service.
The UUID for a service is derived from the full GetCapabilities URL
(including connect point, version and type) and not only the service
connect point which could be the same for different types of services
for example.
The basic idea making UUID based on GetCapabilities URL was to be able
to do an update of the metadata record instead of delete/insert
operation. Using this approach, a merge mechanism could have been
implemented to have both automatically generated content and user
editing (even if not planned yet).
> So if you
> have more than one harvester setup for the same service (url) you will
> get the unique constraint error. Probably not unreasonable, but perhaps
> the error could be a little clearer.
Service metadata uuid should not be the source of these unique constraint error.
Ciao.
Francois
> Cheers,
> CraigJ
>
> On Fri, 2009-11-13 at 14:02 +0800, Michael.Stegherr@anonymised.com wrote:
>> Hi Craig,
>>
>> Thanks a lot for the explanation!
>>
>> This sounds reasonable. My colleague deleted all metadata records and setup harvesting again and is not able to reproduce the problem anymore. So I'm guessing the metadata record with the duplicate uuid has been deleted.
>>
>> There is a question remaining though, why all the other harvesting tasks where affected as well with an 'InternalError'.
>>
>> Anyway, thanks for clearing that up!
>>
>> Cheers,
>> Michael
>>
>> > -----Original Message-----
>> > From: Craig Jones [mailto:jonescc@anonymised.com]
>> > Sent: Wednesday, 11 November 2009 1:42 PM
>> > To: Stegherr, Michael (E&M, Kensington)
>> > Cc: geonetwork-users@lists.sourceforge.net
>> > Subject: Re: [GeoNetwork-users] OgcWxS Harvesting fails
>> > (MySQLIntegrityConstraintViolationException and InternalError)
>> >
>> > Hi Michael/Wolfgang,
>> >
>> > I don't think the thumbnail errors are relevant. Geonetwork tries to
>> > remove thumbnails when harvesting from OgcWxS regardless of whether
>> > there were any there already or not - causing the error. It should
>> > probably be changed to check first before trying to delete them.
>> >
>> > The mysql error is probably occurring because there is
>> > another metadata
>> > record harvested from another source or manually created in the
>> > geonetwork instance with the same uuid.
>> >
>> > Geonetwork 2.2 used to tolerate this, but this was changed in 2.2 in
>> > r1311 and in 2.4 via trunk in r1723. Previously only uuid, source and
>> > harvestid had to be unique now uuid by itself must be unique.
>> >
>> > I assume this was meant to ensure only one metadata record could exist
>> > in geonetwork for a given uuid. It requires uuids to actually BE
>> > unique across all sources and different metadata standards which may
>> > actually be a bit much to ask. You may have discovered this.
>> >
>> > Cheers,
>> > --
>> > Craig Jones
>> > eMII Infrastructure Programmer
>> > IMOS e-Marine Information Infrastructure Facility (eMII)
>> > Ph: +61 3 6226 8567
>> >
>> >
>> >
>> > On Wed, 2009-11-11 at 12:22 +0800, Michael.Stegherr@anonymised.com wrote:
>> > > Hi,
>> > >
>> > > I forgot to mention that I'm running GeoNetwork 2.4.1 on
>> > Debian Lenny with MySQL as well.
>> > >
>> > > Cheers,
>> > > Michael
>> > >
>> > > > -----Original Message-----
>> > > > From: Wolfgang Grunberg [mailto:wgrunberg@anonymised.com]
>> > > > Sent: Wednesday, 11 November 2009 6:53 AM
>> > > > To: geonetwork-users@lists.sourceforge.net
>> > > > Subject: Re: [GeoNetwork-users] OgcWxS Harvesting fails
>> > > > (MySQLIntegrityConstraintViolationException and InternalError)
>> > > >
>> > > > I have also observed the same "Failed to remove thumbnail for
>> > > > metadata"
>> > > > error on a new GeoNetwork 2.4.2 instance on Ubuntu with MySQL.
>> > > > Could this be some kind of permissions problem?
>> > > > Ciao, Wolfgang
>> > > >
>> > > > _______________________________
>> > > > Wolfgang Grunberg
>> > > > Arizona Geological Survey
>> > > > wgrunberg@anonymised.com
>> > > > 520-770-3500
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > Michael.Stegherr@anonymised.com wrote:
>> > > > > Hi all,
>> > > > >
>> > > > > Our GeoNetwork node is harvesting a couple of WFS/WMSes.
>> > > > Recently all the harvests fail and a restart of the
>> > > > GeoNetwork servlet didn't seem to change anything.
>> > > > >
>> > > > > As I couldn't find the errors in the mailing list archives
>> > > > or on Trac I hope that someone can shed a light on this.
>> > > > Extracts from the log file are attached below.
>> > > > >
>> > > > > Thanks,
>> > > > > Michael
>> > > > >
>> > > > >
>> > > > > 2009-11-02 17:05:20,344 ERROR [geonetwork.harvester] - -
>> > > > Failed to remove thumbnail for metadata: 30295
>> > > > > 2009-11-02 17:05:20,413 ERROR [geonetwork.harvester] - -
>> > > > Failed to remove thumbnail for metadata: 30294
>> > > > > 2009-11-02 17:05:20,420 ERROR [geonetwork.harvester] - -
>> > > > Failed to remove thumbnail for metadata: 30296
>> > > > > 2009-11-02 17:05:23,071 WARN [geonetwork.harvester] - -
>> > > > Failed to extract layer bbox from metadata : empty String
>> > > > > 2009-11-02 17:05:23,159 WARN [geonetwork.harvester] - -
>> > > > Failed to extract layer bbox from metadata : empty String
>> > > > > 2009-11-02 17:05:23,193 WARN [geonetwork.harvester] - -
>> > > > Failed to extract layer bbox from metadata : empty String
>> > > > > 2009-11-02 17:05:23,204 WARN [geonetwork.harvester] -
>> > > > Raised exception while harvesting from : North Ryde NVCL Web
>> > > > Services (OgcWxSHarvester)
>> > > > > 2009-11-02 17:05:23,204 WARN [geonetwork.harvester] - (C)
>> > > > Class : MySQLIntegrityConstraintViolationException
>> > > > > 2009-11-02 17:05:23,204 WARN [geonetwork.harvester] - (C)
>> > > > Message : Duplicate entry
>> > > > 'e615fe32f7d70caae3f94a91daee2b479ff593' for key 2
>> > > > > 2009-11-02 17:05:38,059 INFO [jeeves.request] -
>> > > > ==========================================================
>> > > > > 2009-11-02 17:05:38,059 INFO [jeeves.request] - HTML
>> > > > Request (from 130.116.145.28) :
>> > /geonetwork/srv/en/xml.harvesting.run
>> > > > > 2009-11-02 17:05:38,059 DEBUG [jeeves.request] - Method
>> > : POST
>> > > > > 2009-11-02 17:05:38,059 DEBUG [jeeves.request] - Content
>> > > > type : application/xml; charset=UTF-8
>> > > > > 2009-11-02 17:05:38,059 DEBUG [jeeves.request] - Accept
>> > > > : text/javascript, text/html, application/xml, text/xml, */*
>> > > > > 2009-11-02 17:05:38,059 DEBUG [jeeves.request] - Session id
>> > > > is 20A886452C2E779CE9874E2A403E65E6
>> > > > > 2009-11-02 17:05:38,061 INFO [jeeves.service] -
>> > > > Dispatching : xml.harvesting.run
>> > > > > 2009-11-02 17:05:38,061 DEBUG [jeeves.service] - ->
>> > > > parameters are :
>> > > > > <request>
>> > > > > <id>743</id>
>> > > > > </request>
>> > > > > 2009-11-02 17:05:38,062 INFO [jeeves.service] - ->
>> > > > dispatching to output for : xml.harvesting.run
>> > > > > 2009-11-02 17:05:38,062 INFO [jeeves.service] - ->
>> > > > writing xml for : xml.harvesting.run
>> > > > > 2009-11-02 17:05:38,062 DEBUG [jeeves.service] -
>> > Service xml is :
>> > > > > <response>
>> > > > > <id status="ok">743</id>
>> > > > > </response>
>> > > > > 2009-11-02 17:05:38,062 INFO [jeeves.service] - ->
>> > > > output ended for : xml.harvesting.run
>> > > > > 2009-11-02 17:05:38,062 INFO [jeeves.service] - ->
>> > > > dispatch ended for : xml.harvesting.run
>> > > > >
>> > > > > [...]
>> > > > >
>> > > > > 2009-11-02 17:05:39,923 ERROR [geonetwork.harvester] - -
>> > > > Failed to remove thumbnail for metadata: 30298
>> > > > > 2009-11-02 17:05:40,104 WARN [geonetwork.harvester] -
>> > > > Raised exception while harvesting from : AuScope-TestWMS
>> > > > (OgcWxSHarvester)
>> > > > > 2009-11-02 17:05:40,104 WARN [geonetwork.harvester] - (C)
>> > > > Class : InternalError
>> > > > > 2009-11-02 17:05:40,104 WARN [geonetwork.harvester] - (C)
>> > > > Message : randomStream == null
>> > > > > 2009-11-02 17:05:44,739 INFO [jeeves.request] -
>> > > > ==========================================================
>> > > > > 2009-11-02 17:05:44,739 INFO [jeeves.request] - HTML
>> > > > Request (from 130.116.145.28) :
>> > /geonetwork/srv/en/xml.harvesting.get
>> > > > > 2009-11-02 17:05:44,739 DEBUG [jeeves.request] - Method
>> > : POST
>> > > > > 2009-11-02 17:05:44,739 DEBUG [jeeves.request] - Content
>> > > > type : application/xml; charset=UTF-8
>> > > > > 2009-11-02 17:05:44,739 DEBUG [jeeves.request] - Accept
>> > > > : text/javascript, text/html, application/xml, text/xml, */*
>> > > > > 2009-11-02 17:05:44,739 DEBUG [jeeves.request] - Session id
>> > > > is 20A886452C2E779CE9874E2A403E65E6
>> > > > > 2009-11-02 17:05:44,740 INFO [jeeves.service] -
>> > > > Dispatching : xml.harvesting.get
>> > > > > 2009-11-02 17:05:44,740 DEBUG [jeeves.service] - -> no
>> > > > input parameters
>> > > > > 2009-11-02 17:05:44,748 INFO [jeeves.service] - ->
>> > > > dispatching to output for : xml.harvesting.get
>> > > > > 2009-11-02 17:05:44,748 INFO [jeeves.service] - ->
>> > > > writing xml for : xml.harvesting.get
>> > > > > 2009-11-02 17:05:44,749 DEBUG [jeeves.service] -
>> > Service xml is :
>> > > > >
>> > > > > [...]
>> > > > >
>> > > > >
>> > > >
>> > > > --------------------------------------------------------------
>> > > > ----------------
>> > > > Let Crystal Reports handle the reporting - Free Crystal
>> > > > Reports 2008 30-Day
>> > > > trial. Simplify your report design, integration and
>> > > > deployment - and focus on
>> > > > what you do best, core application coding. Discover
>> > what's new with
>> > > > Crystal Reports now. http://p.sf.net/sfu/bobj-july
>> > > > _______________________________________________
>> > > > GeoNetwork-users mailing list
>> > > > GeoNetwork-users@lists.sourceforge.net
>> > > > https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>> > > > GeoNetwork OpenSource is maintained at
>> > > > http://sourceforge.net/projects/geonetwork
>> > > >
>> > >
>> > --------------------------------------------------------------
>> > ----------------
>> > > Let Crystal Reports handle the reporting - Free Crystal
>> > Reports 2008 30-Day
>> > > trial. Simplify your report design, integration and
>> > deployment - and focus on
>> > > what you do best, core application coding. Discover what's new with
>> > > Crystal Reports now. http://p.sf.net/sfu/bobj-july
>> > > _______________________________________________
>> > > GeoNetwork-users mailing list
>> > > GeoNetwork-users@lists.sourceforge.net
>> > > https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>> > > GeoNetwork OpenSource is maintained at
>> > http://sourceforge.net/projects/geonetwork
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
>> trial. Simplify your report design, integration and deployment - and focus on
>> what you do best, core application coding. Discover what's new with
>> Crystal Reports now. http://p.sf.net/sfu/bobj-july
>> _______________________________________________
>> GeoNetwork-users mailing list
>> GeoNetwork-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>> GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> _______________________________________________
> GeoNetwork-users mailing list
> GeoNetwork-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
> GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
>
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork