Hello
I'm trying to migrate my GN 2.10.2 installation to the last release (3.0.2)
This process is also synchronized with a server change: new server with debian 8 and tomcat8
I installed GN by copying the geonetwork.war to the tomcat8 webapps dir.
After moving the datadir to the new server, i created a setenv.sh script (/usr/share/tomcat8/bin) in order to increase memory for tomcat, and also tell to the GN webapp where data resides. I also edited the jdbc.properties file in order to point to the existing postgre database dedicated to this GN instance.
The problem seems to be in the dababase migration process.
I joined the beginning of the catalina.out file for you to check if there's something obvious.
the first significant error i see in the log is : /java.lang.ClassNotFoundException: v300.SetSequenceValueToMaxOfMetadataAndStats/
But the /SetSequenceValueToMaxOfMetadataAndStats.jar /file exists in the WEB-INF/classes/setup/sql/migrate/v300
I don't understand why the inside class isn't found.
The second error is /2015-10-28 09:48:50,698 ERROR [org.hibernate.tool.hbm2ddl.SchemaUpdate] - HHH000388: Unsuccessful: alter table ServiceParameters add constraint FK_t32t4xtdqmjhl8xmjpe95e474 foreign key (service) references Services//
//2015-10-28 09:48:50,699 ERROR [org.hibernate.tool.hbm2ddl.SchemaUpdate] - ERROR: there is no primary key for referenced table "services"//
//The result in GN, is that i cannot see any data.//
/This looks strange because before the migration, the primary key existed in the table "services"
I need to indicate that the version of the database is 2.10.1 before the migration.
Any help/suggestion will be appreciated ....
Thanks
--
Re: Publication profil geonetwork
(attachments)
catalina.out (21.5 KB)
Still having trouble while trying to migrate an old GN database (2.10.1) to
the new GN version (3.0.2)
I tried a lot of things, but still crossing the problem of an unreachable
java class during the migration process
(v300.SetSequenceValueToMaxOfMetadaAndStats)
* Tried to change GN version step by step: same error occurs when reaching
the v300 version
* Tried to manually run the migration sql scripts which were not run : same
error
* Tried to export metadata and import them in a totally new install: GN said
"import succesfull" but actually none of the metadata were present.
Futhermore, i cannot manage to import categories, and points of contact
database.
* Tried to install from a jar file instead from a war: exactly the same
problem with the missing java class.
The GN documentation says "it is not recommended to run a recent GN version
with an old database", which seems obviously a good advise and in my case,
this is just impossible because of huge trouble with the database are
reported by geonetwork log.
Something i could try is to install GN from source code, but that scares me
...
Waiting for your suggestions and/or advise
Philippe
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Gn-2-10-2-to-3-0-2-trouble-with-postgre-database-migration-tp5233017p5239821.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.
Hello Philippe,
On Wed, Oct 28, 2015 at 11:12 AM, Philippe Clastre <
Philippe.Clastre@anonymised.com> wrote:
the first significant error i see in the log is :
/java.lang.ClassNotFoundException:
v300.SetSequenceValueToMaxOfMetadataAndStats/
But the /SetSequenceValueToMaxOfMetadataAndStats.jar /file exists in the
WEB-INF/classes/setup/sql/migrate/v300
I don't understand why the inside class isn't found.
This seems a bug in GN (the class SetSequenceValueToMaxOfMetadataAndStats
[1]) has no package. A workarond for this is to modify the migration list
configuration [2] and remove the package name. You have to modify the file
WEB-INF/config-db/database_migration.xml in the geonetwork folder in tomcat
webapps folder.
[1]
https://github.com/geonetwork/core-geonetwork/blob/3.0.2/web/src/main/webapp/WEB-INF/classes/setup/sql/migrate/v300/SetSequenceValueToMaxOfMetadataAndStats.java#L1
[2]
https://github.com/geonetwork/core-geonetwork/blob/3.0.2/web/src/main/webapp/WEB-INF/config-db/database_migration.xml#L97
--
*Vriendelijke groeten / Kind regards,Juan Luis Rodríguez.
<http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
(0)318 416664 <+31318416664> <https://www.facebook.com/geocatbv>
<https://twitter.com/geocat_bv>
<https://plus.google.com/u/1/+GeocatNetbv/posts>Please consider the
environment before printing this email.*
Dear Juan Louis
Thank you for this suggestion.
As you suggested, i suppressed the call to the java class
v300.SetSequenceValueToMaxOfMetadataAndStats and of course, no more message
about this.
But, the migration is still not correct.
When i start GN, i can see:
* metadata are detected and counted
* categorie are correct
* thesaurus are available
* catalog settings seem to be correct also
But:
- Points of contact are counted in the directory panel but not visible, nor
editable
- Metadata are not visible nor editable (empty form)
In the catalina.out log file, i can many errors.
I join to this thread two versions of this log file:
log_catalina.txt
<http://osgeo-org.1560.x6.nabble.com/file/n5239989/log_catalina.txt> which
is generated during the first start of GN. So it includes the database
migration process log. It is detailed because i raise the log level to DEBUG
for geonetwork.database and org.hibernate.sql. You can see that there is a
problem of primary and foreign key.
Then i stopped GN, and added these constraints using pgadmin, and start GN
again.
Here is the new catalina.txt
<http://osgeo-org.1560.x6.nabble.com/file/n5239989/catalina.txt>
As is it, the catalog isn't usable...
Thank for your help or suggestion.
Philippe
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Gn-2-10-2-to-3-0-2-trouble-with-postgre-database-migration-tp5233017p5239989.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.
On Fri, Dec 4, 2015 at 10:18 AM, pclastre <Philippe.Clastre@anonymised.com>
wrote:
Thank you for this suggestion.
As you suggested, i suppressed the call to the java class
v300.SetSequenceValueToMaxOfMetadataAndStats and of course, no more message
about this.
But did you remove the class from database-migration.xml or only the
package name? This is how it should be to execute the
SetSequenceValueToMaxOfMetadataAndStats migration (without v300.):
<entry key="3.0.0"> <list> <value
java:SetSequenceValueToMaxOfMetadataAndStats</value> <value
WEB-INF/classes/setup/sql/migrate/v300/migrate-</value> <value
WEB-INF/classes/setup/sql/migrate/v300/migrate-cswservice-</value> </list>
</entry>
--
*Vriendelijke groeten / Kind regards,Juan Luis Rodríguez.
<http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
(0)318 416664 <+31318416664> <https://www.facebook.com/geocatbv>
<https://twitter.com/geocat_bv>
<https://plus.google.com/u/1/+GeocatNetbv/posts>Please consider the
environment before printing this email.*
You were right: i've done it the wrong way !!
Now, after the last modification of the database-migration.xml (just
suppress the package name and leave the class), here is the new log file
new_catalina.out
<http://osgeo-org.1560.x6.nabble.com/file/n5240019/new_catalina.out>
The problem with constraints remains the same (see lines 283&284), and the
catalog isn't usable.
other suggestion ?
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Gn-2-10-2-to-3-0-2-trouble-with-postgre-database-migration-tp5233017p5240019.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.
On 12/04/15 14:11, pclastre wrote:
You were right: i've done it the wrong way !!
Now, after the last modification of the database-migration.xml (just
suppress the package name and leave the class), here is the new log file
new_catalina.out
<http://osgeo-org.1560.x6.nabble.com/file/n5240019/new_catalina.out>
The problem with constraints remains the same (see lines 283&284), and the
catalog isn't usable.
Like you, i saw *many* errors when trying the 'automigration' procedure from older geonetwork instances to 3.0, and i really think that all usecases havent been properly tested.
See for reference..
https://github.com/geonetwork/core-geonetwork/issues/782
https://github.com/geonetwork/core-geonetwork/issues/739
https://github.com/geonetwork/core-geonetwork/issues/736
some of those issues are now closed, but there are still corner cases. Last i tried, it was impossible to migrate a catalog with harversters set, i had to remove all harvesters prior to run the migration.
That said, for this issue, i remember at some moment manually creating this primary key on services before letting the scripts run:
alter table services add constraint services_pkey PRIMARY KEY(id);
That *might* help for the specific issue in your log.
--
Landry Breuil
Mouton a 5 pattes du CRAIG
Many Thanks for your answer
I tried to follow indicated steps, but this drives me to the same problem
Here's the new log file: new_catalina.out
<http://osgeo-org.1560.x6.nabble.com/file/n5240262/new_catalina.out>
What i did:
restore the database and ran the following sql commands:
delete from harvester;
delete from harvesthistory;
update metadata set data = replace(data,
'http://147.100.20.29/geonetwork_anaee/srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
update metadata set data = replace(data,
'http://147.100.20.29/geonetwork_anaee/apps/tabsearch/../../srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
update metadata set data = replace(data,
'http://w3.avignon.inra.fr/geonetwork_anaee/srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
update metadata set data = replace(data,
'http://w3.avignon.inra.fr:80/geonetwork_anaee/srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
# these 4 lines are intended to show you exactly what im a doing. It seems
that during the catalog's life, a parameter for xlinks reference has been
change several times...
Cleaned index, spatialindex, data/subversion folders
Started GN
After the database migration, stopped GN, and ran these commands
alter table services add constraint services_pkey primary key (id);
alter table ServiceParameters add constraint FK_t32t4xtdqmjhl8xmjpe95e474
foreign key (service) references Services;
The catalog shows only empty forms, even if i can log in. I cannot see the
records in the "manage directory tab" (organizations & contacts)
Waiting for another suggestions ...
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Gn-2-10-2-to-3-0-2-trouble-with-postgre-database-migration-tp5233017p5240262.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.
On 12/07/15 14:40, pclastre wrote:
Many Thanks for your answer
I tried to follow indicated steps, but this drives me to the same problem
Here's the new log file: new_catalina.out
<http://osgeo-org.1560.x6.nabble.com/file/n5240262/new_catalina.out>
What i did:
restore the database and ran the following sql commands:
delete from harvester;
delete from harvesthistory;
update metadata set data = replace(data,
'http://147.100.20.29/geonetwork_anaee/srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
update metadata set data = replace(data,
'http://147.100.20.29/geonetwork_anaee/apps/tabsearch/../../srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
update metadata set data = replace(data,
'http://w3.avignon.inra.fr/geonetwork_anaee/srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
update metadata set data = replace(data,
'http://w3.avignon.inra.fr:80/geonetwork_anaee/srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
# these 4 lines are intended to show you exactly what im a doing. It seems
that during the catalog's life, a parameter for xlinks reference has been
change several times...
You'll be able to do this via the batch runner/replacer in the admin interface once your gn3 is working, it's much 'nicer'.
Cleaned index, spatialindex, data/subversion folders
Started GN
After the database migration, stopped GN, and ran these commands
alter table services add constraint services_pkey primary key (id);
alter table ServiceParameters add constraint FK_t32t4xtdqmjhl8xmjpe95e474
foreign key (service) references Services;
I would say you have to add the pkey on services table *before* (or add the line in one of the sql scripts, before the line that add the fkey from serviceparameters to services) running the migration, because if it's not here, it will fail (as it does right now) and *not* finish running the scripts/migration.
But, looking at the scripts, this pkey is added in setup/sql/migrate/v2100/migrate-db-*.sql (which means you should already have it in your 2.10.2 installation) and then removed in setup/sql/migrate/v2110/2-migrate-default.sql. this doesnt make sense (as usual with GN db model migrations, sadly..)
--
Landry Breuil
Mouton a 5 pattes du CRAIG
Hi, In the log the main error is:
Error at xsl:variable on line 116 column 81 of functions.xsl:
XPST0017: XPath syntax error at char 39 on line 116 in
{...eISODateTimes($value1,$valu...}:
Cannot find a matching 2-argument function named
{java:org.fao.geonet.util.JODAISODate}parseISODateTimes()
2015-12-07 14:28:29,896 ERROR [geonetwork.index] - Indexing stylesheet
contains errors: Failed to compile stylesheet. 1 error detected.
and is related to indexing (not to migration) which probably explains why
you can't see records.
Do you have any custom profiles ? You should check a functions.xsl with
this invalid function call.
Francois
2015-12-07 14:40 GMT+01:00 pclastre <Philippe.Clastre@anonymised.com>:
Many Thanks for your answer
I tried to follow indicated steps, but this drives me to the same problem
Here's the new log file: new_catalina.out
<http://osgeo-org.1560.x6.nabble.com/file/n5240262/new_catalina.out>
What i did:
restore the database and ran the following sql commands:
delete from harvester;
delete from harvesthistory;
update metadata set data = replace(data,
'http://147.100.20.29/geonetwork_anaee/srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
update metadata set data = replace(data,
'http://147.100.20.29/geonetwork_anaee/apps/tabsearch/../../srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
update metadata set data = replace(data,
'http://w3.avignon.inra.fr/geonetwork_anaee/srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
update metadata set data = replace(data,
'http://w3.avignon.inra.fr:80/geonetwork_anaee/srv’,
'http://localhost:8082/geonetwork_anaee/srv’);
# these 4 lines are intended to show you exactly what im a doing. It
seems
that during the catalog's life, a parameter for xlinks reference has been
change several times...
Cleaned index, spatialindex, data/subversion folders
Started GN
After the database migration, stopped GN, and ran these commands
alter table services add constraint services_pkey primary key (id);
alter table ServiceParameters add constraint
FK_t32t4xtdqmjhl8xmjpe95e474
foreign key (service) references Services;
The catalog shows only empty forms, even if i can log in. I cannot see the
records in the "manage directory tab" (organizations & contacts)
Waiting for another suggestions ...
--
View this message in context:
http://osgeo-org.1560.x6.nabble.com/Gn-2-10-2-to-3-0-2-trouble-with-postgre-database-migration-tp5233017p5240262.html
Sent from the GeoNetwork users mailing list archive at Nabble.com.
------------------------------------------------------------------------------
Go from Idea to Many App Stores Faster with Intel(R) XDK
Give your users amazing mobile app experiences with Intel(R) XDK.
Use one codebase in this all-in-one HTML5 development environment.
Design, debug & build mobile apps & 2D/3D high-impact games for multiple
OSs.
http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork
2015-12-07 15:57 GMT+01:00 Philippe Clastre <
Philippe.Clastre@anonymised.com>:
HI,
No custom profiles at all.
I just built metadata templates based on the iso 19139 standard. Never add
a function.xsl function anywhere (probably unable to do so !).
Not sure profiles are related to metadata standards ...
ok. Using a custom datadir ? (check in
http://localhost:8080/geonetwork/srv/eng/admin.console#/dashboard/information
the
data directory). If yes, drop the schema_plugins folder which may be the
one from your previous installation ?
Just guessing.
I'm trying to success in upgrading GN version ( 2.10.2 to 3.0.2), and the
very first problem i met was obviously related to the db migration.
That why, i'm still stucked to this ...
Don't know what to do now ...
Le 07/12/2015 15:16, Francois Prunayre a écrit :
Hi, In the log the main error is:
Error at xsl:variable on line 116 column 81 of functions.xsl:
XPST0017: XPath syntax error at char 39 on line 116 in
{...eISODateTimes($value1,$valu...}:
Cannot find a matching 2-argument function named
{java:org.fao.geonet.util.JODAISODate}parseISODateTimes()
2015-12-07 14:28:29,896 ERROR [geonetwork.index] - Indexing stylesheet
contains errors: Failed to compile stylesheet. 1 error detected.
and is related to indexing (not to migration) which probably explains why
you can't see records.
Do you have any custom profiles ? You should check a functions.xsl with
this invalid function call.
Francois
2015-12-07 14:40 GMT+01:00 pclastre < <Philippe.Clastre@anonymised.com>
Philippe.Clastre@anonymised.com>:
>
> Many Thanks for your answer
>
> I tried to follow indicated steps, but this drives me to the same problem
> Here's the new log file: new_catalina.out
> <http://osgeo-org.1560.x6.nabble.com/file/n5240262/new_catalina.out>
>
> What i did:
>
>
> restore the database and ran the following sql commands:
> delete from harvester;
> delete from harvesthistory;
> update metadata set data = replace(data,
> 'http://147.100.20.29/geonetwork_anaee/srv’,
> 'http://localhost:8082/geonetwork_anaee/srv’);
> update metadata set data = replace(data,
> 'http://147.100.20.29/geonetwork_anaee/apps/tabsearch/../../srv’,
> 'http://localhost:8082/geonetwork_anaee/srv’);
> update metadata set data = replace(data,
> 'http://w3.avignon.inra.fr/geonetwork_anaee/srv’,
> 'http://localhost:8082/geonetwork_anaee/srv’);
> update metadata set data = replace(data,
> 'http://w3.avignon.inra.fr:80/geonetwork_anaee/srv’,
> 'http://localhost:8082/geonetwork_anaee/srv’);
> # these 4 lines are intended to show you exactly what im a doing. It
seems
> that during the catalog's life, a parameter for xlinks reference has been
> change several times...
>
> Cleaned index, spatialindex, data/subversion folders
> Started GN
>
> After the database migration, stopped GN, and ran these commands
> alter table services add constraint services_pkey primary key (id);
> alter table ServiceParameters add constraint
FK_t32t4xtdqmjhl8xmjpe95e474
> foreign key (service) references Services;
>
> The catalog shows only empty forms, even if i can log in. I cannot see
the
> records in the "manage directory tab" (organizations & contacts)
>
> Waiting for another suggestions ...
>
>
>
> --
> View this message in context:
http://osgeo-org.1560.x6.nabble.com/Gn-2-10-2-to-3-0-2-trouble-with-postgre-database-migration-tp5233017p5240262.html
> Sent from the GeoNetwork users mailing list archive at Nabble.com.
>
>
------------------------------------------------------------------------------
> Go from Idea to Many App Stores Faster with Intel(R) XDK
> Give your users amazing mobile app experiences with Intel(R) XDK.
> Use one codebase in this all-in-one HTML5 development environment.
> Design, debug & build mobile apps & 2D/3D high-impact games for multiple
OSs.
> http://pubads.g.doubleclick.net/gampad/clk?id=254741911&iu=/4140
> _______________________________________________
> GeoNetwork-users mailing list
> GeoNetwork-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
> GeoNetwork OpenSource is maintained at
<http://sourceforge.net/projects/geonetwork>
http://sourceforge.net/projects/geonetwork
<snip logo image is too big for mailman/>
BINGO ! Thank to all of you
In order to help others, here are the needed steps to move a 2.10.2 geonetwork installation to 3.0.2
in this specific context:
Existing installation: GN 2.10.2, postgresql database, custom datadir (elsewhere than the default datadir)
The aim: a 3.0.2 GN version running in a tomcat servlet, on a new debian server.
Save the database !
Deploy the new war file
If you want to use a custom datadir, you have to follow this documentation <http://geonetwork-opensource.org/manuals/2.10.4/eng/users/admin/advanced-configuration/index.html#geonetwork-data-directory>
Start the tomcat
Wait until GN is fully started (check the web page) and stop the tomcat
Edit configuration files (.../WEB-INF/config-node/srv.xml and .../WEB-INF/config-db/jdbc.properties) in order to point to your postgres database server, username, passwd ...
Copy the existing custom datadir and delete index, indexspatial, data/subversion, config/schema_plugins folders
Fill the custom datadir/config/schema_plugins with an existing Gn 3.0.2 installation, or get the scheme from github
Start the tomcat
In the log file (catalina.out), you should see errors about the primary key on services tables and about a foreign key on serviceparameters table.
Stop the tomcat
Run the following sql commands with a client like pgadmin:
* alter table services add constraint services_pkey primary key (id);
* alter table ServiceParameters add constraint FK_t32t4xtdqmjhl8xmjpe95e474 foreign key (service) references Services;
Restart GN
If you get trouble with the overviews (no display), this may be due to server URL changes. Check in the log file, looking for error messages about the previous URL. Then, you can use the URL replacer in the admin console/tools/batch process to change URL for a set (or all) of metadata you've previously selected. After all, you'll have to rebuild the index.
Hope this may help