[GeoNetwork-users] Problem updating from GN4.0.6 to GN4.2.2

Thank you for your excellent assistance, Francois!

Today I was able to try the upgrade, and followed that with your two recommended Calls to cleanup the JS cache. GeoNetwork took a minute or so to respond, which I guess because it was busy, but then everything loaded and ran fine. I did not need to use the "?debug" suffix.

The minor error message I reported when manually migrating the database was because there is no "editable" column in the settings database table when manually migrating at the v407 step; the column appears later when GeoNetwork is run. When the migration is done automatically, it probably works slightly differently.

I find the new splash screen quite inviting. The searches seem much faster - and entering even a few characters in a word reveals relevant records. The team has done a great job with this release, IMHO.

-- terry

-------- Forwarded Message --------
Subject: Re: [GeoNetwork-users] Problem updating from GN4.0.6 to GN4.2.2
Date: Thu, 4 May 2023 07:56:53 +0200
From: Francois Prunayre <fx.prunayre@anonymised.com>
To: Terry <terry.curran@anonymised.com>

Indeed, no issue in the log. SQL migration sounds to have completed successfully (version was updated in settings table).
H2 databases are only used for caches.

I would try to startup. Call
http://domain.org/geonetwork/static/wroAPI/reloadModel
http://domain.org/geonetwork/static/wroAPI/reloadCache
to force a cleanup of JS cache

and then load the home page and check any error in the log.

To increase log level, you can update the settings and change log4j2.xml by log4j2-dev.xml

HTH
Francois

Le mer. 3 mai 2023 à 20:10, Terry <terry.curran@anonymised.com> a écrit :

    <!-- SNIP -->

    On 2023-05-02 11:28 p.m., Francois Prunayre wrote:

    Hi Terry,

    I would
    * check the following in the settings table
    UPDATE Settings SET editable = 'y' WHERE editable IS NULL;
    to be sure that the value is properly set and there is no null
    (which should not be possible).

    * About H2, check
    https://geonetwork-opensource.org/manuals/4.0.x/en/overview/change-log/version-4.2.0.html#migration-instructions

    * Any errors in the logs ?

    Cheers.
    Francois

    Le mer. 3 mai 2023 à 05:37, Terry via GeoNetwork-users
    <geonetwork-users@lists.sourceforge.net> a écrit :

        We are trying to avoid a full metadata dump and reload.

        We have a RHEL8 VM with an Apache Reverse Proxy ahead of a Jetty
        webserver hosting GN4.0.6, GS and ERDDAP.We also have a PostGIS
        database, and a remote gn_dir.Startup and linkages are by a Jetty
        daemon.Everything seemed to be working for more than a year.

        To upgrade to GN4.2.2, as usual we downloaded the war,
        unzipped it, and
        made changes to three files: config-node/srv.xml to point
        exclusively at
        config-db/jndi.xml, jetty-environment.xml to contain the database
        credentials (name, port, user, pw, and gnDatabaseDialect [to
        POSTGRESQL]) and config-db/jndi.xml to contain the usual PostGIS
        parameters.On some attempts we also set the jdbc.properties to
        the same,
        but usually just kept the defaults.

        Upon reboot with the modified GN, the database got migrated to
        4.2.2,
        with two we think insignificant errors from v407 migration:

        gndb422=# UPDATE Settings SET editable = 'n' WHERE name =
        'system/userFeedback/lastNotificationDate';
        ERROR: column "editable" of relation "settings" does not exist
        LINE 1: UPDATE Settings SET editable = 'n' WHERE name =
        'system/user...
         ^
        gndb422=# UPDATE Settings SET editable = 'n' WHERE name =
        'system/security/passwordEnforcement/pattern';
        ERROR: column "editable" of relation "settings" does not exist
        LINE 1: UPDATE Settings SET editable = 'n' WHERE name =
        'system/secu...

        We got the title on the GN splash screen from the settings, but
        otherwise blank screen.We noticed the log had many h2 errors
        regarding
        the gn-wro4j disk-cache, and related h2 errors, but also said
        that
        geonetwork was available.Stopped jetty, deleted the
        wro4j-cache.mv.db in
        remote gn_dir folder, because we knew it would get rebuilt on
        restart.

        Rebooted computer and thus jetty, and no "gn-wro4j disk-cache"
        errors.
        Got the same header and blank screen.However, appending
        “?debug” allowed
        login and we could then see the data.Cleared the “JS & CSS
        cache” and
        rebooted.Things looked okay.Tried reindexing and got a 502 Proxy
        Error.Metadata could no longer be seen.

        Any thoughts about next step, and/or what we are doing wrong?

        Terry

        Pacific Salmon Foundation

        _______________________________________________
        GeoNetwork-users mailing list
        GeoNetwork-users@lists.sourceforge.net
        https://lists.sourceforge.net/lists/listinfo/geonetwork-users
        GeoNetwork OpenSource is maintained at
        http://sourceforge.net/projects/geonetwork

I have been looking a little further into my update, and it seems there is still something unresolved. The problems show up when I sign in as administrator. When I select "Statistics and status" in the Admin console, I get "Query returned an error. Check console for details". Similarly for Contribute/BatchEditing and Console/EditorBoard.

The jetty log has a message:

    "Error at xsl:call-template on line 578 column 62 of index.xsl:
      XTSE0650: No template exists named build-thesaurus-fields
    2023-05-09T09:32:16,764 ERROR [geonetwork.index] - Indexing
    stylesheet contains errors: Failed to compile stylesheet. 1 error
    detected.
      Marking the metadata as _indexingError=1 in index"

In the BatchEditing and EditorBoard tabs, the normal list of files is initially blank. However if I place any character in the search box and try to search, the files then appear.

Finally, the parent-child metadata links seem to have disappeared.

On 2023-05-06 5:32 p.m., Terry wrote:

Thank you for your excellent assistance, Francois!

Today I was able to try the upgrade, and followed that with your two recommended Calls to cleanup the JS cache. GeoNetwork took a minute or so to respond, which I guess because it was busy, but then everything loaded and ran fine. I did not need to use the "?debug" suffix.

The minor error message I reported when manually migrating the database was because there is no "editable" column in the settings database table when manually migrating at the v407 step; the column appears later when GeoNetwork is run. When the migration is done automatically, it probably works slightly differently.

I find the new splash screen quite inviting. The searches seem much faster - and entering even a few characters in a word reveals relevant records. The team has done a great job with this release, IMHO.

-- terry

-------- Forwarded Message --------
Subject: Re: [GeoNetwork-users] Problem updating from GN4.0.6 to GN4.2.2
Date: Thu, 4 May 2023 07:56:53 +0200
From: Francois Prunayre <fx.prunayre@anonymised.com>
To: Terry <terry.curran@anonymised.com>

Indeed, no issue in the log. SQL migration sounds to have completed successfully (version was updated in settings table).
H2 databases are only used for caches.

I would try to startup. Call
http://domain.org/geonetwork/static/wroAPI/reloadModel
http://domain.org/geonetwork/static/wroAPI/reloadCache
to force a cleanup of JS cache

and then load the home page and check any error in the log.

To increase log level, you can update the settings and change log4j2.xml by log4j2-dev.xml

HTH
Francois

Le mer. 3 mai 2023 à 20:10, Terry <terry.curran@anonymised.com> a écrit :

    <!-- SNIP -->

    On 2023-05-02 11:28 p.m., Francois Prunayre wrote:

    Hi Terry,

    I would
    * check the following in the settings table
    UPDATE Settings SET editable = 'y' WHERE editable IS NULL;
    to be sure that the value is properly set and there is no null
    (which should not be possible).

    * About H2, check
    https://geonetwork-opensource.org/manuals/4.0.x/en/overview/change-log/version-4.2.0.html#migration-instructions

    * Any errors in the logs ?

    Cheers.
    Francois

    Le mer. 3 mai 2023 à 05:37, Terry via GeoNetwork-users
    <geonetwork-users@lists.sourceforge.net> a écrit :

        We are trying to avoid a full metadata dump and reload.

        We have a RHEL8 VM with an Apache Reverse Proxy ahead of a Jetty
        webserver hosting GN4.0.6, GS and ERDDAP.We also have a PostGIS
        database, and a remote gn_dir.Startup and linkages are by a
        Jetty
        daemon.Everything seemed to be working for more than a year.

        To upgrade to GN4.2.2, as usual we downloaded the war,
        unzipped it, and
        made changes to three files: config-node/srv.xml to point
        exclusively at
        config-db/jndi.xml, jetty-environment.xml to contain the
        database
        credentials (name, port, user, pw, and gnDatabaseDialect [to
        POSTGRESQL]) and config-db/jndi.xml to contain the usual PostGIS
        parameters.On some attempts we also set the jdbc.properties
        to the same,
        but usually just kept the defaults.

        Upon reboot with the modified GN, the database got migrated
        to 4.2.2,
        with two we think insignificant errors from v407 migration:

        gndb422=# UPDATE Settings SET editable = 'n' WHERE name =
        'system/userFeedback/lastNotificationDate';
        ERROR: column "editable" of relation "settings" does not exist
        LINE 1: UPDATE Settings SET editable = 'n' WHERE name =
        'system/user...
         ^
        gndb422=# UPDATE Settings SET editable = 'n' WHERE name =
        'system/security/passwordEnforcement/pattern';
        ERROR: column "editable" of relation "settings" does not exist
        LINE 1: UPDATE Settings SET editable = 'n' WHERE name =
        'system/secu...

        We got the title on the GN splash screen from the settings, but
        otherwise blank screen.We noticed the log had many h2 errors
        regarding
        the gn-wro4j disk-cache, and related h2 errors, but also said
        that
        geonetwork was available.Stopped jetty, deleted the
        wro4j-cache.mv.db in
        remote gn_dir folder, because we knew it would get rebuilt on
        restart.

        Rebooted computer and thus jetty, and no "gn-wro4j
        disk-cache" errors.
        Got the same header and blank screen.However, appending
        “?debug” allowed
        login and we could then see the data.Cleared the “JS & CSS
        cache” and
        rebooted.Things looked okay.Tried reindexing and got a 502 Proxy
        Error.Metadata could no longer be seen.

        Any thoughts about next step, and/or what we are doing wrong?

        Terry

        Pacific Salmon Foundation

        _______________________________________________
        GeoNetwork-users mailing list
        GeoNetwork-users@lists.sourceforge.net
        https://lists.sourceforge.net/lists/listinfo/geonetwork-users
        GeoNetwork OpenSource is maintained at
        http://sourceforge.net/projects/geonetwork

Hi Terry, are you using an external data directory ? and if yes with the
schema plugin in it ?

To check, admin console > statistics > Information >
Standard
folder:/opt/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins

If the schema plugin is not the one from the webapp, then you are using the
one from your previous version probably.

A good practice is to externalize the data directory but keep the schema
plugin from the webapp (unless you need to customize it) see
https://geonetwork-opensource.org/manuals/4.0.x/en/install-guide/customizing-data-directory.html?highlight=schemaplugin#advanced-data-directory-configuration
eg.
-Dgeonetwork.dir=/data/catalogue
-Dgeonetwork.schema.dir=/app/tomcat/webapps/geonetwork/WEB-INF/data/config/schema_plugins

Let us know if that works for you

Francois

Le mar. 9 mai 2023 à 19:45, Terry via GeoNetwork-users <
geonetwork-users@lists.sourceforge.net> a écrit :

I have been looking a little further into my update, and it seems there
is still something unresolved. The problems show up when I sign in as
administrator. When I select "Statistics and status" in the Admin
console, I get "Query returned an error. Check console for details".
Similarly for Contribute/BatchEditing and Console/EditorBoard.

The jetty log has a message:

    "Error at xsl:call-template on line 578 column 62 of index.xsl:
         XTSE0650: No template exists named build-thesaurus-fields
    2023-05-09T09:32:16,764 ERROR [geonetwork.index] - Indexing
    stylesheet contains errors: Failed to compile stylesheet. 1 error
    detected.
        Marking the metadata as _indexingError=1 in index"

In the BatchEditing and EditorBoard tabs, the normal list of files is
initially blank. However if I place any character in the search box and
try to search, the files then appear.

Finally, the parent-child metadata links seem to have disappeared.

On 2023-05-06 5:32 p.m., Terry wrote:
>
> Thank you for your excellent assistance, Francois!
>
> Today I was able to try the upgrade, and followed that with your two
> recommended Calls to cleanup the JS cache. GeoNetwork took a minute
> or so to respond, which I guess because it was busy, but then
> everything loaded and ran fine. I did not need to use the "?debug"
> suffix.
>
> The minor error message I reported when manually migrating the
> database was because there is no "editable" column in the settings
> database table when manually migrating at the v407 step; the column
> appears later when GeoNetwork is run. When the migration is done
> automatically, it probably works slightly differently.
>
> I find the new splash screen quite inviting. The searches seem much
> faster - and entering even a few characters in a word reveals relevant
> records. The team has done a great job with this release, IMHO.
>
> -- terry
>
> -------- Forwarded Message --------
> Subject: Re: [GeoNetwork-users] Problem updating from GN4.0.6 to
GN4.2.2
> Date: Thu, 4 May 2023 07:56:53 +0200
> From: Francois Prunayre <fx.prunayre@anonymised.com>
> To: Terry <terry.curran@anonymised.com>
>
>
>
> Indeed, no issue in the log. SQL migration sounds to have completed
> successfully (version was updated in settings table).
> H2 databases are only used for caches.
>
> I would try to startup. Call
> http://domain.org/geonetwork/static/wroAPI/reloadModel
> http://domain.org/geonetwork/static/wroAPI/reloadCache
> to force a cleanup of JS cache
>
> and then load the home page and check any error in the log.
>
> To increase log level, you can update the settings and change
> log4j2.xml by log4j2-dev.xml
>
> HTH
> Francois
>
> Le mer. 3 mai 2023 à 20:10, Terry <terry.curran@anonymised.com> a écrit :
>
> <!-- SNIP -->
>
> On 2023-05-02 11:28 p.m., Francois Prunayre wrote:
>> Hi Terry,
>>
>> I would
>> * check the following in the settings table
>> UPDATE Settings SET editable = 'y' WHERE editable IS NULL;
>> to be sure that the value is properly set and there is no null
>> (which should not be possible).
>>
>> * About H2, check
>>
https://geonetwork-opensource.org/manuals/4.0.x/en/overview/change-log/version-4.2.0.html#migration-instructions
>>
>> * Any errors in the logs ?
>>
>> Cheers.
>> Francois
>>
>>
>> Le mer. 3 mai 2023 à 05:37, Terry via GeoNetwork-users
>> <geonetwork-users@lists.sourceforge.net> a écrit :
>>
>> We are trying to avoid a full metadata dump and reload.
>>
>> We have a RHEL8 VM with an Apache Reverse Proxy ahead of a Jetty
>> webserver hosting GN4.0.6, GS and ERDDAP.We also have a PostGIS
>> database, and a remote gn_dir.Startup and linkages are by a
>> Jetty
>> daemon.Everything seemed to be working for more than a year.
>>
>> To upgrade to GN4.2.2, as usual we downloaded the war,
>> unzipped it, and
>> made changes to three files: config-node/srv.xml to point
>> exclusively at
>> config-db/jndi.xml, jetty-environment.xml to contain the
>> database
>> credentials (name, port, user, pw, and gnDatabaseDialect [to
>> POSTGRESQL]) and config-db/jndi.xml to contain the usual PostGIS
>> parameters.On some attempts we also set the jdbc.properties
>> to the same,
>> but usually just kept the defaults.
>>
>> Upon reboot with the modified GN, the database got migrated
>> to 4.2.2,
>> with two we think insignificant errors from v407 migration:
>>
>> gndb422=# UPDATE Settings SET editable = 'n' WHERE name =
>> 'system/userFeedback/lastNotificationDate';
>> ERROR: column "editable" of relation "settings" does not exist
>> LINE 1: UPDATE Settings SET editable = 'n' WHERE name =
>> 'system/user...
>> ^
>> gndb422=# UPDATE Settings SET editable = 'n' WHERE name =
>> 'system/security/passwordEnforcement/pattern';
>> ERROR: column "editable" of relation "settings" does not exist
>> LINE 1: UPDATE Settings SET editable = 'n' WHERE name =
>> 'system/secu...
>>
>> We got the title on the GN splash screen from the settings, but
>> otherwise blank screen.We noticed the log had many h2 errors
>> regarding
>> the gn-wro4j disk-cache, and related h2 errors, but also said
>> that
>> geonetwork was available.Stopped jetty, deleted the
>> wro4j-cache.mv.db in
>> remote gn_dir folder, because we knew it would get rebuilt on
>> restart.
>>
>> Rebooted computer and thus jetty, and no "gn-wro4j
>> disk-cache" errors.
>> Got the same header and blank screen.However, appending
>> “?debug” allowed
>> login and we could then see the data.Cleared the “JS & CSS
>> cache” and
>> rebooted.Things looked okay.Tried reindexing and got a 502 Proxy
>> Error.Metadata could no longer be seen.
>>
>> Any thoughts about next step, and/or what we are doing wrong?
>>
>> Terry
>>
>> Pacific Salmon Foundation
>>
>>
>> _______________________________________________
>> GeoNetwork-users mailing list
>> GeoNetwork-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>> GeoNetwork OpenSource is maintained at
>> http://sourceforge.net/projects/geonetwork
>>
>

_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

Answers Interspersed:

On 2023-05-09 10:19 p.m., Francois Prunayre wrote:

Hi Terry, are you using an external data directory ? and if yes with the schema plugin in it ?
*** yes, external (/data/gn_dir) and yes - see below

To check, admin console > statistics > Information >
Standard folder:/opt/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins
*** yes: Standard folder => /data/gn_dir/config/schema_plugins

If the schema plugin is not the one from the webapp, then you are using the one from your previous version probably.

 \*\*\* yes it is the one from the former GN4\.0\.6 app

A good practice is to externalize the data directory but keep the schema plugin from the webapp (unless you need to customize it) see
https://geonetwork-opensource.org/manuals/4.0.x/en/install-guide/customizing-data-directory.html?highlight=schemaplugin#advanced-data-directory-configuration eg.
-Dgeonetwork.dir=/data/catalogue
-Dgeonetwork.schema.dir=/app/tomcat/webapps/geonetwork/WEB-INF/data/config/schema_plugins

*** jetty.service CHANGED TO (and verified in statistics > Information) :
[Unit]
Description=Jetty
After=network-online.target

[Service]
Type=simple
User=jetty
Group=jetty
ExecStart= /bin/java -Xms2g -Xmx2g -Djava.awt.headless=true -Djava.io.tmpdir=/data/jettytemp \
-Dgeonetwork.dir=/data/gn_dir -Dlog_dir=/var/www/jetty -DSTOP.PORT=8079 -DSTOP.KEY=geonetwork \
-Dgeonetwork.schema.dir=/var/www/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins \
-Dmime-mappings=/var/www/jetty/webapps/geonetwork/WEB-INF/mime-types.properties \
-jar /usr/share/java/jetty-distribution-9.4.36.v20210114/start.jar jetty.base=/var/www/jetty
ExecStop=/bin/kill ${MAINPID}
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

Let us know if that works for you

*** but still same responses - same error msg, same BatchEditing and EditorBoard responses, same lost parent-child

*** should I try re-indexing?

Francois

Le mar. 9 mai 2023 à 19:45, Terry via GeoNetwork-users <geonetwork-users@lists.sourceforge.net> a écrit :

    I have been looking a little further into my update, and it seems
    there
    is still something unresolved. The problems show up when I sign in as
    administrator. When I select "Statistics and status" in the Admin
    console, I get "Query returned an error. Check console for details".
    Similarly for Contribute/BatchEditing and Console/EditorBoard.

    The jetty log has a message:

     "Error at xsl:call-template on line 578 column 62 of index.xsl:
     XTSE0650: No template exists named build-thesaurus-fields
     2023-05-09T09:32:16,764 ERROR [geonetwork.index] - Indexing
     stylesheet contains errors: Failed to compile stylesheet. 1 error
     detected.
     Marking the metadata as _indexingError=1 in index"

    In the BatchEditing and EditorBoard tabs, the normal list of files is
    initially blank. However if I place any character in the search
    box and
    try to search, the files then appear.

    Finally, the parent-child metadata links seem to have disappeared.

    On 2023-05-06 5:32 p.m., Terry wrote:
    >
    > Thank you for your excellent assistance, Francois!
    >
    > Today I was able to try the upgrade, and followed that with your
    two
    > recommended Calls to cleanup the JS cache. GeoNetwork took a
    minute
    > or so to respond, which I guess because it was busy, but then
    > everything loaded and ran fine. I did not need to use the "?debug"
    > suffix.
    >
    > The minor error message I reported when manually migrating the
    > database was because there is no "editable" column in the settings
    > database table when manually migrating at the v407 step; the column
    > appears later when GeoNetwork is run. When the migration is done
    > automatically, it probably works slightly differently.
    >
    > I find the new splash screen quite inviting. The searches seem
    much
    > faster - and entering even a few characters in a word reveals
    relevant
    > records. The team has done a great job with this release, IMHO.
    >
    > -- terry
    >
    > -------- Forwarded Message --------
    > Subject: Re: [GeoNetwork-users] Problem updating from
    GN4.0.6 to GN4.2.2
    > Date: Thu, 4 May 2023 07:56:53 +0200
    > From: Francois Prunayre <fx.prunayre@anonymised.com>
    > To: Terry <terry.curran@anonymised.com>
    >
    > Indeed, no issue in the log. SQL migration sounds to have completed
    > successfully (version was updated in settings table).
    > H2 databases are only used for caches.
    >
    > I would try to startup. Call
    > http://domain.org/geonetwork/static/wroAPI/reloadModel
    > http://domain.org/geonetwork/static/wroAPI/reloadCache
    > to force a cleanup of JS cache
    >
    > and then load the home page and check any error in the log.
    >
    > To increase log level, you can update the settings and change
    > log4j2.xml by log4j2-dev.xml
    >
    > HTH
    > Francois
    >
    > Le mer. 3 mai 2023 à 20:10, Terry <terry.curran@anonymised.com> a écrit :
    >
    > <!-- SNIP -->
    >
    > On 2023-05-02 11:28 p.m., Francois Prunayre wrote:
    >> Hi Terry,
    >>
    >> I would
    >> * check the following in the settings table
    >> UPDATE Settings SET editable = 'y' WHERE editable IS NULL;
    >> to be sure that the value is properly set and there is no null
    >> (which should not be possible).
    >>
    >> * About H2, check
    >>
    https://geonetwork-opensource.org/manuals/4.0.x/en/overview/change-log/version-4.2.0.html#migration-instructions
    >>
    >> * Any errors in the logs ?
    >>
    >> Cheers.
    >> Francois
    >>
    >> Le mer. 3 mai 2023 à 05:37, Terry via GeoNetwork-users
    >> <geonetwork-users@lists.sourceforge.net> a écrit :
    >>
    >> We are trying to avoid a full metadata dump and reload.
    >>
    >> We have a RHEL8 VM with an Apache Reverse Proxy ahead
    of a Jetty
    >> webserver hosting GN4.0.6, GS and ERDDAP.We also have a
    PostGIS
    >> database, and a remote gn_dir.Startup and linkages are by a
    >> Jetty
    >> daemon.Everything seemed to be working for more than a
    year.
    >>
    >> To upgrade to GN4.2.2, as usual we downloaded the war,
    >> unzipped it, and
    >> made changes to three files: config-node/srv.xml to point
    >> exclusively at
    >> config-db/jndi.xml, jetty-environment.xml to contain the
    >> database
    >> credentials (name, port, user, pw, and
    gnDatabaseDialect [to
    >> POSTGRESQL]) and config-db/jndi.xml to contain the
    usual PostGIS
    >> parameters.On some attempts we also set the jdbc.properties
    >> to the same,
    >> but usually just kept the defaults.
    >>
    >> Upon reboot with the modified GN, the database got migrated
    >> to 4.2.2,
    >> with two we think insignificant errors from v407 migration:
    >>
    >> gndb422=# UPDATE Settings SET editable = 'n' WHERE name =
    >> 'system/userFeedback/lastNotificationDate';
    >> ERROR: column "editable" of relation "settings" does
    not exist
    >> LINE 1: UPDATE Settings SET editable = 'n' WHERE name =
    >> 'system/user...
    >> ^
    >> gndb422=# UPDATE Settings SET editable = 'n' WHERE name =
    >> 'system/security/passwordEnforcement/pattern';
    >> ERROR: column "editable" of relation "settings" does
    not exist
    >> LINE 1: UPDATE Settings SET editable = 'n' WHERE name =
    >> 'system/secu...
    >>
    >> We got the title on the GN splash screen from the
    settings, but
    >> otherwise blank screen.We noticed the log had many h2
    errors
    >> regarding
    >> the gn-wro4j disk-cache, and related h2 errors, but
    also said
    >> that
    >> geonetwork was available.Stopped jetty, deleted the
    >> wro4j-cache.mv.db in
    >> remote gn_dir folder, because we knew it would get
    rebuilt on
    >> restart.
    >>
    >> Rebooted computer and thus jetty, and no "gn-wro4j
    >> disk-cache" errors.
    >> Got the same header and blank screen.However, appending
    >> “?debug” allowed
    >> login and we could then see the data.Cleared the “JS & CSS
    >> cache” and
    >> rebooted.Things looked okay.Tried reindexing and got a
    502 Proxy
    >> Error.Metadata could no longer be seen.
    >>
    >> Any thoughts about next step, and/or what we are doing
    wrong?
    >>
    >> Terry
    >>
    >> Pacific Salmon Foundation
    >>
    >> _______________________________________________
    >> GeoNetwork-users mailing list
    >> GeoNetwork-users@lists.sourceforge.net
    >> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
    >> GeoNetwork OpenSource is maintained at
    >> http://sourceforge.net/projects/geonetwork
    >>
    >

    _______________________________________________
    GeoNetwork-users mailing list
    GeoNetwork-users@lists.sourceforge.net
    https://lists.sourceforge.net/lists/listinfo/geonetwork-users
    GeoNetwork OpenSource is maintained at
    http://sourceforge.net/projects/geonetwork

Yes every time you change the schema dir, the indexing will be affected so
try to reindex first.

About parent child, you are probably talking about the "create child"
action which was removed by
https://github.com/geonetwork/core-geonetwork/pull/6708 because the
function was only relevant for ISO19139 records. You can still
manage parent/child relations in the editor but not anymore on record
creation.

Cheers.
Francois

Le mer. 10 mai 2023 à 08:14, Terry <terry.curran@anonymised.com> a écrit :

Answers Interspersed:

On 2023-05-09 10:19 p.m., Francois Prunayre wrote:

Hi Terry, are you using an external data directory ? and if yes with the
schema plugin in it ?
  *** yes, external (/data/gn_dir) and yes - see below

To check, admin console > statistics > Information >
Standard
folder:/opt/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins
  *** yes: Standard folder => /data/gn_dir/config/schema_plugins

If the schema plugin is not the one from the webapp, then you are using
the one from your previous version probably.

    *** yes it is the one from the former GN4.0.6 app

A good practice is to externalize the data directory but keep the schema
plugin from the webapp (unless you need to customize it) see

https://geonetwork-opensource.org/manuals/4.0.x/en/install-guide/customizing-data-directory.html?highlight=schemaplugin#advanced-data-directory-configuration
eg.
-Dgeonetwork.dir=/data/catalogue

-Dgeonetwork.schema.dir=/app/tomcat/webapps/geonetwork/WEB-INF/data/config/schema_plugins

*** jetty.service CHANGED TO (and verified in statistics > Information) :
[Unit]
Description=Jetty
After=network-online.target

[Service]
Type=simple
User=jetty
Group=jetty
ExecStart= /bin/java -Xms2g -Xmx2g -Djava.awt.headless=true
-Djava.io.tmpdir=/data/jettytemp \
          -Dgeonetwork.dir=/data/gn_dir -Dlog_dir=/var/www/jetty
-DSTOP.PORT=8079 -DSTOP.KEY=geonetwork \

-Dgeonetwork.schema.dir=/var/www/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins
\

-Dmime-mappings=/var/www/jetty/webapps/geonetwork/WEB-INF/mime-types.properties
\
          -jar
/usr/share/java/jetty-distribution-9.4.36.v20210114/start.jar
jetty.base=/var/www/jetty
ExecStop=/bin/kill ${MAINPID}
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

Let us know if that works for you

  *** but still same responses - same error msg, same BatchEditing and
EditorBoard responses, same lost parent-child

  *** should I try re-indexing?

Francois

Le mar. 9 mai 2023 à 19:45, Terry via GeoNetwork-users <
geonetwork-users@lists.sourceforge.net> a écrit :

I have been looking a little further into my update, and it seems there
is still something unresolved. The problems show up when I sign in as
administrator. When I select "Statistics and status" in the Admin
console, I get "Query returned an error. Check console for details".
Similarly for Contribute/BatchEditing and Console/EditorBoard.

The jetty log has a message:

    "Error at xsl:call-template on line 578 column 62 of index.xsl:
         XTSE0650: No template exists named build-thesaurus-fields
    2023-05-09T09:32:16,764 ERROR [geonetwork.index] - Indexing
    stylesheet contains errors: Failed to compile stylesheet. 1 error
    detected.
        Marking the metadata as _indexingError=1 in index"

In the BatchEditing and EditorBoard tabs, the normal list of files is
initially blank. However if I place any character in the search box and
try to search, the files then appear.

Finally, the parent-child metadata links seem to have disappeared.

On 2023-05-06 5:32 p.m., Terry wrote:
>
> Thank you for your excellent assistance, Francois!
>
> Today I was able to try the upgrade, and followed that with your two
> recommended Calls to cleanup the JS cache. GeoNetwork took a minute
> or so to respond, which I guess because it was busy, but then
> everything loaded and ran fine. I did not need to use the "?debug"
> suffix.
>
> The minor error message I reported when manually migrating the
> database was because there is no "editable" column in the settings
> database table when manually migrating at the v407 step; the column
> appears later when GeoNetwork is run. When the migration is done
> automatically, it probably works slightly differently.
>
> I find the new splash screen quite inviting. The searches seem much
> faster - and entering even a few characters in a word reveals relevant
> records. The team has done a great job with this release, IMHO.
>
> -- terry
>
> -------- Forwarded Message --------
> Subject: Re: [GeoNetwork-users] Problem updating from GN4.0.6 to
GN4.2.2
> Date: Thu, 4 May 2023 07:56:53 +0200
> From: Francois Prunayre <fx.prunayre@anonymised.com>
> To: Terry <terry.curran@anonymised.com>
>
>
>
> Indeed, no issue in the log. SQL migration sounds to have completed
> successfully (version was updated in settings table).
> H2 databases are only used for caches.
>
> I would try to startup. Call
> http://domain.org/geonetwork/static/wroAPI/reloadModel
> http://domain.org/geonetwork/static/wroAPI/reloadCache
> to force a cleanup of JS cache
>
> and then load the home page and check any error in the log.
>
> To increase log level, you can update the settings and change
> log4j2.xml by log4j2-dev.xml
>
> HTH
> Francois
>
> Le mer. 3 mai 2023 à 20:10, Terry <terry.curran@anonymised.com> a écrit :
>
> <!-- SNIP -->
>
> On 2023-05-02 11:28 p.m., Francois Prunayre wrote:
>> Hi Terry,
>>
>> I would
>> * check the following in the settings table
>> UPDATE Settings SET editable = 'y' WHERE editable IS NULL;
>> to be sure that the value is properly set and there is no null
>> (which should not be possible).
>>
>> * About H2, check
>>
https://geonetwork-opensource.org/manuals/4.0.x/en/overview/change-log/version-4.2.0.html#migration-instructions
>>
>> * Any errors in the logs ?
>>
>> Cheers.
>> Francois
>>
>>
>> Le mer. 3 mai 2023 à 05:37, Terry via GeoNetwork-users
>> <geonetwork-users@lists.sourceforge.net> a écrit :
>>
>> We are trying to avoid a full metadata dump and reload.
>>
>> We have a RHEL8 VM with an Apache Reverse Proxy ahead of a
Jetty
>> webserver hosting GN4.0.6, GS and ERDDAP.We also have a PostGIS
>> database, and a remote gn_dir.Startup and linkages are by a
>> Jetty
>> daemon.Everything seemed to be working for more than a year.
>>
>> To upgrade to GN4.2.2, as usual we downloaded the war,
>> unzipped it, and
>> made changes to three files: config-node/srv.xml to point
>> exclusively at
>> config-db/jndi.xml, jetty-environment.xml to contain the
>> database
>> credentials (name, port, user, pw, and gnDatabaseDialect [to
>> POSTGRESQL]) and config-db/jndi.xml to contain the usual
PostGIS
>> parameters.On some attempts we also set the jdbc.properties
>> to the same,
>> but usually just kept the defaults.
>>
>> Upon reboot with the modified GN, the database got migrated
>> to 4.2.2,
>> with two we think insignificant errors from v407 migration:
>>
>> gndb422=# UPDATE Settings SET editable = 'n' WHERE name =
>> 'system/userFeedback/lastNotificationDate';
>> ERROR: column "editable" of relation "settings" does not exist
>> LINE 1: UPDATE Settings SET editable = 'n' WHERE name =
>> 'system/user...
>> ^
>> gndb422=# UPDATE Settings SET editable = 'n' WHERE name =
>> 'system/security/passwordEnforcement/pattern';
>> ERROR: column "editable" of relation "settings" does not exist
>> LINE 1: UPDATE Settings SET editable = 'n' WHERE name =
>> 'system/secu...
>>
>> We got the title on the GN splash screen from the settings, but
>> otherwise blank screen.We noticed the log had many h2 errors
>> regarding
>> the gn-wro4j disk-cache, and related h2 errors, but also said
>> that
>> geonetwork was available.Stopped jetty, deleted the
>> wro4j-cache.mv.db in
>> remote gn_dir folder, because we knew it would get rebuilt on
>> restart.
>>
>> Rebooted computer and thus jetty, and no "gn-wro4j
>> disk-cache" errors.
>> Got the same header and blank screen.However, appending
>> “?debug” allowed
>> login and we could then see the data.Cleared the “JS & CSS
>> cache” and
>> rebooted.Things looked okay.Tried reindexing and got a 502
Proxy
>> Error.Metadata could no longer be seen.
>>
>> Any thoughts about next step, and/or what we are doing wrong?
>>
>> Terry
>>
>> Pacific Salmon Foundation
>>
>>
>> _______________________________________________
>> GeoNetwork-users mailing list
>> GeoNetwork-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>> GeoNetwork OpenSource is maintained at
>> http://sourceforge.net/projects/geonetwork
>>
>

_______________________________________________
GeoNetwork-users mailing list
GeoNetwork-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork

I tried "delete and reindex" and simply "reindex". I got a reply "reBuildIndexError 502 Proxy Error The proxy server received an invalid response from an upstream server. The proxy server could not handle the request"

However, on the Editor Board I saw the the indexing progressing, and finally "Indexing finished. 1648 records processed."

So - I think GN reindexed in spite of the message. Our reverse proxy is Apache, and just redirects to jetty as https://localhost:8443/ on the same server.

Now that I understand how to work around the BatchEditing and EditorBoard queries by putting a space or two characters in the search box, I can live with that. Only I will experience that anyway.

Otherwise, the application is a remarkable step forward.

I will stop consuming your time - you have been very generous.

Thank you.

On 2023-05-09 11:31 p.m., Francois Prunayre wrote:

Yes every time you change the schema dir, the indexing will be affected so try to reindex first.

About parent child, you are probably talking about the "create child" action which was removed by https://github.com/geonetwork/core-geonetwork/pull/6708 because the function was only relevant for ISO19139 records. You can still manage parent/child relations in the editor but not anymore on record creation.

Cheers.
Francois

Le mer. 10 mai 2023 à 08:14, Terry <terry.curran@anonymised.com> a écrit :

    Answers Interspersed:

    On 2023-05-09 10:19 p.m., Francois Prunayre wrote:

    Hi Terry, are you using an external data directory ? and if yes
    with the schema plugin in it ?
     *** yes, external (/data/gn_dir) and yes - see below

    To check, admin console > statistics > Information >
    Standard
    folder:/opt/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins
     *** yes: Standard folder => /data/gn_dir/config/schema_plugins

    If the schema plugin is not the one from the webapp, then you are
    using the one from your previous version probably.

     *** yes it is the one from the former GN4.0.6 app

    A good practice is to externalize the data directory but keep the
    schema plugin from the webapp (unless you need to customize it) see
    https://geonetwork-opensource.org/manuals/4.0.x/en/install-guide/customizing-data-directory.html?highlight=schemaplugin#advanced-data-directory-configuration
    eg.
    -Dgeonetwork.dir=/data/catalogue
    -Dgeonetwork.schema.dir=/app/tomcat/webapps/geonetwork/WEB-INF/data/config/schema_plugins

    *** jetty.service CHANGED TO (and verified in statistics >
    Information) :
    [Unit]
    Description=Jetty
    After=network-online.target

    [Service]
    Type=simple
    User=jetty
    Group=jetty
    ExecStart= /bin/java -Xms2g -Xmx2g -Djava.awt.headless=true
    -Djava.io.tmpdir=/data/jettytemp \
     -Dgeonetwork.dir=/data/gn_dir -Dlog_dir=/var/www/jetty
    -DSTOP.PORT=8079 -DSTOP.KEY=geonetwork \
    -Dgeonetwork.schema.dir=/var/www/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins
    \
    -Dmime-mappings=/var/www/jetty/webapps/geonetwork/WEB-INF/mime-types.properties
    \
     -jar
    /usr/share/java/jetty-distribution-9.4.36.v20210114/start.jar
    jetty.base=/var/www/jetty
    ExecStop=/bin/kill ${MAINPID}
    SuccessExitStatus=143

    [Install]
    WantedBy=multi-user.target

    Let us know if that works for you

     *** but still same responses - same error msg, same BatchEditing
    and EditorBoard responses, same lost parent-child

     *** should I try re-indexing?

    Francois

    Le mar. 9 mai 2023 à 19:45, Terry via GeoNetwork-users
    <geonetwork-users@lists.sourceforge.net> a écrit :

        I have been looking a little further into my update, and it
        seems there
        is still something unresolved. The problems show up when I
        sign in as
        administrator. When I select "Statistics and status" in the
        Admin
        console, I get "Query returned an error. Check console for
        details".
        Similarly for Contribute/BatchEditing and Console/EditorBoard.

        The jetty log has a message:

         "Error at xsl:call-template on line 578 column 62 of
        index.xsl:
         XTSE0650: No template exists named
        build-thesaurus-fields
         2023-05-09T09:32:16,764 ERROR [geonetwork.index] - Indexing
         stylesheet contains errors: Failed to compile stylesheet.
        1 error
         detected.
         Marking the metadata as _indexingError=1 in index"

        In the BatchEditing and EditorBoard tabs, the normal list of
        files is
        initially blank. However if I place any character in the
        search box and
        try to search, the files then appear.

        Finally, the parent-child metadata links seem to have
        disappeared.

        On 2023-05-06 5:32 p.m., Terry wrote:
        >
        > Thank you for your excellent assistance, Francois!
        >
        > Today I was able to try the upgrade, and followed that with
        your two
        > recommended Calls to cleanup the JS cache. GeoNetwork took
        a minute
        > or so to respond, which I guess because it was busy, but then
        > everything loaded and ran fine. I did not need to use the
        "?debug"
        > suffix.
        >
        > The minor error message I reported when manually migrating the
        > database was because there is no "editable" column in the
        settings
        > database table when manually migrating at the v407 step;
        the column
        > appears later when GeoNetwork is run. When the migration
        is done
        > automatically, it probably works slightly differently.
        >
        > I find the new splash screen quite inviting. The searches
        seem much
        > faster - and entering even a few characters in a word
        reveals relevant
        > records. The team has done a great job with this release,
        IMHO.
        >
        > -- terry
        >
        > -------- Forwarded Message --------
        > Subject: Re: [GeoNetwork-users] Problem updating from
        GN4.0.6 to GN4.2.2
        > Date: Thu, 4 May 2023 07:56:53 +0200
        > From: Francois Prunayre <fx.prunayre@anonymised.com>
        > To: Terry <terry.curran@anonymised.com>
        >
        > Indeed, no issue in the log. SQL migration sounds to have
        completed
        > successfully (version was updated in settings table).
        > H2 databases are only used for caches.
        >
        > I would try to startup. Call
        > http://domain.org/geonetwork/static/wroAPI/reloadModel
        > http://domain.org/geonetwork/static/wroAPI/reloadCache
        > to force a cleanup of JS cache
        >
        > and then load the home page and check any error in the log.
        >
        > To increase log level, you can update the settings and change
        > log4j2.xml by log4j2-dev.xml
        >
        > HTH
        > Francois
        >
        > Le mer. 3 mai 2023 à 20:10, Terry <terry.curran@anonymised.com> a
        écrit :
        >
        > <!-- SNIP -->
        >
        > On 2023-05-02 11:28 p.m., Francois Prunayre wrote:
        >> Hi Terry,
        >>
        >> I would
        >> * check the following in the settings table
        >> UPDATE Settings SET editable = 'y' WHERE editable IS NULL;
        >> to be sure that the value is properly set and there is
        no null
        >> (which should not be possible).
        >>
        >> * About H2, check
        >>
        https://geonetwork-opensource.org/manuals/4.0.x/en/overview/change-log/version-4.2.0.html#migration-instructions
        >>
        >> * Any errors in the logs ?
        >>
        >> Cheers.
        >> Francois
        >>
        >> Le mer. 3 mai 2023 à 05:37, Terry via GeoNetwork-users
        >> <geonetwork-users@lists.sourceforge.net> a écrit :
        >>
        >> We are trying to avoid a full metadata dump and
        reload.
        >>
        >> We have a RHEL8 VM with an Apache Reverse Proxy
        ahead of a Jetty
        >> webserver hosting GN4.0.6, GS and ERDDAP.We also
        have a PostGIS
        >> database, and a remote gn_dir.Startup and linkages
        are by a
        >> Jetty
        >> daemon.Everything seemed to be working for more
        than a year.
        >>
        >> To upgrade to GN4.2.2, as usual we downloaded the war,
        >> unzipped it, and
        >> made changes to three files: config-node/srv.xml
        to point
        >> exclusively at
        >> config-db/jndi.xml, jetty-environment.xml to
        contain the
        >> database
        >> credentials (name, port, user, pw, and
        gnDatabaseDialect [to
        >> POSTGRESQL]) and config-db/jndi.xml to contain the
        usual PostGIS
        >> parameters.On some attempts we also set the
        jdbc.properties
        >> to the same,
        >> but usually just kept the defaults.
        >>
        >> Upon reboot with the modified GN, the database got
        migrated
        >> to 4.2.2,
        >> with two we think insignificant errors from v407
        migration:
        >>
        >> gndb422=# UPDATE Settings SET editable = 'n' WHERE
        name =
        >> 'system/userFeedback/lastNotificationDate';
        >> ERROR: column "editable" of relation "settings"
        does not exist
        >> LINE 1: UPDATE Settings SET editable = 'n' WHERE
        name =
        >> 'system/user...
        >> ^
        >> gndb422=# UPDATE Settings SET editable = 'n' WHERE
        name =
        >> 'system/security/passwordEnforcement/pattern';
        >> ERROR: column "editable" of relation "settings"
        does not exist
        >> LINE 1: UPDATE Settings SET editable = 'n' WHERE
        name =
        >> 'system/secu...
        >>
        >> We got the title on the GN splash screen from the
        settings, but
        >> otherwise blank screen.We noticed the log had many
        h2 errors
        >> regarding
        >> the gn-wro4j disk-cache, and related h2 errors,
        but also said
        >> that
        >> geonetwork was available.Stopped jetty, deleted the
        >> wro4j-cache.mv.db in
        >> remote gn_dir folder, because we knew it would get
        rebuilt on
        >> restart.
        >>
        >> Rebooted computer and thus jetty, and no "gn-wro4j
        >> disk-cache" errors.
        >> Got the same header and blank screen.However,
        appending
        >> “?debug” allowed
        >> login and we could then see the data.Cleared the
        “JS & CSS
        >> cache” and
        >> rebooted.Things looked okay.Tried reindexing and
        got a 502 Proxy
        >> Error.Metadata could no longer be seen.
        >>
        >> Any thoughts about next step, and/or what we are
        doing wrong?
        >>
        >> Terry
        >>
        >> Pacific Salmon Foundation
        >>
        >> _______________________________________________
        >> GeoNetwork-users mailing list
        >> GeoNetwork-users@lists.sourceforge.net
        >> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
        >> GeoNetwork OpenSource is maintained at
        >> http://sourceforge.net/projects/geonetwork
        >>
        >

        _______________________________________________
        GeoNetwork-users mailing list
        GeoNetwork-users@lists.sourceforge.net
        https://lists.sourceforge.net/lists/listinfo/geonetwork-users
        GeoNetwork OpenSource is maintained at
        http://sourceforge.net/projects/geonetwork