WFS Insert responds with incorrect id following a Delete Transaction

A WFS Insert Transaction, if performed following a WFS Delete Transaction does an insert but, the return value shows an incorrect ID for the inserted record. More specifically, this happens when the delete is of the record with the highest primary key.

Database table, ISS_FEATURES, has primary key FEATURE_ID which is populated by an insert trigger fetching from sequence ISS_FEATURES_FEATURE_ID_SEQ.

Table GT_PK_METADATA entry is:

TABLE_SCHEMA: APP_ISS
TABLE_NAME: ISS_FEATURES
PK_COLUMN: FEATURE_ID
PK_COLUMN_IDX: null
PK_POLICY: sequence
PK_SEQUENCE: ISS_FEATURES_FEATURE_ID_SEQ

GeoServer Data Source has
Expose primary keys checked
Primary key metadata table: GT_PK_METADATA

If the highest PK is 2348, and the next highest PK is 2344, then a delete POST of:

<Transaction xmlns="http://www.opengis.net/wfs" service="WFS" version="1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.1.0/wfs.xsd">
<Delete typeName="iss:ISS_FEATURES" xmlns:feature="iss">
<Filter xmlns="http://www.opengis.net/ogc"><FeatureId fid="ISS_FEATURES.2348”/></Filter>
</Delete>
</Transaction>

Followed by and insert POST of:

<Transaction xmlns="http://www.opengis.net/wfs" service="WFS" version="1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wfs http://schemas.opengis.net/wfs/1.1.0/wfs.xsd">
<Insert>
<ISS_FEATURES xmlns="iss">
<GEOMETRY><Point xmlns="http://www.opengis.net/gml" srsName="EPSG:3857"><pos srsDimension="2">-13856700.583603898 6269882.445366477</pos></Point></GEOMETRY>
<NAME>expecting ID of 2349</NAME>
</ISS_FEATURES>
</Insert>
</Transaction>

Produces this response:

<?xml version="1.0" encoding="UTF-8"?><wfs:TransactionResponse xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:wfs="http://www.opengis.net/wfs" xmlns:gml="http://www.opengis.net/gml" xmlns:ogc="http://www.opengis.net/ogc" xmlns:ows="http://www.opengis.net/ows" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.1.0" xsi:schemaLocation="http://www.opengis.net/wfs http://hixon.th.gov.bc.ca:8081/ogs-geoV06/schemas/wfs/1.1.0/wfs.xsd"><wfs:TransactionSummary>
<wfs:totalInserted>1</wfs:totalInserted>
<wfs:totalUpdated>0</wfs:totalUpdated><wfs:totalDeleted>0</wfs:totalDeleted></wfs:TransactionSummary><wfs:TransactionResults/><wfs:InsertResults>
<wfs:Feature>
<ogc:FeatureId fid="ISS_FEATURES.2345"/>
</wfs:Feature>
</wfs:InsertResults>
</wfs:TransactionResponse>

Examining the inserted record shows it has a FEATURE_ID = 2349.

My suspicion is that GeoServer is unable to view the NEXTVAL of the sequence in GT_PK_METADATA, and falling back on something like

SELECT MAX(FEATURE_ID) + 1 FROM ISS_FEATURES

to report the inserted PK.

Running on:

GeoServer 2.25.2
Tomcat 9.0.102
Open JDK 17.0.11_9
OS RHEL 9.5
Oracle 12c

WFS 1.1.0 requests return an incorrect ogc:FeatureId fid.

Repeating using WFS Version 2.0.0 Insert returns an incorrect fes:ResourceId rid.

Repeated using GT_PK_METADATA.POLICY of assigned and autogenerated.

Anyone else experienced this?

Any thoughts as to where to look next?

Many thanks in advance for clues or hints.

Peter

Not many developers have easy access to an Oracle environment; the next step for me would be to trouble shoot in a debugger to watch the error occur.

The same effect may be possible by turning on GEOTOOLS_DEVELOPER_LOGGING profile and carefully watch the record being inserted to confirm your FEATURE_ID + 1 suspicion.

Learning enough to reproduce the problem first in GeoServer, and then in GeoTools library directly would be ideal.

aside: The GeoTools library is responsible for Feature input / output and does have automated tests running against oracle.

Hi Jody.
The GEOTOOLS_DEVELOPER_LOGGING shows the INSERT followed by a CLOSE CONNECTION. No sign of a SELECT MAX(FEATURE_ID)+1 (log output below). So not clear how it is getting the ID value.
I will look into the GeoTools code to see if there are any clues there since I’m not sure about navigating the Oracle Online Test.

Thanks!

11 Apr 16:05:47 INFO   [geoserver.gwc] - DataStoreChange: {http://th.gov.bc.ca/iss}ISS_FEATURES TransactionEventType[PreInsert]
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - CREATE CONNECTION
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - SELECT FEATURE_ID,NAME,DESCRIPTION,CREATED_TIMESTAMP,UPDATED_TIMESTAMP,GEOMETRY as GEOMETRY,IMAGE1 FROM APP_ISS.ISS_FEATURES WHERE 0 = 1
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting FEATURE_ID to 0
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting 0 to 0
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting NAME to INSERTING kangaroo after largest PK deleted
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting 1 to INSERTING kangaroo after largest PK deleted
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting DESCRIPTION to null
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting 2 to null
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting CREATED_TIMESTAMP to null
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting 3 to null
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting UPDATED_TIMESTAMP to null
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting 4 to null
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting GEOMETRY to POINT (1111722.982839495 440511.1363148169)
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting 5 to POINT (1111722.982839495 440511.1363148169)
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting IMAGE1 to null
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting 6 to null
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Inserting new features with ps: INSERT INTO APP_ISS.ISS_FEATURES ( NAME,DESCRIPTION,CREATED_TIMESTAMP,UPDATED_TIMESTAMP,GEOMETRY,IMAGE1,FEATURE_ID ) VALUES ( ?,?,?,?,?,?,?)
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - 1 = INSERTING after largest PK deleted
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - 2 = null
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - 3 = null
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - 4 = null
11 Apr 16:05:47 DEBUG  [oracle.sdo] - Using layer SRID: 3005
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - Setting parameter 5 as MDSYS.SDO_GEOMETRY(2001,3005,MDSYS.SDO_POINT_TYPE(1111722.982839495,440511.1363148169,NULL),NULL,NULL)
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - 5 = POINT (1111722.982839495 440511.1363148169)
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - 6 = null
11 Apr 16:05:47 INFO   [geoserver.gwc] - DataStoreChange: {http://th.gov.bc.ca/iss}ISS_FEATURES TransactionEventType[PostInsert]
11 Apr 16:05:47 DEBUG  [geotools.jdbc] - CLOSE CONNECTION
11 Apr 16:05:47 INFO   [geoserver.wfs] - 
Request: transaction
    service = WFS
    version = 2.0.0
    baseUrl = http://hixon.th.gov.bc.ca:8081/ogs-geoV06/
    group[0] = wfs:abstractTransactionAction=net.opengis.wfs20.impl.InsertTypeImpl@713f8cee (handle: null) (any: [SimpleFeatureImpl:ISS_FEATURES=[SimpleFeatureImpl.Attribute: FEATURE_ID<FEATURE_ID id=fid--48bbd3cb_19626f56d98_-2661>=0, SimpleFeatureImpl.Attribute: NAME<NAME id=fid--48bbd3cb_19626f56d98_-2661>=INSERTING kangaroo after largest PK deleted, SimpleFeatureImpl.Attribute: DESCRIPTION<DESCRIPTION id=fid--48bbd3cb_19626f56d98_-2661>=null, SimpleFeatureImpl.Attribute: CREATED_TIMESTAMP<CREATED_TIMESTAMP id=fid--48bbd3cb_19626f56d98_-2661>=null, SimpleFeatureImpl.Attribute: UPDATED_TIMESTAMP<UPDATED_TIMESTAMP id=fid--48bbd3cb_19626f56d98_-2661>=null, SimpleFeatureImpl.Attribute: GEOMETRY<GEOMETRY id=fid--48bbd3cb_19626f56d98_-2661>=POINT (-13856600.583603898 6269782.445366477), SimpleFeatureImpl.Attribute: IMAGE1<IMAGE1 id=fid--48bbd3cb_19626f56d98_-2661>=null]], inputFormat: <unset>, srsName: null)
    abstractTransactionActionGroup[0] = wfs:abstractTransactionAction=net.opengis.wfs20.impl.InsertTypeImpl@713f8cee (handle: null) (any: [SimpleFeatureImpl:ISS_FEATURES=[SimpleFeatureImpl.Attribute: FEATURE_ID<FEATURE_ID id=fid--48bbd3cb_19626f56d98_-2661>=0, SimpleFeatureImpl.Attribute: NAME<NAME id=fid--48bbd3cb_19626f56d98_-2661>=INSERTING after largest PK deleted, SimpleFeatureImpl.Attribute: DESCRIPTION<DESCRIPTION id=fid--48bbd3cb_19626f56d98_-2661>=null, SimpleFeatureImpl.Attribute: CREATED_TIMESTAMP<CREATED_TIMESTAMP id=fid--48bbd3cb_19626f56d98_-2661>=null, SimpleFeatureImpl.Attribute: UPDATED_TIMESTAMP<UPDATED_TIMESTAMP id=fid--48bbd3cb_19626f56d98_-2661>=null, SimpleFeatureImpl.Attribute: GEOMETRY<GEOMETRY id=fid--48bbd3cb_19626f56d98_-2661>=POINT (-13856600.583603898 6269782.445366477), SimpleFeatureImpl.Attribute: IMAGE1<IMAGE1 id=fid--48bbd3cb_19626f56d98_-2661>=null]], inputFormat: <unset>, srsName: null)
    abstractTransactionAction[0]:
        any[0] = SimpleFeatureImpl:ISS_FEATURES=[SimpleFeatureImpl.Attribute: FEATURE_ID<FEATURE_ID id=fid--48bbd3cb_19626f56d98_-2661>=0, SimpleFeatureImpl.Attribute: NAME<NAME id=fid--48bbd3cb_19626f56d98_-2661>=INSERTING after largest PK deleted, SimpleFeatureImpl.Attribute: DESCRIPTION<DESCRIPTION id=fid--48bbd3cb_19626f56d98_-2661>=null, SimpleFeatureImpl.Attribute: CREATED_TIMESTAMP<CREATED_TIMESTAMP id=fid--48bbd3cb_19626f56d98_-2661>=null, SimpleFeatureImpl.Attribute: UPDATED_TIMESTAMP<UPDATED_TIMESTAMP id=fid--48bbd3cb_19626f56d98_-2661>=null, SimpleFeatureImpl.Attribute: GEOMETRY<GEOMETRY id=fid--48bbd3cb_19626f56d98_-2661>=POINT (-13856600.583603898 6269782.445366477), SimpleFeatureImpl.Attribute: IMAGE1<IMAGE1 id=fid--48bbd3cb_19626f56d98_-2661>=null]
        inputFormat = application/gml+xml; version=3.2
    releaseAction = ALL
11 Apr 16:05:47 INFO   [geoserver.filters] - 142.35.221.104 "POST /ogs-geoV06/wfs" took 104ms

Hi,
as far as I can see, there is very little logging in the primary key management
(see the KeysFetcher class in gt-jdbc, as well as the PrimaryKeyFetcher subclasses)
so a debugging session is needed.

If you go down that road, it would be nice if you could also add the missing log statements in the
code, at FINE level, and share a PR to ease future debugging.

If I had to make a blind guess… the primary key table is not visible the the user, or it’s not in the
same schema setup in the datastore configuration panel, as I don’t see anywhere the few log
messages MetadataTablePrimaryKeyFinder would generate (then again, they might have been
emitted earlier).

Turn on statement logging on for a while on the Oracle side and capture the SQL that Geoserver is sending.
I know that WFS-T worked fine for us with Oracle, but it was 16 years ago.

-Jukka Rahkonen-

Indeed there were two visibility issues.

  1. Proxy account used for inserts did not have SELECT on GT_PK_METADATA
  2. GT_PK_METADATA.PK_SEQUENCE value did not specify the schema.

Once those two issues were fixed, GEO_TOOLS_DEVELOPER logging shows:
14 Apr 10:08:31 DEBUG [geotools.jdbc] - Inserting new features with ps: INSERT INTO APP_ISS.ISS_FEATURES ( NAME,DESCRIPTION,CREATED_TIMESTAMP,UPDATED_TIMESTAMP,GEOMETRY,IMAGE1,FEATURE_ID ) VALUES ( ?,?,?,?,?,?,APP_ISS.ISS_FEATURES_FEATURE_ID_SEQ.NEXTVAL)

Many thanks Andrea!!