[Geoserver-devel] Javadocs for DataSource locking extension

Attached are java docs for extending the DataSource API to handle locking.
Please have a look at the attached files, as I would love some feedback before coding next week.

To recap:
- the goal is to provide strong transaction support for geoserver (use the database rather than just geoserver)
- using per row locking, while easy to implement for geotools2, does not meet the locking requirements for geoserver

The API proposed:
- and can be implement in Postgis.
- can be implemented in geotools2 by porting geoservers locking code over to an AbstractTransactionalDatasource

After getting this up and going for Postgis I will be working on providing support for OracleSpatial and ArcSDE.

Thanks,

Jody Garnett

Example Use: TypeRepository.addToLock

  List fids = getFidFeatures(typeName, filter);
  DataSource ds = dataSourceByName(typeName);

  Lock lock = Lock.create(1800);
  if ( lockAll && ds instanceof TransactionalDataSource ){
    TransactionalDataSource tds = (TransactionalDataSource) ds;
        try {
          tds.setCurrentLock(locky);
      tds.lock(filter);
      tds.commit();
    } catch (DataSourceException e) {
      throw new WfsException(e);
    }
    }
  InternalLock internallock = (InternalLock)locks.get( lock.getID() );
  internallock.addFeatures(lockAll, getFidFeatures(typeName, filter));

(attachments)

TransactionalDataSource.html (22 KB)
Lock.html (13.2 KB)
PostgisDataSource.html (80.4 KB)

After getting this up and going for Postgis I will be working on providing support for OracleSpatial and ArcSDE.

Hi Jody,

Are you planning to creating a separate datasources for PostGIS and Oracle Spatial that provide locking, or extending the existing ones? Also, do you have a working ArcSDE datasource that you will be adding locking to?

Thanks,
Sean Geoghegan

Oh, as an additional thought, do you have any requirements for connection pooling in the datasources? I plan to add connection pooling to the oracle datasource next week. The way I have planned to do it was that for each connection url a ConnectionPool is created. The DataSource is passed a reference to this connection pool, instead of a Connection object. Then for each method that requires database access, the DataSource gets a connection from the pool and then closes it (returns it the pool) when the method is finished. This should stop DataSources holding on to connections for extended periods or time and reduce the connection load.

Is this going to work okay with your locking plans?

Thanks

Sean Geoghegan wrote:

After getting this up and going for Postgis I will be working on providing support for OracleSpatial and ArcSDE.

Hi Jody,

Are you planning to creating a separate datasources for PostGIS and Oracle Spatial that provide locking, or extending the existing ones? Also, do you have a working ArcSDE datasource that you will be adding locking to?

Thanks,
Sean Geoghegan

-------------------------------------------------------
This SF.Net email sponsored by: Free pre-built ASP.NET sites including
Data Reports, E-commerce, Portals, and Forums are available now.
Download today and enter to win an XBOX or Visual Studio .NET.
http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01

_______________________________________________
Geotools-devel mailing list
Geotools-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geotools-devel

I spent the morning looking this over and I think it looks great. I like
setCurrentLock operation as a way to provide short or long transactions.
GeoServer probably won't use it, but it makes good sense to provide it for
geotools, as users could desire that sort of locking, as for many it makes
more sense than the lockId long transaction style. And the locking table
in postgis definitely seems the way to go. I thought of something like
this when I was looking for initial solutions to the locking problem, but
yours is far more elegant than anything I came up with.
  One question I do have is about the
AbstractTransactionalDatasource - is it going to be JDBC based or not?
You can obviously reduce a lot of code duplication if you do that, as you
can probably handle most of the connection handling there. But if you do
that we should figure out if it will ever be possible to have file based
datasources be transactional. Obviously that functionaliy will be hacked
in, as files don't lend themselves naturally to transaction operations.
The notion has been kicked around in geotools for a bit, at least
implementing add, modify and remove in shapefile by having some sort of
transaction recorder that holds the modified features in memory until
commit() is called, at which point setFeatures() would be called. I
suppose a locking similar to GeoServer could be implemented there as well.
If done correctly I could see an AbstractFileTransactionalDatasource that
could handle add, remove and modify operations, as well as commit/rollback
and perhaps even locking if the file datasources were to just implement
setFeatures. The overlap in code between FileTransactionalDatasource and
JDBCTransactionalDatasource would probably be fairly minimal. This
obviously does not have to be figured out right now, but if you are
planning on common jdbc code in abstractTransDS, then you might consider
renaming to rbdmDataSource or something to indicate that file based
datasources can't extend it. But we can also just cross that bridge when
someone actually tries to make shapefile transactional.
  As for your note about addFeatures() with locking, we should ask
the spec author. I could see that if the whole datasource was locked then
an insert should not be allowed, but we should find out for sure first, as
implementing it only has the one purpose. But yes, the javadocs look
great, I'm excited to see it coded up.

  Chris

On Fri, 25 Jul 2003, Jody Garnett wrote:

Attached are java docs for extending the DataSource API to handle locking.
Please have a look at the attached files, as I would love some feedback
before coding next week.

To recap:
- the goal is to provide strong transaction support for geoserver (use
the database rather than just geoserver)
- using per row locking, while easy to implement for geotools2, does not
meet the locking requirements for geoserver

The API proposed:
- and can be implement in Postgis.
- can be implemented in geotools2 by porting geoservers locking code
over to an AbstractTransactionalDatasource

After getting this up and going for Postgis I will be working on
providing support for OracleSpatial and ArcSDE.

Thanks,

Jody Garnett

Example Use: TypeRepository.addToLock

  List fids = getFidFeatures(typeName, filter);
  DataSource ds = dataSourceByName(typeName);

  Lock lock = Lock.create(1800);
  if ( lockAll && ds instanceof TransactionalDataSource ){
    TransactionalDataSource tds = (TransactionalDataSource) ds;
        try {
          tds.setCurrentLock(locky);
      tds.lock(filter);
      tds.commit();
    } catch (DataSourceException e) {
      throw new WfsException(e);
    }
    }
  InternalLock internallock = (InternalLock)locks.get( lock.getID() );
  internallock.addFeatures(lockAll, getFidFeatures(typeName, filter));

The way I have planned to do it was
that for each connection url a ConnectionPool is created. The
DataSource is passed a reference to this connection pool, instead of a
Connection object. Then for each method that requires database access,
the DataSource gets a connection from the pool and then closes it
(returns it the pool) when the method is finished. This should stop
DataSources holding on to connections for extended periods or time and
reduce the connection load.

That sounds good, the one thing to be sure of is that the datasources can
hold on to connections for transactions, so that more than one method can
operate on the same connection. If setAutoCommit is set to true then the
datasource should hold onto the same connection for all its
update/insert/delete operations, until a rollback or a commit. If
getFeatures is called, though, it should _not_ use that connection, since
we don't want it to have the same view until the features are committed.
You probably already have this in mind, I just want to be sure so you
don't have to go back and recode it. And Jody can correct me if I'm
wrong, but I'm pretty sure his locking plans will work fine if you do what
I just described, keep the connection for the updating operations, and he
can use it for the short transaction locking operations, and the long
transaction stuff should work fine on different connections.

  Chris

Thanks

Sean Geoghegan wrote:

>
>>
>> After getting this up and going for Postgis I will be working on
>> providing support for OracleSpatial and ArcSDE.
>
>
>
> Hi Jody,
>
> Are you planning to creating a separate datasources for PostGIS and
> Oracle Spatial that provide locking, or extending the existing ones?
> Also, do you have a working ArcSDE datasource that you will be adding
> locking to?
>
> Thanks,
> Sean Geoghegan
>
>
>
> -------------------------------------------------------
> This SF.Net email sponsored by: Free pre-built ASP.NET sites including
> Data Reports, E-commerce, Portals, and Forums are available now.
> Download today and enter to win an XBOX or Visual Studio .NET.
> http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
>
> _______________________________________________
> Geotools-devel mailing list
> Geotools-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/geotools-devel