[Geoserver-devel] Avoding two styles to get connected, GEOS-5545

GeoServer GeoServer Developer
1

Hi,
as described in http://jira.codehaus.org/browse/GEOS-5545
there are situations in which we can get two styles “related” in such
a way that if you modify one, you’ll modify the other as well.

I proposed a solution… one year ago, and tried to get some feedback
for the past month, without success, so I’m turning this to the dev list.

The idea would be to append a progressive number to the files
in case of conflict, so that the link between the style name and the
file is kept apparent, as people sometimes edit those files directly,
but without exposing the filenames to the general population of users

Feedback?

Cheers
Andrea

–

==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

2

I would rather expose the file name than start hacking it up, it seems this is a bug that could be fixed with impact then changing file names.

If we had to start hacking it up we may as well consider it not editable on the file system and then embedded just the style part we use in our xstream.

It is not like we use a full SLD to configure an entire map of layers, usually we only do one style for one layer.

On Saturday, May 31, 2014, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
as described in http://jira.codehaus.org/browse/GEOS-5545
there are situations in which we can get two styles “related” in such
a way that if you modify one, you’ll modify the other as well.

I proposed a solution… one year ago, and tried to get some feedback
for the past month, without success, so I’m turning this to the dev list.

The idea would be to append a progressive number to the files
in case of conflict, so that the link between the style name and the
file is kept apparent, as people sometimes edit those files directly,
but without exposing the filenames to the general population of users

Feedback?

Cheers
Andrea

–

==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

–

Jody Garnett

3

On Fri, May 30, 2014 at 10:14 PM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

I would rather expose the file name than start hacking it up, it seems
this is a bug that could be fixed with impact then changing file names.

I find it really ugly... from the user point of view the style is one, it
has a name and a body,
the fact there are two files is really an accident of implementation. Plus,
once exposed, it's hard to remove

If we had to start hacking it up we may as well consider it not editable
on the file system and then embedded just the style part we use in our
xstream.

That would be better imho... but I guess it's something we can do only on
trunk right?
Another angle we can look at this, is that the "mistake" is that we let the
people rename the style
without renaming the associated sld file, the reason the issue happens is
that they can drift apart
from each other.

It is not like we use a full SLD to configure an entire map of layers,
usually we only do one style for one layer.

Leave the usually out, the only way to define a map with SLD is to make a
GetMap request with no layers and styles,
but only the SLD (as reference, or sld_body), indeed, all we need out of a
SLD file is first UserStyle, everything
else is ignored.

Cheers
Andrea

--

Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

4

I like having files on disk (look a standard!). If we could fix the rename it would be easiest.

Long term I would like to use SEnfiles to reflect our use ( no more SLDwrapping ). And folding the SE into a CDATA section does make a lot of sense.

Q: What extra stuff does our style catalog entry have? Can we encode the details in the SLD wrapping?

···

If we had to start hacking it up we may as well consider it not editable on the file system and then embedded just the style part we use in our xstream.

That would be better imho… but I guess it’s something we can do only on trunk right?
Another angle we can look at this, is that the “mistake” is that we let the people rename the style
without renaming the associated sld file, the reason the issue happens is that they can drift apart
from each other.

It is not like we use a full SLD to configure an entire map of layers, usually we only do one style for one layer.

Leave the usually out, the only way to define a map with SLD is to make a GetMap request with no layers and styles,
but only the SLD (as reference, or sld_body), indeed, all we need out of a SLD file is first UserStyle, everything
else is ignored.

Cheers
Andrea

–

==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

5

On Fri, May 30, 2014 at 11:32 PM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

I like having files on disk (look a standard!). If we could fix the

rename it would be easiest.

You mean, have the sld file name follow with xml name? Yeah, we could go
there too,
it's not a problem unless the catalog modifications are protected by the
global config lock
(it does not have to be global, long term it would be nice to have
transactions...)

Long term I would like to use SEnfiles to reflect our use ( no more
SLDwrapping ). And folding the SE into a CDATA section does make a lot of
sense.

Yeah, makes sense, but we first need first class SE support to go there, so
far I'm not even
sure we're on par with SLD 1.0 (face it, there is simply no funding to go
there, whilst funding and
interest shows up for CSS instead).

Q: What extra stuff does our style catalog entry have? Can we encode the
details in the SLD wrapping?

The xml file is just id, name and sld location:

<style>
  <id>StyleInfoImpl--570ae188:124761b8d78:-7fe0</id>
  <name>polygon</name>
  <filename>default_polygon.sld</filename>
</style>

It could easily become:

<style>
  <id>StyleInfoImpl--570ae188:124761b8d78:-7fe0</id>
  <name>polygon</name>
  <content>
     ... sld file here
  </content>
</style>

Cheers
Andrea

--

Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

6

Q: What extra stuff does our style catalog entry have? Can we encode the
details in the SLD wrapping?

The xml file is just id, name and sld location:

<style>
  <id>StyleInfoImpl--570ae188:124761b8d78:-7fe0</id>
  <name>polygon</name>
  <filename>default_polygon.sld</filename>
</style>

It could easily become:

<style>
  <id>StyleInfoImpl--570ae188:124761b8d78:-7fe0</id>
  <name>polygon</name>
  <content>
     ... sld file here
  </content>
</style>

I am really liking that, however I was trying to ask about the other way
around (since SLD is already a wrapper around our core SE content).

<StyledLayerDescriptor>
  <Name>polygon</Name>
<Description>StyleInfoImpl--570ae188:124761b8d78:-7fe0</Description>
<UserLayer>
   ...se content goes here
</UserLayer>
</StyledLayerDescriptor>

7

On Sat, May 31, 2014 at 3:35 PM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

Q: What extra stuff does our style catalog entry have? Can we encode the

details in the SLD wrapping?

The xml file is just id, name and sld location:

<style>
  <id>StyleInfoImpl--570ae188:124761b8d78:-7fe0</id>
  <name>polygon</name>
  <filename>default_polygon.sld</filename>
</style>

It could easily become:

<style>
  <id>StyleInfoImpl--570ae188:124761b8d78:-7fe0</id>
  <name>polygon</name>
  <content>
     ... sld file here
  </content>
</style>

I am really liking that, however I was trying to ask about the other way
around (since SLD is already a wrapper around our core SE content).

<StyledLayerDescriptor>
  <Name>polygon</Name>
<Description>StyleInfoImpl--570ae188:124761b8d78:-7fe0</Description>
<UserLayer>
   ...se content goes here
</UserLayer>
</StyledLayerDescriptor>

Could work, but since we allow the user to edit the full SLD files, it
would end up being right
in the user face, and if the user ends up modifying that section, we'd have
to rewrite it

Cheers
Andrea

--

Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

8

Sorry I assumed we would only let them edit the middle bit, was just thinking that SLD could be our wrapper.

Still either way works … so back on tract:

a) can we fix rename?
b) can we get down to a single file (either embed SLD, our use SLD as a wrapper)

Did I miss anything?

···

Jody Garnett

On Sat, May 31, 2014 at 11:50 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Sat, May 31, 2014 at 3:35 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Could work, but since we allow the user to edit the full SLD files, it would end up being right
in the user face, and if the user ends up modifying that section, we’d have to rewrite it

Cheers

Andrea

–

==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

I am really liking that, however I was trying to ask about the other way around (since SLD is already a wrapper around our core SE content).

polygon
StyleInfoImpl–570ae188:124761b8d78:-7fe0

…se content goes here

Q: What extra stuff does our style catalog entry have? Can we encode the details in the SLD wrapping?

The xml file is just id, name and sld location:

StyleInfoImpl--570ae188:124761b8d78:-7fe0 polygon default_polygon.sld

It could easily become:

StyleInfoImpl--570ae188:124761b8d78:-7fe0 polygon ... sld file here
9

On Sat, May 31, 2014 at 4:05 PM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

Sorry I assumed we would only let them edit the middle bit, was just
thinking that SLD could be our wrapper.

Still either way works ... so back on tract:

a) can we fix rename?
b) can we get down to a single file (either embed SLD, our use SLD as a
wrapper)

I would go for a) since I'd need this fix for the stable series, and I had
made a proposal:
* rename the sld file as we rename the xml file (as the style name changes)
* append random numbers between 0 and 1000 to the file name if the target
file name is already in use (it may happen if we start from a legacy data
dir, since the file xml and sld names might be out of synch)

Cheers
Andrea

--

Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

10

Sounds good Andrea, you can also consider simply rejecting the filename in the event of conflict (and asking the user to rename the file).

···

Jody Garnett

On Sun, Jun 1, 2014 at 6:19 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Sat, May 31, 2014 at 4:05 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sorry I assumed we would only let them edit the middle bit, was just thinking that SLD could be our wrapper.

Still either way works … so back on tract:

a) can we fix rename?
b) can we get down to a single file (either embed SLD, our use SLD as a wrapper)

I would go for a) since I’d need this fix for the stable series, and I had made a proposal:

  • rename the sld file as we rename the xml file (as the style name changes)
  • append random numbers between 0 and 1000 to the file name if the target file name is already in use (it may happen if we start from a legacy data dir, since the file xml and sld names might be out of synch)

Cheers

Andrea

–

==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

11

Sorry, chiming in on this one late. If we are talking about moving the SLD content inline then doesn’t that rule out storing styles in other languages (ie CSS)? This is actually something I am looking at doing sooner rather than later for a client. So if this gets moved inline I’ll need a way to specify that the content is stored externally.

In general I don’t really like the idea of storing the contents inline although that may just be because i am used to the current way of things. But i do feel that having the contents separated is more flexible. All in all I like the idea Andrea originally proposed and that is detecting the conflict and just using a suffix to ensure there are no file names clashes.

···

On Sat, May 31, 2014 at 7:57 PM, Jody Garnett <jody.garnett@anonymised.com> wrote:

Sounds good Andrea, you can also consider simply rejecting the filename in the event of conflict (and asking the user to rename the file).

Jody

Time is money. Stop wasting it! Get your web API in 5 minutes.
www.restlet.com/download
http://p.sf.net/sfu/restlet

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–

Justin Deoliveira
Vice President, Engineering | Boundless
jdeolive@anonymised.com
@j_deolive

Jody Garnett

On Sun, Jun 1, 2014 at 6:19 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Sat, May 31, 2014 at 4:05 PM, Jody Garnett <jody.garnett@anonymised.com.> wrote:

Sorry I assumed we would only let them edit the middle bit, was just thinking that SLD could be our wrapper.

Still either way works … so back on tract:

a) can we fix rename?
b) can we get down to a single file (either embed SLD, our use SLD as a wrapper)

I would go for a) since I’d need this fix for the stable series, and I had made a proposal:

  • rename the sld file as we rename the xml file (as the style name changes)
  • append random numbers between 0 and 1000 to the file name if the target file name is already in use (it may happen if we start from a legacy data dir, since the file xml and sld names might be out of synch)

Cheers

Andrea

–

==
Meet us at GEO Business 2014! in London! Visit http://goo.gl/fES3aK
for more information.

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

12

That is where we ended up (again) … so good that you are happy.

One thing you may be able to speak to is the suffix to prevent name clashes - it looks like this is only to untangle updating an old catalog. Is there any way we can confine that logic to just the upgrade process (and thus have an easier validation story to tell when renaming styles).

···

–

Jody Garnett

On Tue, Jun 3, 2014 at 3:21 AM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Sorry, chiming in on this one late. If we are talking about moving the SLD content inline then doesn’t that rule out storing styles in other languages (ie CSS)? This is actually something I am looking at doing sooner rather than later for a client. So if this gets moved inline I’ll need a way to specify that the content is stored externally.

In general I don’t really like the idea of storing the contents inline although that may just be because i am used to the current way of things. But i do feel that having the contents separated is more flexible. All in all I like the idea Andrea originally proposed and that is detecting the conflict and just using a suffix to ensure there are no file names clashes.

13

On Tue, Jun 3, 2014 at 1:56 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

That is where we ended up (again) ... so good that you are happy.

One thing you may be able to speak to is the suffix to prevent name
clashes - it looks like this is only to untangle updating an old catalog.
Is there any way we can confine that logic to *just* the upgrade process
(and thus have an easier validation story to tell when renaming styles).

Old is relative. In this case, any data dir in 2.x format is affected, and
the upgrade process is available only for the 1.7.x data dirs so... no can
do

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

14

Hi,
so I went ahead and applied the patch that renames files in synch and eventually tries up to other 100 names
in case the target sld file is busy.
Works fine, but there is something fishy… I totally expected to have to work against Resource when renaming
the files.

Instead, I’ve found that GeoServer persister works against files only and the xml file rename happens in there
against files. Also, the patch applies 1-1 to 2.5.x as well.

I guess I’m missing something… is the database backed catalog using its own version of the persister?
And if so, I guess the fix I’ve just made is not affecting it at all right?
How do we handle this “double maintenance”, also considering the module is at the moment unsupported?

Cheers
Andrea

···

On Thu, Jun 5, 2014 at 6:58 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

On Tue, Jun 3, 2014 at 1:56 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

That is where we ended up (again) … so good that you are happy.

One thing you may be able to speak to is the suffix to prevent name clashes - it looks like this is only to untangle updating an old catalog. Is there any way we can confine that logic to just the upgrade process (and thus have an easier validation story to tell when renaming styles).

Old is relative. In this case, any data dir in 2.x format is affected, and the upgrade process is available only for the 1.7.x data dirs so… no can do

Cheers

Andrea

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

15

As I understand it JDBCConfig takes care of saving the same objects as GeoServerPersister. Kevin’s patch for GeoServerPersister is still under review is it not?

···

Jody Garnett

On Tue, Jun 10, 2014 at 7:20 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
so I went ahead and applied the patch that renames files in synch and eventually tries up to other 100 names
in case the target sld file is busy.
Works fine, but there is something fishy… I totally expected to have to work against Resource when renaming
the files.

Instead, I’ve found that GeoServer persister works against files only and the xml file rename happens in there
against files. Also, the patch applies 1-1 to 2.5.x as well.

I guess I’m missing something… is the database backed catalog using its own version of the persister?
And if so, I guess the fix I’ve just made is not affecting it at all right?
How do we handle this “double maintenance”, also considering the module is at the moment unsupported?

Cheers

Andrea

On Thu, Jun 5, 2014 at 6:58 PM, Andrea Aime <andrea.aime@anonymised.com> wrote:

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

On Tue, Jun 3, 2014 at 1:56 AM, Jody Garnett <jody.garnett@anonymised.com> wrote:

That is where we ended up (again) … so good that you are happy.

One thing you may be able to speak to is the suffix to prevent name clashes - it looks like this is only to untangle updating an old catalog. Is there any way we can confine that logic to just the upgrade process (and thus have an easier validation story to tell when renaming styles).

Old is relative. In this case, any data dir in 2.x format is affected, and the upgrade process is available only for the 1.7.x data dirs so… no can do

Cheers

Andrea

–

==

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

==

Ing. Andrea Aime

@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

16

On Tue, Jun 10, 2014 at 11:50 AM, Jody Garnett <jody.garnett@anonymised.com>
wrote:

As I understand it JDBCConfig takes care of saving the same objects as
GeoServerPersister. Kevin's patch for GeoServerPersister is still under
review is it not?

Err uh? I thought the last pull request from him was merged days ago?
Ah, no, here it is: https://github.com/geoserver/geoserver/pull/591

Well, that explains it then...

Cheers
Andrea

--

GeoServer Professional Services from the experts! Visit
http://goo.gl/NWWaa2 for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------