[Geoserver-devel] Considering a port of the CSS module to Java

GeoServer GeoServer Developer
1

Hi,
at GeoSolutions we are considering a port of the CSS module to Java, under funding geared to “promote the CSS module with the direction of making it, in the future, a core module”.

Now, the work would not go as far as trying to push the CSS module into core, the idea would be to make it a supported extension, but in such a way that it’s possible, if the module gets enough users, to turn it into a core module without further modifications.

Given the recent discussion about usage of non Java languages in community, extension and core modules, the only way to ensure a possible core future for the CSS module is to rewrite it in Java

First off, I would like to ask if anybody else would be interested, now or in the future, to work on the Java version of the module.
Secondly, I would like to hear what would be the future of the Scala version of the module if there was a Java port of it: would we end up with a single module, or two competing modules that get evolved in separate ways?

One issue I see in the port is about limits in the documentation, as far as I understand the current documentation is lagging behind the actual module abilities, and in order to do a full port one would really have to dig into the Scala code to see what’s actually supported.

The other is technological, parsing CSS can be done in several ways.
The classic approach would be to use a JavaCC parser, an old technology that is however already used in GeoTools and GeoServer for CQL, ECQL, and WCS rangesubset parsing.
Some pre-existing CSS parser grammar could also be used to help in the work:
http://cssparser.sourceforge.net/

Of the many open source parser generators in Java (http://java-source.net/open-source/parser-generators) I guess the other famous one is Antlr, people speak well of it, but it carries around a rather large runtime dependency that JavaCC does not need.

Another approach would be to give up parser generations and go with with a parser combinator approach, which is the same as the Scala approach (supposedly, it might make the port easier).
The only library I’ve found supporting this approach in Java is JParsec,
http://jparsec.codehaus.org/Downloads
Looks nice and easy to use, however there are a few gotchas:

  • does not have a community
  • there is no version control, just the source of the 2.0.1 release
    I’ve mailed a bit with the author, he confirmed that the library is “done”, received no bug reports and thus made no more releases.
    Now, the source code builds, and after some manual tweaking of the build.xml file, there is a ton of tests too that pass properly.
    However… if there is any issue, we’ll be on our own…

Any other options?

And more in general, what’s the community feedback on this matter?
Hopefully David will chime in too.

Cheers
Andrea

–

==
GeoServer training in Milan, 6th & 7th June 2013! Visit http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

2

If i recall the conversation correctly it seemed like there was an informal consensus that only the core of GeoServer be kept to pure java. My opinion was that extensions or community modules could be implemented in other languages as long as those modules had a stable maintainer.

On one hand as a newbie to scala I can appreciate a pure java port as it would allow me to contribute more efficiently. On the other hand I think this sets a bad precedent. We have an active maintainer who is maintaining a module that has more than a handful of users. I certainly can’t speak for David in this case but if i were contributing to a project that did this to me I wouldn’t be a contributor for very long.

$0.02

···

On Thu, May 30, 2013 at 3:00 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
at GeoSolutions we are considering a port of the CSS module to Java, under funding geared to “promote the CSS module with the direction of making it, in the future, a core module”.

Now, the work would not go as far as trying to push the CSS module into core, the idea would be to make it a supported extension, but in such a way that it’s possible, if the module gets enough users, to turn it into a core module without further modifications.

Given the recent discussion about usage of non Java languages in community, extension and core modules, the only way to ensure a possible core future for the CSS module is to rewrite it in Java

First off, I would like to ask if anybody else would be interested, now or in the future, to work on the Java version of the module.
Secondly, I would like to hear what would be the future of the Scala version of the module if there was a Java port of it: would we end up with a single module, or two competing modules that get evolved in separate ways?

One issue I see in the port is about limits in the documentation, as far as I understand the current documentation is lagging behind the actual module abilities, and in order to do a full port one would really have to dig into the Scala code to see what’s actually supported.

The other is technological, parsing CSS can be done in several ways.
The classic approach would be to use a JavaCC parser, an old technology that is however already used in GeoTools and GeoServer for CQL, ECQL, and WCS rangesubset parsing.
Some pre-existing CSS parser grammar could also be used to help in the work:
http://cssparser.sourceforge.net/

Of the many open source parser generators in Java (http://java-source.net/open-source/parser-generators) I guess the other famous one is Antlr, people speak well of it, but it carries around a rather large runtime dependency that JavaCC does not need.

Another approach would be to give up parser generations and go with with a parser combinator approach, which is the same as the Scala approach (supposedly, it might make the port easier).
The only library I’ve found supporting this approach in Java is JParsec,
http://jparsec.codehaus.org/Downloads
Looks nice and easy to use, however there are a few gotchas:

  • does not have a community
  • there is no version control, just the source of the 2.0.1 release
    I’ve mailed a bit with the author, he confirmed that the library is “done”, received no bug reports and thus made no more releases.
    Now, the source code builds, and after some manual tweaking of the build.xml file, there is a ton of tests too that pass properly.
    However… if there is any issue, we’ll be on our own…

Any other options?

And more in general, what’s the community feedback on this matter?
Hopefully David will chime in too.

Cheers
Andrea

–

==
GeoServer training in Milan, 6th & 7th June 2013! Visit http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with <2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

3

On Thu, May 30, 2013 at 4:03 PM, Justin Deoliveira <jdeolive@anonymised.com>wrote:

If i recall the conversation correctly it seemed like there was an
informal consensus that only the core of GeoServer be kept to pure java. My
opinion was that extensions or community modules could be implemented in
other languages as long as those modules had a stable maintainer.

Yes, I've been explaining this in detail to the potential sponsor, but they
want to prepare the module for core. Citing their response to my
explanation that a Scala module can be made an extension, verbatim:

"Yes, think it should be rewritten in Java so most developers in the team
can potentially understand the code when it is an extension, and so that it
can potentially be promoted to core"

On one hand as a newbie to scala I can appreciate a pure java port as it
would allow me to contribute more efficiently. On the other hand I think
this sets a bad precedent. We have an active maintainer who is maintaining
a module that has more than a handful of users. I certainly can't speak for
David in this case but if i were contributing to a project that did this to
me I wouldn't be a contributor for very long.

Active maintainer is a bit of a stretch, as the module is in unsupported
land and as such, by definition, unmaintained.
It's up to David to decide whether to push the module to extension, so far
he did not feel like doing so.
I'm the last person that wants to push contributors out of the door, but if
we want to grab this funding opportunity and make CSS a supported module,
with a strong possibility for core inclusion (once in extension, the
blockers to core would be its Scala status and a check on the user base,
which seems to be already appreciating the module) later down the road,
what alternatives are there?
I'm fully open to suggestions (I mean it) :slight_smile:

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

4

On Thu, May 30, 2013 at 8:12 AM, Andrea Aime
<andrea.aime@anonymised.com>wrote:

On Thu, May 30, 2013 at 4:03 PM, Justin Deoliveira <jdeolive@anonymised.com>wrote:

If i recall the conversation correctly it seemed like there was an
informal consensus that only the core of GeoServer be kept to pure java. My
opinion was that extensions or community modules could be implemented in
other languages as long as those modules had a stable maintainer.

Yes, I've been explaining this in detail to the potential sponsor, but
they want to prepare the module for core. Citing their response to my
explanation that a Scala module can be made an extension, verbatim:

"Yes, think it should be rewritten in Java so most developers in the team
can potentially understand the code when it is an extension, and so that it
can potentially be promoted to core"

I see. This is just my opinion but this feels like one funding organization
asserting too much control at the developer community level.

On one hand as a newbie to scala I can appreciate a pure java port as it
would allow me to contribute more efficiently. On the other hand I think
this sets a bad precedent. We have an active maintainer who is maintaining
a module that has more than a handful of users. I certainly can't speak for
David in this case but if i were contributing to a project that did this to
me I wouldn't be a contributor for very long.

Active maintainer is a bit of a stretch, as the module is in unsupported
land and as such, by definition, unmaintained.

It seems to me that there are two types of community modules. Those that
are truly unmaintained. And those that are incubating in community in
preparation for promotion to extension. To me modules like css, script,
etc... seem to be the latter. Afaik css checks all the requirements
for promotion, and it's now just a matter of moving the module over.

It's up to David to decide whether to push the module to extension, so far
he did not feel like doing so.

Right. I guess i'll wait to hear from David on his official stance and
whether he wishes to push it to extension.

I'm the last person that wants to push contributors out of the door, but if

we want to grab this funding opportunity and make CSS a supported module,
with a strong possibility for core inclusion (once in extension, the
blockers to core would be its Scala status and a check on the user base,
which seems to be already appreciating the module) later down the road,
what alternatives are there?
I'm fully open to suggestions (I mean it) :slight_smile:

I guess i don't understand why it's a blocker for this organization not to
have css support in the core? Styling language has already been for the
most part abstracted out into an extension point when WMS 1.3 support was
added and we had to support initially SE. So to me it seems good css
integration is just a matter of copying over a plugin. Perhaps i am wrong
or missing something though.

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

5

Well, I’m open to having the CSS module as a supported extension. Previously I have been holding off because CSS is not standardized, but there are lots of extensions that are not and “map css” projects do not seem to be moving toward any standardization right now. However, given the previous discussion about non-Java languages I haven’t brought up the topic of promotion on the list. Aside from the language discussion, there’s also the question of whether the level of support I’m currently providing is sufficient to make it a supported extension (I am not prepared to be any more responsive about CSS issues than I have been.)

On the topic of creating a Java port, I’m not just uninterested in doing the work on my own time, I wouldn’t continue to contribute CSS work if a Java port replaced the Scala one currently used.

FWIW, the code that is actually stored in the GeoServer source tree for the CSS module is mostly Java (I made some error in transliterating CSSDemoPage.scala and had to revert it.) Does it affect fitness-for-core if the Scala code is all behind a managed Maven dependency?

···

On Thu, May 30, 2013 at 10:12 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, May 30, 2013 at 4:03 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:

Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with <2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

If i recall the conversation correctly it seemed like there was an informal consensus that only the core of GeoServer be kept to pure java. My opinion was that extensions or community modules could be implemented in other languages as long as those modules had a stable maintainer.

Yes, I’ve been explaining this in detail to the potential sponsor, but they want to prepare the module for core. Citing their response to my explanation that a Scala module can be made an extension, verbatim:

“Yes, think it should be rewritten in Java so most developers in the team can potentially understand the code when it is an extension, and so that it can potentially be promoted to core”

On one hand as a newbie to scala I can appreciate a pure java port as it would allow me to contribute more efficiently. On the other hand I think this sets a bad precedent. We have an active maintainer who is maintaining a module that has more than a handful of users. I certainly can’t speak for David in this case but if i were contributing to a project that did this to me I wouldn’t be a contributor for very long.

Active maintainer is a bit of a stretch, as the module is in unsupported land and as such, by definition, unmaintained.
It’s up to David to decide whether to push the module to extension, so far he did not feel like doing so.
I’m the last person that wants to push contributors out of the door, but if we want to grab this funding opportunity and make CSS a supported module, with a strong possibility for core inclusion (once in extension, the blockers to core would be its Scala status and a check on the user base, which seems to be already appreciating the module) later down the road, what alternatives are there?
I’m fully open to suggestions (I mean it) :slight_smile:

Cheers

Andrea

–

==
GeoServer training in Milan, 6th & 7th June 2013! Visit http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

6

On Thu, May 30, 2013 at 4:42 PM, Justin Deoliveira <jdeolive@anonymised.com>wrote:

I'm the last person that wants to push contributors out of the door, but
if we want to grab this funding opportunity and make CSS a supported
module, with a strong possibility for core inclusion (once in extension,
the blockers to core would be its Scala status and a check on the user
base, which seems to be already appreciating the module) later down the
road, what alternatives are there?

I'm fully open to suggestions (I mean it) :slight_smile:

I guess i don't understand why it's a blocker for this organization not to
have css support in the core? Styling language has already been for the
most part abstracted out into an extension point when WMS 1.3 support was
added and we had to support initially SE. So to me it seems good css
integration is just a matter of copying over a plugin. Perhaps i am wrong
or missing something though.

Extensions are somewhat weak, only one developer officially supporting
them, and there is the little hassle of installing them.
They are also not user covered as core code, because you have to install
them in order to use them, just see what happened
lately with the monitoring, xslt and inspire extensions, the packages were
not properly setup and it took time for people to notice.
I'm not saying that core module never have failures, but they normally have
more eyes on it.

Moreover, core module normally are not getting dropped along the way. Yes,
it might happen, but it's quite a bit more unlikely
than with a single person community/extension module.

And here lies the weakness of modules made on top with scripting languages,
that often they are single person businesses.
While I cannot speak for the sponsor, I believe they will invest in CSS
usage only if the CSS is there to stay, something that
is hardly guaranteed with a truly single module maintainer.

I have it in good authority that several people are not using the WPS
module because it's not on core. They look and ask
"why is it not on core, what's missing", and normally they don't like the
answer.
I mean, it's not strange that a portion of the user base is rather risk
adverse...

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

7

On Thu, May 30, 2013 at 5:00 PM, David Winslow <dwinslow@anonymised.com> wrote:

FWIW, the code that is actually stored in the GeoServer source tree for
the CSS module is mostly Java (I made some error in transliterating
CSSDemoPage.scala and had to revert it.) Does it affect fitness-for-core
if the Scala code is all behind a managed Maven dependency?

Yes, that does not change some of the reasons against having modules in
core that are made with scripting languages:
1) forced dependency on a large runtime that makes it hard to start
GeoServer without tweaking max permgen
2) dependency on an external library that is a one man job and that does
the 99% of the actual job, that is, if you need to
    do any improvement to it, you have to program it in Scala, and need to
be able to publish updates to the library in a timely
    manner (compare with the GeoServer dependency on GeoTools).
    I mean, if this was something generic like common-utils who cares if
it's a one man job, you can
    work on top and around of it, but with CSS processing being fully done
in Scala, there is no other option than getting into it.

Also, the CSS Scala code is based on GeoScript and thus on GeoTools no? So
I guess we'd also need multiple supported versions
of it, one per version of GeoTools used, and have updates over time as
GeoTools versions are released and used in GeoServer
releases.

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

8

Regarding the version tracking of geotools and geoscript.scala, i thought (and david can correct me here) that geocss could be used standalone. I believe Jared makes use of this in the groovy geoscript implementation without having to pull in all of geoscript.scala.

With that said, I would suggest the following course. With David’s consent let’s go through the motions and promote the css module to an extension.

In the meantime obviously anyone that wishes to work on a pure java port is free to do so, perhaps starting it as a community module. If it gets to the point where it is stable and can become a replacement we can evaluate pushing it into core or promoting it as a separate extension.

···

Thoughts?

On Thu, May 30, 2013 at 9:06 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

On Thu, May 30, 2013 at 5:00 PM, David Winslow <dwinslow@anonymised.com> wrote:

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

FWIW, the code that is actually stored in the GeoServer source tree for the CSS module is mostly Java (I made some error in transliterating CSSDemoPage.scala and had to revert it.) Does it affect fitness-for-core if the Scala code is all behind a managed Maven dependency?

Yes, that does not change some of the reasons against having modules in core that are made with scripting languages:

  1. forced dependency on a large runtime that makes it hard to start GeoServer without tweaking max permgen
  2. dependency on an external library that is a one man job and that does the 99% of the actual job, that is, if you need to
    do any improvement to it, you have to program it in Scala, and need to be able to publish updates to the library in a timely
    manner (compare with the GeoServer dependency on GeoTools).
    I mean, if this was something generic like common-utils who cares if it’s a one man job, you can
    work on top and around of it, but with CSS processing being fully done in Scala, there is no other option than getting into it.

Also, the CSS Scala code is based on GeoScript and thus on GeoTools no? So I guess we’d also need multiple supported versions
of it, one per version of GeoTools used, and have updates over time as GeoTools versions are released and used in GeoServer
releases.

Cheers

Andrea

–

==
GeoServer training in Milan, 6th & 7th June 2013! Visit http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

9

On Thu, May 30, 2013 at 10:24 PM, Justin Deoliveira <jdeolive@anonymised.com>wrote:

Regarding the version tracking of geotools and geoscript.scala, i thought
(and david can correct me here) that geocss could be used standalone. I
believe Jared makes use of this in the groovy geoscript implementation
without having to pull in all of geoscript.scala.

This is correct. The CSS module predates GeoScript Scala, and although I
have merged it in at the source level it does not have any dependency on
the rest of GeoScript. There is a small (one or two methods) Java
compatibility API which is what the Groovy version of GeoScript uses. I'll
also toss in that Scala is not really a "scripting language" in any
significant sense - the code is compiled to normal Java bytecode (albeit
sometimes with funny method names) and the normal classloader rules apply
in terms of unused classes not taking up PermGen space, etc. Scala classes
do tend to be a little more PermGen heavy than Java ones though, since some
Scala idioms are compiled into lots of small classes or methods, and I'm
not trying to discount the cost of having yet another collections and
parser library loaded.

With that said, I would suggest the following course. With David's consent
let's go through the motions and promote the css module to an extension.

In the meantime obviously anyone that wishes to work on a pure java port
is free to do so, perhaps starting it as a community module. If it gets to
the point where it is stable and can become a replacement we can evaluate
pushing it into core or promoting it as a separate extension.

Thoughts?

Essentially this is the situation we are in now, but with the CSS module
promoted to extension status, right? This is fine with me.

On Andrea's notes about extensions, I think that if a client wants to use a
module but can't because their policy is not to use extensions, that's not
a very good reason to bring the extension into core. Maybe we should be
investigating how to overcome this perception that extensions are not well
supported (they are endorsed by the project, right?) Or maybe a technical
solution would be to add some extension management facilities
(enabling/disabling, graphically installing, etc) in order to make it
easier to reduce the PermGen and other impacts of GeoServer's growing core.

--
David Winslow
OpenGeo - http://opengeo.org/

On Thu, May 30, 2013 at 9:06 AM, Andrea Aime
<andrea.aime@anonymised.com>wrote:

On Thu, May 30, 2013 at 5:00 PM, David Winslow <dwinslow@anonymised.com>wrote:

FWIW, the code that is actually stored in the GeoServer source tree for
the CSS module is mostly Java (I made some error in transliterating
CSSDemoPage.scala and had to revert it.) Does it affect fitness-for-core
if the Scala code is all behind a managed Maven dependency?

Yes, that does not change some of the reasons against having modules in
core that are made with scripting languages:
1) forced dependency on a large runtime that makes it hard to start
GeoServer without tweaking max permgen
2) dependency on an external library that is a one man job and that does
the 99% of the actual job, that is, if you need to
    do any improvement to it, you have to program it in Scala, and need
to be able to publish updates to the library in a timely
    manner (compare with the GeoServer dependency on GeoTools).
    I mean, if this was something generic like common-utils who cares if
it's a one man job, you can
    work on top and around of it, but with CSS processing being fully
done in Scala, there is no other option than getting into it.

Also, the CSS Scala code is based on GeoScript and thus on GeoTools no?
So I guess we'd also need multiple supported versions
of it, one per version of GeoTools used, and have updates over time as
GeoTools versions are released and used in GeoServer
releases.

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

10

On Fri, May 31, 2013 at 3:10 PM, David Winslow <dwinslow@anonymised.com> wrote:

On Andrea's notes about extensions, I think that if a client wants to use
a module but can't because their policy is not to use extensions, that's
not a very good reason to bring the extension into core. Maybe we should
be investigating how to overcome this perception that extensions are not
well supported (they are endorsed by the project, right?)

Given the rules for promoting a module to extension, I cannot really say
"they are endorsed by the project", three users, enough unit testing, one
maintainer, considere "stable" by the PSC:
http://docs.geoserver.org/stable/en/developer/policies/community-modules.html#promoting-a-community-module
Anyone can get a module in extension by following those rules, regardless
of whether the PSC thinks the module is important, useless, annoying, or
trivial, provided it cannot be argued that it's not stable.
If the maintainer drops, so goes the module.

My perception of community space is different than Justin's one, community
space is either nursery, dumpster or limbo, a fresh module is nursery for
sure, the others... well...
It also matches well how people can access the modules in community, only
by nightly builds, which means only people interested in "playing" with
them and enough technically skilled to use them (provided they are getting
in the community release at all).

Extension modules are tight and clean, but they are not in core because
they are either not popular enough or problematic in some way. And they are
still normally managed by a single dev (app-schema and to a lesser extent
WPS are the notable exceptions I believe). And I believe it's fine like
this, it is and should stay easy to get into extension, since only there
you start getting real usage by the user base beyond enthusiasts.
By they still are lights year away from a core module in terms of user base
usage and support. To give you a simple example, think the Oracle module.
Important? Well, I'd say very. Supported as well as PostGIS which is
available right in core? We had to squash 90% of their code base to avoid
Oracle drifting away, and despite it PostGIS still works better.
Now compare with something minor like "querylayer". It is still an
extension module, it is still supported. Comparable to a core module? Nah.
Comparable to a module like WPS or Oracle in usage and support term? Not a
chance either.
Just being in extension is a guarantee, docs, tests, maintainer, but does
not come even close to being in core, there is actually a lot of
variability among the extension modules.

Or maybe a technical solution would be to add some extension management
facilities (enabling/disabling, graphically installing, etc) in order to
make it easier to reduce the PermGen and other impacts of GeoServer's
growing core.

Yes, this would be great :slight_smile:
I'm afraid we'd have to give up the simple spring oriented pluggability and
start looking into something like OSGI to get there. Tall order...

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

11

Going back to the top of the thread, taking it from a purely technical aspect.

The CSS module actually incurs a lot of complexity trying to translate the CSS cascading properties model into SLD (where the painter’s model reigns.) There’s some complicated filter analysis stuff going on 1 and some fancy graph stuff to avoid enumerating all possible combinations of CSS rules 2. As functional techniques generally shine for algorithmic code I would expect these to be more problematic to convert to Java than the parser itself. However, it might be interesting to think about modifying the renderer in GeoTools to be able to work with CSS styles directly and avoid requiring this conversion in the first place. Actually from speaking with the Carto guys it sounds like they found a middle ground that made style conversion simpler without requiring a ton of changes to the renderer by adding an option to switch to “first match” instead of “all match” mode.

On the topic of parsing, I expect that a pre-made CSS parser will be insufficient as I am pretty sure that embedded CQL expressions can sometimes deviate from normal CSS syntax (of course a pre-existing one could be forked and modified.) I think parboiled1 might be a better option than JParsec (just a hunch, I have little experience with either.)

And as a general strategy note, if I were working on a Java port, I would probably just start from the existing code and port one class at a time. The SBT project is already configured for mixed Java/Scala sources so this would (theoretically) allow the port to be completed without ever giving up the unit test suite. Maven can be set up that way as well, but I’d probably wait until the project was pure Java to make the build tool transition.

···

On Thu, May 30, 2013 at 5:00 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi,
at GeoSolutions we are considering a port of the CSS module to Java, under funding geared to “promote the CSS module with the direction of making it, in the future, a core module”.

Now, the work would not go as far as trying to push the CSS module into core, the idea would be to make it a supported extension, but in such a way that it’s possible, if the module gets enough users, to turn it into a core module without further modifications.

Given the recent discussion about usage of non Java languages in community, extension and core modules, the only way to ensure a possible core future for the CSS module is to rewrite it in Java

First off, I would like to ask if anybody else would be interested, now or in the future, to work on the Java version of the module.
Secondly, I would like to hear what would be the future of the Scala version of the module if there was a Java port of it: would we end up with a single module, or two competing modules that get evolved in separate ways?

One issue I see in the port is about limits in the documentation, as far as I understand the current documentation is lagging behind the actual module abilities, and in order to do a full port one would really have to dig into the Scala code to see what’s actually supported.

The other is technological, parsing CSS can be done in several ways.
The classic approach would be to use a JavaCC parser, an old technology that is however already used in GeoTools and GeoServer for CQL, ECQL, and WCS rangesubset parsing.
Some pre-existing CSS parser grammar could also be used to help in the work:
http://cssparser.sourceforge.net/

Of the many open source parser generators in Java (http://java-source.net/open-source/parser-generators) I guess the other famous one is Antlr, people speak well of it, but it carries around a rather large runtime dependency that JavaCC does not need.

Another approach would be to give up parser generations and go with with a parser combinator approach, which is the same as the Scala approach (supposedly, it might make the port easier).
The only library I’ve found supporting this approach in Java is JParsec,
http://jparsec.codehaus.org/Downloads
Looks nice and easy to use, however there are a few gotchas:

  • does not have a community
  • there is no version control, just the source of the 2.0.1 release
    I’ve mailed a bit with the author, he confirmed that the library is “done”, received no bug reports and thus made no more releases.
    Now, the source code builds, and after some manual tweaking of the build.xml file, there is a ton of tests too that pass properly.
    However… if there is any issue, we’ll be on our own…

Any other options?

And more in general, what’s the community feedback on this matter?
Hopefully David will chime in too.

Cheers
Andrea

–

==
GeoServer training in Milan, 6th & 7th June 2013! Visit http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with <2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

12

On Fri, May 31, 2013 at 3:31 PM, David Winslow <dwinslow@anonymised.com> wrote:

Going back to the top of the thread, taking it from a purely technical
aspect.

The CSS module actually incurs a lot of complexity trying to translate the
CSS cascading properties model into SLD (where the painter's model reigns.)
There's some complicated filter analysis stuff going on [1] and some fancy
graph stuff to avoid enumerating all possible combinations of CSS rules
[2]. As functional techniques generally shine for algorithmic code I would
expect these to be more problematic to convert to Java than the parser
itself. However, it might be interesting to think about modifying the
renderer in GeoTools to be able to work with CSS styles directly and avoid
requiring this conversion in the first place. Actually from speaking with
the Carto guys it sounds like they found a middle ground that made style
conversion simpler without requiring a ton of changes to the renderer by
adding an option to switch to "first match" instead of "all match" mode.

Interesting concept, can you elaborate a bit?
Without having to rewrite the Style objects in GeoTools, what would be the
closest parallel?
Maybe setup the FeatureTypeStyle in such a way that symbolizers of the same
kind override each other instead of being painted in sequence?
E.g., if I have three rules with PointSymbolizers, for each feature we'd
first first find the matching rules, and then have the various symbolizers
override each other in turn, until we're left with a single one that is
used to actually paint the feature?

Mind, today the code already paints all the matching rules, not just the
first one, but it paints each rule in turn, instead of overriding the
directives in the symbolizers.
The relevant code snippet being:

        boolean doElse = true;
        final int length = ruleList.length;
        for (int t = 0; t < length; t++) {
            r = ruleList[t];
            filter = r.getFilter();

            if (filter == null || filter.evaluate(rf.content)) {
                doElse = false;
                processSymbolizers(graphics, rf, r.symbolizers(),
scaleRange, at, destinationCrs, layerId);
            }
        }

        if (doElse) {
            final int elseLength = elseRuleList.length;
            for (int tt = 0; tt < elseLength; tt++) {
                r = elseRuleList[tt];

                processSymbolizers(graphics, rf, r.symbolizers(),
scaleRange,
                        at, destinationCrs, layerId);

            }
        }

On the topic of parsing, I expect that a pre-made CSS parser will be
insufficient as I am pretty sure that embedded CQL expressions can
sometimes deviate from normal CSS syntax (of course a pre-existing one
could be forked and modified.) I think parboiled[1] might be a better
option than JParsec (just a hunch, I have little experience with either.)

Yep, I just wanted to use it as template/starting point, not as-is

And as a general strategy note, if I were working on a Java port, I would
probably just start from the existing code and port one class at a time.
The SBT project is already configured for mixed Java/Scala sources so this
would (theoretically) allow the port to be completed without ever giving up
the unit test suite. Maven can be set up that way as well, but I'd
probably wait until the project was pure Java to make the build tool
transition.

Right, I believe that makes a lot of sense if you are familiar with both
Java and Scala... but I'm not, only read a bit about Scala, and I would
have to read a bit more to do the port, the geocss code does not appear to
have comments and some parts of it have no meaning to my untrained eye.
Unfortunately at work I don't need Scala, and during the weekends I feel
it's my duty as a PSC/PMC member of GeoTools and GeoServer to support
people making pull requests or providing patches, make bug fixes during
beta periods and the like. That unfortunately leaves no space for
recreational coding anymore....

[3]: https://github.com/sirthias/parboiled/wiki

Nice one, thanks! Will surely keep it in mind

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------

13

On Fri, May 31, 2013 at 3:27 PM, Andrea Aime
<andrea.aime@anonymised.com>wrote:

Just being in extension is a guarantee, docs, tests, maintainer, but does
not come even close to being in core, there is actually a lot of
variability among the extension modules.

By the way, about user base usage of the modules, there are a bit of hard
data in the weekly download counts for GeoServer:

http://sourceforge.net/projects/geoserver/files/GeoServer/2.3.2/
http://sourceforge.net/projects/geoserver/files/GeoServer%20Extensions/2.3.2/

Right now the most downloaded extension is the gdal one, with 88 weekly
downloads.
By comparison, the most downloaded GeoServer is the exe installer, with
1164 download... and this
is not even a fair comparison, as extensions are the same for all versions,
so if we sum up
exe, dmg, bin and war it's 2643 downloads.
It means that the best extension gets roughly 30 times less exposure to end
users than core modules.

Now, I'm aware that not all modules in core are used the same way (e.g., I
believe WCS is not that popular,
hopefully WCS 2.0 lighter weight KVP syntax will change that), but still,
the difference in download
numbers is _large_

Cheers
Andrea

--

GeoServer training in Milan, 6th & 7th June 2013! Visit
http://geoserver.geo-solutions.it for more information.

Ing. Andrea Aime
@geowolf
Technical Lead

GeoSolutions S.A.S.
Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy
phone: +39 0584 962313
fax: +39 0584 1660272
mob: +39 339 8844549

http://www.geo-solutions.it
http://twitter.com/geosolutions_it

-------------------------------------------------------