[Geoserver-devel] GeoServer on a diet: loose 6-7MB in a day

GeoServer GeoServer Developer
1

Hi all,
welcome to Andrea's extreme diet program! :slight_smile:

Following yesterday's mail I've tried to get rid of the largest
dependencies and came up with a 6-7MB reduction without
removing the data directory.

Here is the list of removed offenders.

Getting rid of Xalan
-------------------------------

Removed dependency from main, from wms, and fixed the wms
capabilities code that depended on it more based on laziness than necessity.
Tested capabilities and SVG generation and usage (with burg style)
in both jdk 1.5 and jdk 1.6, it appears nothing broke.

2MB less, down to 51MB with current release directory

Getting rid of FOP
----------------------------

Apparently this is needed only to generate PDF out of SVG, which
we don't do.
Again, tried SVG usage and generation as above, all seemed to work
fine.

Removing this one got rid of other jars as well, result,
another 2MB removed. Down to 49MB.

Getting rid of BouncyCastle
----------------------------------------

These jars are iText dependencies, I guess they are used to generated
encrypted PDF but we don't have that atm.
Excluded them, tested PDF output, works fine too.

Down to 48MB

Using jpeg instead of PNG for the world image layer
--------------------------------------------------------------------------

This one was easy. Down to 46MB ("ls -lh" unit rounding is tricking
us into thinking 2MB less, that's not the case)

EditArea removal from web/core
----------------------------------------------

Confirmed nothing is using it.
Down only 100KB more, but hey, it was dead stuff anyways

ant-optional
-----------------

Not sure about this one, it is no more in the .war for me...
As said yesterday, not sure why it was in he 2.1.0 war to
start with?

Summary
--------------------------------------------------

I have the above patches ready for commit in a local GIT branch.
The pass a full build and interactive testing with jdk 1.50_22
and jdk 1.6.0_24.
I would suggest to commit them right away on trunk, have
developers give it a kick, and then ask on the user list for people
that heavily use SVG and PDF, telling them to please try out the
slimmed down version of GeoServer.
If nothing bad pops up we can backport on 2.1.x

Opinions?

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

2

Great work, thanks for doing that Andrea. Now I know who to go to when I need to drop a few pounds :slight_smile:

I agree letting the changes settle a bit on trunk before a backport makes sense.

On Thu, Jun 2, 2011 at 8:43 AM, Andrea Aime <andrea.aime@anonymised.com> wrote:

Hi all,
welcome to Andrea’s extreme diet program! :slight_smile:

Following yesterday’s mail I’ve tried to get rid of the largest
dependencies and came up with a 6-7MB reduction without
removing the data directory.

Here is the list of removed offenders.

Getting rid of Xalan

Removed dependency from main, from wms, and fixed the wms
capabilities code that depended on it more based on laziness than necessity.
Tested capabilities and SVG generation and usage (with burg style)
in both jdk 1.5 and jdk 1.6, it appears nothing broke.

2MB less, down to 51MB with current release directory

Getting rid of FOP

Apparently this is needed only to generate PDF out of SVG, which
we don’t do.
Again, tried SVG usage and generation as above, all seemed to work
fine.

Removing this one got rid of other jars as well, result,
another 2MB removed. Down to 49MB.

Getting rid of BouncyCastle

These jars are iText dependencies, I guess they are used to generated
encrypted PDF but we don’t have that atm.
Excluded them, tested PDF output, works fine too.

Down to 48MB

Using jpeg instead of PNG for the world image layer

This one was easy. Down to 46MB (“ls -lh” unit rounding is tricking
us into thinking 2MB less, that’s not the case)

EditArea removal from web/core

Confirmed nothing is using it.
Down only 100KB more, but hey, it was dead stuff anyways

ant-optional

Not sure about this one, it is no more in the .war for me…
As said yesterday, not sure why it was in he 2.1.0 war to
start with?

Summary

I have the above patches ready for commit in a local GIT branch.
The pass a full build and interactive testing with jdk 1.50_22
and jdk 1.6.0_24.
I would suggest to commit them right away on trunk, have
developers give it a kick, and then ask on the user list for people
that heavily use SVG and PDF, telling them to please try out the
slimmed down version of GeoServer.
If nothing bad pops up we can backport on 2.1.x

Opinions?

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

Simplify data backup and recovery for your virtual environment with vRanger.
Installation’s a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It’s vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

3

On Thu, 2011-06-02 at 08:58 -0600, Justin Deoliveira wrote:

Great work, thanks for doing that Andrea. Now I know who to go to when
I need to drop a few pounds :slight_smile:

I agree letting the changes settle a bit on trunk before a backport
makes sense.

++
thanks Andrea.

On Thu, Jun 2, 2011 at 8:43 AM, Andrea Aime
<andrea.aime@anonymised.com> wrote:
        Hi all,
        welcome to Andrea's extreme diet program! :slight_smile:
        
        Following yesterday's mail I've tried to get rid of the
        largest
        dependencies and came up with a 6-7MB reduction without
        removing the data directory.
        
        Here is the list of removed offenders.
        
        Getting rid of Xalan
        -------------------------------
        
        Removed dependency from main, from wms, and fixed the wms
        capabilities code that depended on it more based on laziness
        than necessity.
        Tested capabilities and SVG generation and usage (with burg
        style)
        in both jdk 1.5 and jdk 1.6, it appears nothing broke.
        
        2MB less, down to 51MB with current release directory
        
        Getting rid of FOP
        ----------------------------
        
        Apparently this is needed only to generate PDF out of SVG,
        which
        we don't do.
        Again, tried SVG usage and generation as above, all seemed to
        work
        fine.
        
        Removing this one got rid of other jars as well, result,
        another 2MB removed. Down to 49MB.
        
        Getting rid of BouncyCastle
        ----------------------------------------
        
        These jars are iText dependencies, I guess they are used to
        generated
        encrypted PDF but we don't have that atm.
        Excluded them, tested PDF output, works fine too.
        
        Down to 48MB
        
        Using jpeg instead of PNG for the world image layer
        --------------------------------------------------------------------------
        
        This one was easy. Down to 46MB ("ls -lh" unit rounding is
        tricking
        us into thinking 2MB less, that's not the case)
        
        EditArea removal from web/core
        ----------------------------------------------
        
        Confirmed nothing is using it.
        Down only 100KB more, but hey, it was dead stuff anyways
        
        ant-optional
        -----------------
        
        Not sure about this one, it is no more in the .war for me...
        As said yesterday, not sure why it was in he 2.1.0 war to
        start with?
        
        Summary
        --------------------------------------------------
        
        I have the above patches ready for commit in a local GIT
        branch.
        The pass a full build and interactive testing with jdk 1.50_22
        and jdk 1.6.0_24.
        I would suggest to commit them right away on trunk, have
        developers give it a kick, and then ask on the user list for
        people
        that heavily use SVG and PDF, telling them to please try out
        the
        slimmed down version of GeoServer.
        If nothing bad pops up we can backport on 2.1.x
        
        Opinions?
        
        Cheers
        Andrea
        
        --
        -------------------------------------------------------
        Ing. Andrea Aime
        GeoSolutions S.A.S.
        Tech lead
        
        Via Poggio alle Viti 1187
        55054 Massarosa (LU)
        Italy
        
        phone: +39 0584 962313
        fax: +39 0584 962313
        
        http://www.geo-solutions.it
        http://geo-solutions.blogspot.com/
        http://www.youtube.com/user/GeoSolutionsIT
        http://www.linkedin.com/in/andreaaime
        http://twitter.com/geowolf
        
        -------------------------------------------------------
        
        ------------------------------------------------------------------------------
        Simplify data backup and recovery for your virtual environment
        with vRanger.
        Installation's a snap, and flexible recovery options mean your
        data is safe,
        secure and there when you need it. Data protection magic?
        Nope - It's vRanger. Get your free trial download today.
        http://p.sf.net/sfu/quest-sfdev2dev
        _______________________________________________
        Geoserver-devel mailing list
        Geoserver-devel@lists.sourceforge.net
        https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev
_______________________________________________ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel

--
Gabriel Roldan
groldan@anonymised.com
Expert service straight from the developers

4

On Thu, Jun 2, 2011 at 4:43 PM, Andrea Aime
<andrea.aime@anonymised.com> wrote:

Summary
--------------------------------------------------

I have the above patches ready for commit in a local GIT branch.
The pass a full build and interactive testing with jdk 1.50_22
and jdk 1.6.0_24.
I would suggest to commit them right away on trunk, have
developers give it a kick, and then ask on the user list for people
that heavily use SVG and PDF, telling them to please try out the
slimmed down version of GeoServer.
If nothing bad pops up we can backport on 2.1.x

Btw, I forgot a bit about extensions using xalan, I had to fix a few:
- app-schema tests module depend on xalan being in the classpath for
  some transformations/xpath expressions to work. Among the patches
  I've thus added a xalan test dependency to them
- the excel module used a Xalan class to check if a char is a valid
  xml one (in order to write excel new xml format I believe).
  Replaced it with the equivalent class contained in Xerces
- wps GUI uses Xalan TreeWalker to write back into a content handler
  a w3c Document. Xalan 2.7.1 factored out that class in a small
  "serializer" jar which I've added as a dependency (though I'd really
  like to find a replacement and kill that dependency too)

Shane, Ben, any reason why this changes may cause issues?

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

5

Nope, XmlChar should work just fine for the Excel module. Thanks!

On Thu, Jun 2, 2011 at 7:40 AM, Andrea Aime <andrea.aime@anonymised.com68…> wrote:

On Thu, Jun 2, 2011 at 4:43 PM, Andrea Aime
<andrea.aime@anonymised.com> wrote:

Summary

I have the above patches ready for commit in a local GIT branch.
The pass a full build and interactive testing with jdk 1.50_22
and jdk 1.6.0_24.
I would suggest to commit them right away on trunk, have
developers give it a kick, and then ask on the user list for people
that heavily use SVG and PDF, telling them to please try out the
slimmed down version of GeoServer.
If nothing bad pops up we can backport on 2.1.x

Btw, I forgot a bit about extensions using xalan, I had to fix a few:

  • app-schema tests module depend on xalan being in the classpath for
    some transformations/xpath expressions to work. Among the patches
    I’ve thus added a xalan test dependency to them
  • the excel module used a Xalan class to check if a char is a valid
    xml one (in order to write excel new xml format I believe).
    Replaced it with the equivalent class contained in Xerces
  • wps GUI uses Xalan TreeWalker to write back into a content handler
    a w3c Document. Xalan 2.7.1 factored out that class in a small
    “serializer” jar which I’ve added as a dependency (though I’d really
    like to find a replacement and kill that dependency too)

Shane, Ben, any reason why this changes may cause issues?

Cheers
Andrea

–

Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

–
Shane StClair
Software Engineer
Axiom Consulting & Design
523 W 8th Ave
Suite 104
Anchorage, AK 99501
http://www.axiomalaska.com

6

Andrea,

the main change is that we'll need to add Xalan to the app-schema plugin for Java 5 deployments. Victor's performance improvement fix for app-schema WFS uses XSLT to calculate numberOfFeatures to avoid an expensive call to FeatureCollection.size(); I'm assuming this needs Xalan. For this to work at runtime, users will need to deploy Xalan.

Rini will fix webservice-test, which she maintains.

Kind regards,
Ben

On 02/06/11 23:40, Andrea Aime wrote:

On Thu, Jun 2, 2011 at 4:43 PM, Andrea Aime
<andrea.aime@anonymised.com> wrote:

Summary
--------------------------------------------------

I have the above patches ready for commit in a local GIT branch.
The pass a full build and interactive testing with jdk 1.50_22
and jdk 1.6.0_24.
I would suggest to commit them right away on trunk, have
developers give it a kick, and then ask on the user list for people
that heavily use SVG and PDF, telling them to please try out the
slimmed down version of GeoServer.
If nothing bad pops up we can backport on 2.1.x

Btw, I forgot a bit about extensions using xalan, I had to fix a few:
- app-schema tests module depend on xalan being in the classpath for
   some transformations/xpath expressions to work. Among the patches
   I've thus added a xalan test dependency to them
- the excel module used a Xalan class to check if a char is a valid
   xml one (in order to write excel new xml format I believe).
   Replaced it with the equivalent class contained in Xerces
- wps GUI uses Xalan TreeWalker to write back into a content handler
   a w3c Document. Xalan 2.7.1 factored out that class in a small
   "serializer" jar which I've added as a dependency (though I'd really
   like to find a replacement and kill that dependency too)

Shane, Ben, any reason why this changes may cause issues?

Cheers
Andrea

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre

7

On Tue, Jun 7, 2011 at 3:52 AM, Ben Caradoc-Davies
<Ben.Caradoc-Davies@anonymised.com> wrote:

Andrea,

the main change is that we'll need to add Xalan to the app-schema plugin
for Java 5 deployments. Victor's performance improvement fix for
app-schema WFS uses XSLT to calculate numberOfFeatures to avoid an
expensive call to FeatureCollection.size(); I'm assuming this needs
Xalan. For this to work at runtime, users will need to deploy Xalan.

Rini will fix webservice-test, which she maintains.

Yep, I did not look deeply into replacing Xalan with the JDK transformation
abilities, yet it's definitely possible to run xslt/xpath wit the jdk 5.
The attached zip shows how to do each of them without relying on Xalan
or JDK 6.

I guess the reason why things do not work you your case is that they might
be using some xpath/xslt v2 feature?

Cheers
Andrea

--
-------------------------------------------------------
Ing. Andrea Aime
GeoSolutions S.A.S.
Tech lead

Via Poggio alle Viti 1187
55054 Massarosa (LU)
Italy

phone: +39 0584 962313
fax: +39 0584 962313

http://www.geo-solutions.it
http://geo-solutions.blogspot.com/
http://www.youtube.com/user/GeoSolutionsIT
http://www.linkedin.com/in/andreaaime
http://twitter.com/geowolf

-------------------------------------------------------

(attachments)

jaxp.zip (5.42 KB)

8

On 07/06/11 18:10, Andrea Aime wrote:

Yep, I did not look deeply into replacing Xalan with the JDK transformation
abilities, yet it's definitely possible to run xslt/xpath wit the jdk 5.
The attached zip shows how to do each of them without relying on Xalan
or JDK 6.

Thanks, Andrea. I think Victor's implementation already does something similar, and so picks up the JAXP implementation.

I guess the reason why things do not work you your case is that they might
be using some xpath/xslt v2 feature?

The XSLT in question contains 'xsl:stylesheet version="2.0"' and I guess 'Syntax error in '@*|text()|comment()|processing-instruction'.' means JAXP cannot process it. I'm an XSLT n00b so I can't be sure. Adding Xalan to our deployments will surely be the cheapest solution.

Kind regards,

--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre