Hi,
as indicated in the other thread, I looked at the slowest tests in a sequential build (so that they do
not interfere with each other) to see if there is anything worth paying attention to.
I’ve started with this stackoverflow suggestion on how to collect slowest tests:
https://stackoverflow.com/questions/4856030/sorting-according-to-fields-numerical-value-in-bash
However, in GeoServer the test running as first in a test that setups the app context is always going
to rack up most of the time spent, so I’ve modified it as follows to count class time instead of method time, and also to have an
indication of the module hosting the test:
grep -H “<testsuite” find . -iname "TEST-*.xml" | sed ‘s/./(.)/target.:<testsuite .* name=“(.)" time=“([^”])” .*/\3\t\2\t\1/’ | sort -nr | less
The tests taking more than 4 seconds follow:
77.882 org.geoserver.geofence.internal.InternalUserResolverTest extension/geofence-server
44.235 org.geoserver.security.web.jdbc.group.JDBCConfirmRemovalGroupPanelTest web/security/jdbc
39.695 org.geoserver.gwc.GWCIntegrationTest gwc
22.439 org.geoserver.netcdf.NetCDFCRSOverridingAuthorityFactoryTest extension/netcdf
15.878 org.geoserver.security.xml.XMLUserGroupServiceTest security/security-tests
14.182 org.geoserver.web.demo.PreviewLayerProviderMixedModeTest web/demo
13.302 org.geoserver.security.web.data.ConfirmRemovalDataAccessRulePanelTest web/security/core
12.988 org.geoserver.wcs2_0.xml.GetCoverageTest wcs2_0
10.383 org.geoserver.importer.rest.ImporterIntegrationTest extension/importer/rest
10.069 org.geoserver.rest.catalog.StructuredCoverageStoresTest restconfig
10.006 org.geoserver.security.auth.GuavaAuthenticationCacheTest security/security-tests
9.859 org.geoserver.test.BBox3DTest extension/app-schema/app-schema-postgis-test
8.733 org.geoserver.test.BBox3DTest extension/app-schema/app-schema-test
8.404 org.geoserver.wcs2_0.WCSNetCDFMosaicTest extension/netcdf-out
7.527 org.geoserver.security.auth.LRUAuthenticationCacheTest security/security-tests
7.164 org.geoserver.importer.ImporterDataTest extension/importer/core
6.761 org.geoserver.inspire.wcs.WCSExtendedCapabilitiesTest extension/inspire
6.424 org.geoserver.test.FeatureChainingWfsTest extension/app-schema/app-schema-postgis-test
6.153 org.geoserver.geofence.internal.InternalServicesTest extension/geofence-server
5.953 org.geoserver.kml.GeoSearchKMLTest kml
5.949 org.geoserver.security.jdbc.PostGisUserGroupServiceTest security/jdbc
5.687 org.geoserver.security.impl.MemoryUserDetailsServiceTest security/security-tests
5.637 org.geoserver.flow.controller.PriorityFlowControllerTest extension/control-flow
5.569 org.geoserver.geofence.AccessManagerConfigTest extension/geofence
5.482 org.geoserver.security.impl.MemoryUserGroupServiceTest security/security-tests
5.478 org.geoserver.test.FeatureChainingWfsTest extension/app-schema/app-schema-test
5.322 org.geoserver.wfs.xslt.GetFeatureXSLTTest extension/xslt
5.316 org.geoserver.platform.resource.FileSystemResourceTheoryTest platform
5.178 org.geoserver.catalog.CoverageViewTest main
5.177 org.geoserver.security.jdbc.H2UserGroupServiceTest security/jdbc
5.027 org.geoserver.wfs.response.ExcelOutputFormatTest extension/excel
5.004 org.geoserver.monitor.PipeliningTaskQueueTest extension/monitor/core
4.939 org.geoserver.security.BruteForceAttackTest security/security-tests
4.902 org.geoserver.catalog.CoverageViewReaderTest extension/grib
4.794 org.geoserver.catalog.impl.CatalogBuilderIntTest main
4.711 org.geoserver.wps.DescribeProcessTest extension/wps/wps-core
4.653 org.geoserver.gwc.web.layer.CachedLayerProviderTest web/gwc
4.399 org.geoserver.test.SampleDataAccessWfsTest extension/app-schema/sample-data-access-test
4.378 org.geoserver.importer.web.ImportTaskTableTest extension/importer/web
4.338 org.geoserver.wfs.response.DXFOutputFormatTest extension/dxf/core
4.267 org.geoserver.test.PagingTest extension/app-schema/app-schema-postgis-test
4.244 org.geoserver.rest.catalog.YsldStyleControllerTest extension/ysld
4.154 org.geoserver.gwc.web.GWCServiceLinksTest web/gwc
4.122 org.geoserver.wcs.DescribeCoverageTest wcs1_1
4.017 org.geoserver.wps.ExecuteTest extension/wps/wps-core
A few notes on the results:
- The first integration test run in any module is going to take a significant hit, it’s the one doing the bulk of the classloading, and will typically take a few seconds while the others take less than a second, so not all those tests would take 4+ seconds if they weren’t run first
- Some long tests have already been moved out of the way of the main->wfs->wms->gwc critical path, for example, the security ones are slow, but were already moved to their own dedicated security-tests module to let them run in parallel
- Tests taking more than 30 seconds are probably best looked into even if not in the critical path
Well, I’m going to have a look at the GWC one, if you want to jump in too, welcomed to, just let
us know what you’re looking at :-p
Cheers
Andrea
== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
Some follow up:
77.882 org.geoserver.geofence.internal.InternalUserResolverTest extension/geofence-server
Totally weird, if I run the test alone from IDE it takes “only” 7 seconds, running it in a maven build
of the module in question, 1 second, heck, the complete module takes around 22 seconds.
Not sure what’s going on here…
44.235 org.geoserver.security.web.jdbc.group.JDBCConfirmRemovalGroupPanelTest web/security/jdbc
39.695 org.geoserver.gwc.GWCIntegrationTest gwc
Managed to make this run without rerunning the full setup, I’ll follow up with a PR
22.439 org.geoserver.netcdf.NetCDFCRSOverridingAuthorityFactoryTest extension/netcdf
Another weirdo, it’s quick when I run it alone, but it’s one of those integration tests with just one test
method, was easy to merge with another test to avoid the double setup.
There are a lot of these around, some gain could be obtained by merging small tests into larger ones.
15.878 org.geoserver.security.xml.XMLUserGroupServiceTest security/security-tests
14.182 org.geoserver.web.demo.PreviewLayerProviderMixedModeTest web/demo
13.302 org.geoserver.security.web.data.ConfirmRemovalDataAccessRulePanelTest web/security/core
12.988 org.geoserver.wcs2_0.xml.GetCoverageTest wcs2_0
This one was making a lot of network requests due to XSD trying to search for an XML schema that
does not exist, being referenced from the XML request files. Cleaned it up.
There could be other tests trying to access the network, that might be a visible cause of slowdown for some of
them at least. For the ones that go though XSD schema I’ve added a debug statement that can be activated
with a sysvar and got this out of the GeoServer build (after the WCS cleanups):
···
== GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 http://www.geo-solutions.it http://twitter.com/geosolutions_it ------------------------------------------------------- Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
Thanks for caring and pushing forward with this Andrea
Some follow up:
77.882 org.geoserver.geofence.internal.InternalUserResolverTest extension/geofence-server
Totally weird, if I run the test alone from IDE it takes “only” 7 seconds, running it in a maven build
of the module in question, 1 second, heck, the complete module takes around 22 seconds.
Not sure what’s going on here…
44.235 org.geoserver.security.web.jdbc.group.JDBCConfirmRemovalGroupPanelTest web/security/jdbc
39.695 org.geoserver.gwc.GWCIntegrationTest gwc
Managed to make this run without rerunning the full setup, I’ll follow up with a PR
Nice !
22.439 org.geoserver.netcdf.NetCDFCRSOverridingAuthorityFactoryTest extension/netcdf
Another weirdo, it’s quick when I run it alone, but it’s one of those integration tests with just one test
method, was easy to merge with another test to avoid the double setup.
In App-Schema tests we have plenty of those which is a problem indeed, is not easy to create a very complex use case that coves everything and still readable.
There are a lot of these around, some gain could be obtained by merging small tests into larger ones.
15.878 org.geoserver.security.xml.XMLUserGroupServiceTest security/security-tests
14.182 org.geoserver.web.demo.PreviewLayerProviderMixedModeTest web/demo
13.302 org.geoserver.security.web.data.ConfirmRemovalDataAccessRulePanelTest web/security/core
12.988 org.geoserver.wcs2_0.xml.GetCoverageTest wcs2_0
This one was making a lot of network requests due to XSD trying to search for an XML schema that
does not exist, being referenced from the XML request files. Cleaned it up.
There could be other tests trying to access the network, that might be a visible cause of slowdown for some of
them at least. For the ones that go though XSD schema I’ve added a debug statement that can be activated
with a sysvar and got this out of the GeoServer build (after the WCS cleanups):
Cool.
grep opening -B 1 /tmp/gs-remote.log
–
Running org.geoserver.csw.GetRecordsTest
WARNING, opening a connection to http://schemas.opengis.net/csw/2.0.2/profiles/ebrim/1.0/csw-ebrim.xsd
and this from the GeoTools ones:
grep opening -B 1 /tmp/gt.log
85771 [INFO] Running org.geotools.xsd.impl.HTTPURIHandlerTest
WARNING, opening a connection to http://example.com
WARNING, opening a connection to http://example.com
WARNING, opening a connection to http://example.com
WARNING, opening a connection to http://example.com
WARNING, opening a connection to https://example.com
WARNING, opening a connection to http://example.com
–
275066 [INFO] Running org.geotools.se.v1_1.SEExampleTest
WARNING, opening a connection to http://schemas.opengis.net/sld/1.1.0/StyledLayerDescriptor.xsd
–
624705 [INFO] Running org.geotools.wfs.GMLTest
WARNING, opening a connection to http://schemas.opengis.net/wfs/1.1.0/wfs.xsd
WARNING, opening a connection to http://schemas.opengis.net/wfs/1.1.0/wfs.xsd
WARNING, opening a connection to http://schemas.opengis.net/wfs/1.1.0/wfs.xsd
–
628160 [INFO] Running org.geotools.wfs.WFS_2_0_0_ParsingTest
WARNING, opening a connection to http://inspire.ec.europa.eu/draft-schemas/omso/2.0rc3/SpecialisedObservations.xsd
WARNING, opening a connection to http://inspire.ec.europa.eu/schemas/inspire_dls/1.0/inspire_dls.xsd
WARNING, opening a connection to http://inspire.ec.europa.eu/schemas/common/1.0/common.xsd
WARNING, opening a connection to http://inspire.ec.europa.eu/draft-schemas/ef/3.0rc3/EnvironmentalMonitoringFacilities.xsd
–
630462 [INFO] Running org.geotools.wfs.v1_1.WFSParsingTest
WARNING, opening a connection to http://localhost:8080/geoserver/wfs?service=WFS&version=1.1.0&request=DescribeFeatureType&typeName=sf:PrimitiveGeoFeature
–
633671 [INFO] Running org.geotools.wfs.v2_0.bindings.DescribeStoredQueriesTypeBindingTest
WARNING, opening a connection to http://inspire.ec.europa.eu/draft-schemas/omso/2.0rc3/SpecialisedObservations.xsd
WARNING, opening a connection to http://inspire.ec.europa.eu/schemas/common/1.0/common.xsd
–
637740 [INFO] Running org.geotools.wcs.v1_1.DescribeCoverageTest
WARNING, opening a connection to http://schemas.opengis.net/wcs/1.1/wcsAll.xsd
–
639185 [INFO] Running org.geotools.wcs.v1_1.GetCapabilitiesTest
WARNING, opening a connection to http://schemas.opengis.net/wcs/1.1/wcsAll.xsd
–
639536 [INFO] Running org.geotools.wcs.v2_0.GetCoverageTest
WARNING, opening a connection to http://schemas.opengis.net/wcs/geotiff/1.0/wcsGeotiff.xsd
WARNING, opening a connection to http://schemas.opengis.net/wcs/geotiff/1.0/wcsGeotiff.xsd
WARNING, opening a connection to http://schemas.opengis.net/wcs/geotiff/1.0/wcsGeotiff.xsd
WARNING, opening a connection to http://schemas.opengis.net/wcs/geotiff/1.0/wcsGeotiff.xsd
WARNING, opening a connection to http://schemas.opengis.net/wcs/geotiff/1.0/wcsGeotiff.xsd
WARNING, opening a connection to http://schemas.opengis.net/wcs/geotiff/1.0/wcsGeotiff.xsd
WARNING, opening a connection to http://schemas.opengis.net/wcs/geotiff/1.0/wcsGeotiff.xsd
WARNING, opening a connection to http://schemas.opengis.net/wcs/extensions/crs/1.0/wcsCrs.xsd
–
694014 [INFO] Running org.geotools.data.wfs.WFSFeatureStoreTest
WARNING, opening a connection to http://localhost:9090/geoserver/schemas/ows/1.0.0/owsExceptionReport.xsd
–
713406 [INFO] Running org.geotools.data.wfs.internal.parsers.PullParserTest
WARNING, opening a connection to http://localhost:8090/1.7.7/wfs?service=WFS&version=1.0.0&request=DescribeFeatureType&typeName=sf:roads
WARNING, opening a connection to http://nbii-nin.ciesin.columbia.edu/mapserver/wfs/fgdcgubscensustract?REQUEST=DescribeAllFeatureType&SERVICE=WFS
Checking on the internet for ways to detect network access, the third answer about making a SocketImplFactory that logs
any network access attempt might be interesting (don’t think we use NIO much for HTTP):
https://stackoverflow.com/questions/25681670/disable-internet-access-for-jvm
GeoServer centralized test architecture would make it easier to use one of these I guess.
True, this would be something nice to have, with an architecture like this we could even mock network calls.
Cheers
Andrea
_______________________________________________
Geoserver-devel mailing list
[Geoserver-devel@anonymised.comsts.sourceforge.net](mailto:Geoserver-devel@lists.sourceforge.net)
[https://lists.sourceforge.net/lists/listinfo/geoserver-devel](https://lists.sourceforge.net/lists/listinfo/geoserver-devel)
<details class='elided'>
<summary title='Show trimmed content'>···</summary>
== GeoServer Professional Services from the experts! Visit [http://goo.gl/it488V](http://goo.gl/it488V) for more information. == Ing. Andrea Aime @geowolf Technical Lead GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) phone: +39 0584 962313 fax: +39 0584 1660272 mob: +39 339 8844549 [http://www.geo-solutions.it](http://www.geo-solutions.it) [http://twitter.com/geosolutions_it](http://twitter.com/geosolutions_it) ------------------------------------------------------- *Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia. This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.*
</details>