#435: Benchmarking data download server
---------------------+------------------------------------------------------
Reporter: jmckenna | Owner: sac@lists.osgeo.org
Type: task | Status: new
Priority: normal | Component: SAC
Keywords: |
---------------------+------------------------------------------------------
As discussed in the last FOSS4G benchmarking meeting, we need a place to
host the benchmarking data download. It's currently on a USACE server but
connection speed is very slow...and in the long run we think OSGeo should
host the data (and benchmarking test package) for the public to download.
The data is ~32 GB in size.
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/435>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#435: Benchmarking data download server
-----------------------+----------------------------------------------------
Reporter: jmckenna | Owner: sac@lists.osgeo.org
Type: task | Status: new
Priority: normal | Component: SAC
Resolution: | Keywords:
-----------------------+----------------------------------------------------
Comment (by jmckenna):
Can we set this up soon? (I'd like to point ESRI and ERDAS to this)
Eventually this will be how the public can access our Web Server
benchmarking files.
-jeff
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/435#comment:1>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#435: Benchmarking data download server
-----------------------+----------------------------------------------------
Reporter: jmckenna | Owner: sac@lists.osgeo.org
Type: task | Status: new
Priority: normal | Component: SAC
Resolution: | Keywords:
-----------------------+----------------------------------------------------
Changes (by jmckenna):
* cc: dmorissette (added)
Comment:
Daniel, I wonder if maptools.org could host the benchmarking data? (the
data is about 32GB) Can we handle this extra bandwidth demand?
(on IRC FrankW stated that this could not be hosted on telascience
servers)
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/435#comment:2>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#435: Benchmarking data download server
-----------------------+----------------------------------------------------
Reporter: jmckenna | Owner: sac@lists.osgeo.org
Type: task | Status: new
Priority: normal | Component: SAC
Resolution: | Keywords:
-----------------------+----------------------------------------------------
Comment (by aaime):
Mind the biggest part of the data is uncompressed. The shapefile and the
big geotiffs (which are uncompressed) should go down a lot if they are
tar.gz-ed (the ecw will not go down one bit instead).
One tgz per package would be a nice to redistribute the lot.
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/435#comment:3>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#435: Benchmarking data download server
-----------------------+----------------------------------------------------
Reporter: jmckenna | Owner: sac@lists.osgeo.org
Type: task | Status: new
Priority: normal | Component: SAC
Resolution: | Keywords:
-----------------------+----------------------------------------------------
Changes (by warmerdam):
* cc: warmerdam (added)
Comment:
For the raster data I would suggest we only distribute the ecw, and
provide scripts to translate that to bigtiff and tiles and add overviews.
I can prepare those scripts at some point.
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/435#comment:4>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#435: Benchmarking data download server
-----------------------+----------------------------------------------------
Reporter: jmckenna | Owner: sac@lists.osgeo.org
Type: task | Status: new
Priority: normal | Component: SAC
Resolution: | Keywords:
-----------------------+----------------------------------------------------
Comment (by aaime):
Frank, you read my mind. I was actually looking for those scripts to
replicate the tiffs on my box hoping to find them somewhere on goliath,
but I had no such luck
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/435#comment:5>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#435: Benchmarking data download server
-----------------------+----------------------------------------------------
Reporter: jmckenna | Owner: sac@lists.osgeo.org
Type: task | Status: new
Priority: normal | Component: SAC
Resolution: | Keywords:
-----------------------+----------------------------------------------------
Changes (by dmorissette):
* cc: aboudreault (added)
Comment:
Jeff, yes, we could host it on maptools.org as long as we don't get
hundreds of downloads of it per month. However I would not put in the
usual "dl" directory to avoid also wasting space on the master server
which is replicated to the live server. I'd place the files directly in a
directory on the live server bypassing the normal replication mechanism.
Let's look into that with Alan on Monday.
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/435#comment:6>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#435: Benchmarking data download server
-----------------------+----------------------------------------------------
Reporter: jmckenna | Owner: sac@lists.osgeo.org
Type: task | Status: new
Priority: normal | Component: SAC
Resolution: | Keywords:
-----------------------+----------------------------------------------------
Comment (by jmckenna):
Alan, let me know when you can discuss using maptools.org to host the
benchmarking package.
All: I generated a TIGER 2008 dataset for GeoServer and MapServer testing,
that I will upload to this new server (it contains both as-is county files
as well as processed merged files). I'm looking at TIGER 2008 styles now.
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/435#comment:7>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#435: Benchmarking data download server
-----------------------+----------------------------------------------------
Reporter: jmckenna | Owner: sac@lists.osgeo.org
Type: task | Status: new
Priority: normal | Component: SAC
Resolution: | Keywords:
-----------------------+----------------------------------------------------
Comment (by jmckenna):
Alan, the files have transferred to your machine. Please upload the 4 zip
files to maptools.org
raster.zip (.4 GB)
vector-data-original.zip (.5 GB)
vector-data-tiger08-tx-merged.zip (1.1 GB)
vector-date-tiger08-tx-counties.zip (1.1 GB)
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/435#comment:8>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.
#435: Benchmarking data download server
-----------------------+----------------------------------------------------
Reporter: jmckenna | Owner: sac@lists.osgeo.org
Type: task | Status: closed
Priority: normal | Component: SAC
Resolution: fixed | Keywords:
-----------------------+----------------------------------------------------
Changes (by jmckenna):
* status: new => closed
* resolution: => fixed
Comment:
Alan has set this up on maptools.org (notes at
http://wiki.osgeo.org/wiki/Benchmarking_2009#Download).
marking as fixed.
--
Ticket URL: <http://trac.osgeo.org/osgeo/ticket/435#comment:9>
OSGeo <http://www.osgeo.org/>
OSGeo committee and general foundation issue tracker.