[GeoNetwork-devel] [Re: [SAC] Accessing and using projects.osgeo.osuosl.org]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

fyi

- -------- Original Message --------
Subject: Re: [SAC] Accessing and using projects.osgeo.osuosl.org
Date: Thu, 1 Jul 2010 16:54:42 +0200
From: <christopher.schmidt@anonymised.com>
Reply-To: System Administration Committee Discussion/OSGeo
<sac@anonymised.com>
To: <sac@anonymised.com>
CC: geodata@anonymised.com, mapbender_dev@anonymised.com,
geonetwork-devel@lists.sourceforge.net
References: <4C2C45F5.7040801@anonymised.com>

(I"m not on several of the lists CCed here; please feel free to forward
this response as you see fit.)

On Thu, Jul 01, 2010 at 09:38:29AM +0200, Seven (aka Arnulf) wrote:

Hi,
maybe this has all been resolved yet, please jsut send a link where to
read up on.

Sorry, I need to write up a plan for projects; I've been shipping the
kids off to camp the last two weekends, so I got distracted. I'll make
sure to get a writeup this weekend. (though I'm not really sure this
is on the right list, since this seems

This might evolve into a ticket but maybe we need to have a broader
dialog before actually starting to decide and then do things.

We (GN, MB, Geodata Committee) have collected some requirements for what
we need to run metadatata.osgeo.org. I guess that other projects need
similar things and I am absolutely unclear on how this can be organized
so that we don't step on each other's toes.

Here goes things that I believe sudo rights are required + they can
conflict with other's requirements:

Apache
* add aliases
* add modules (rewrite, etc.)
* ssh / certificats (this could probably managed by SAC and does not
need changes often)

I don't understand this. Are you *developing* a service, or *running*
a service? If you're developing software, then the projects server
is not the right place to put it.

PostgreSQL
* create database
* create users
* change ownerships
* add triggers, functions, etc.

This is all easy; none of this requires access to bring down a service.
However, the latter comes down to the same question as above: are you
writing software, or running software?

Installations:
* PostGIS (maybe in different versions for different projects?)
* PHP
* gettext
* all the nitty gritty personally preferred admin tools

These are easy (and already done, except for gettext; which is
installed, but I'm assuming you want something other than the C
library.)

We will need to edit PHP.INI regularly, try different settings. This
requires to restart Apache / force reload.

To be frank: No. This is a perfectly reasonable thing to do in testing
of some new software, but I don't understand why you think this would
be required in restarting apache regularly in a production service;
if you're asking for a machine to develop on, projects isn't it.

We will have to deal with https, connect to LDAP, maybe create users or
groups for LDAP (this could or should probably also be done by SAC)?

Creating LDAP users can be done by having the users create accounts.
Creating groups requires the LDAP Manager password, and should be managed
by SAC. Adding users to groups can be done by other users once the group
is created.

There will probably be more details as we go along, this is just to get
us started.

The Geodata Committee wants to grow a service for all types of metadata
for services, dataset, applications, portals and so on. The Mapbender
and GeoNetwork teams are interested in supporting this project from the
technical side. If sucessful it will be a well received OSGeo service.
If not highly available and stable it has no chance of becoming
sucessful which puts us into this old deadlock.

... You want something highly available and stable, but apparently plan
to develop the software on the production machine? This is definitely
a problem, but the problem is not on SAC's side; it's in the social
aspect.

These are just initial ideas, maybe you already have it all worked out
and can point to a policy page that describes how to deal with this.

Very little of this is not solvable technically. Instead, the problem
here is social: You seem to be proposing developing a service that
you wish to maintain for the public on the 'live'/production service.

If you don't care about uptime, that's fine, but you've said that you
do, so I'm trapped between a rock and a hard place here.

So let me propose something slightly different here:

1. Seperate development and deployment of updates to development.
    This means that your constant PHP tweaks, configuration changes,
    etc. take place on another machine.
2. Deployment takes place on the projects server; one or two members
    of the projects have sudo access on the projects server to change
    the bits on the server that need changing when a deployment is
    taking place. Since these types of deployments should be tested,
    a new one shouldn't be particularly difficult to arrange.
    (Considering our current uptime for some projects is approaching
     the 'nine fives' SLA level, a couple minutes of downtime during
    a transition is unlikely to be a significant impact.)
3. Development takes place on some other server; this is where all
    the constant tweaking takes place. (In any real development setup,
    I'd expect each developer to need their own local development
    setup; certainly that's the way I've developed my software over
    the years, but maybe that's Just Crazy.)

The key difference between #2 and #3 is that #3 can be a 'nothing'
machine; since it has no users other than developers, perforamcne
and uptime are not issues. (The exact location of this development
machine is not something I've settled on; I'll need to check with
some other people to figure out the status of our incoming
resources.)

Even if I/SAC were to hand over a full, high powered VM, the idea
of running a production service on the development machine is just
off; we need to separate the two to avoid the very problems you've
stated you wish to avoid. So I'd recommend avoiding them by using
a separated environment.

If I've missed something significant here, I apologize, and look
forward to a clarification.

Best Regards,
- --
Christopher Schmidt

_______________________________________________
Sac mailing list
Sac@anonymised.com
http://lists.osgeo.org/mailman/listinfo/sac

- --
Arnulf Christl

Exploring Space, Time and Mind
http://arnulf.us
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkwsxXsACgkQXmFKW+BJ1b2Q7QCeNr76RdzKUAAmzcDSEji1VaHq
lIYAn2avuSp8HvsJrhc1mB7KQWXYquXF
=kb6m
-----END PGP SIGNATURE-----

Hi all,

I tried with JMeter using this request with the geonetwork

<?xml version="1.0" encoding="UTF-8"?>
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope&quot;&gt;
  <env:Body>
    <csw:GetRecords
xmlns:csw="http://www.opengis.net/cat/csw/2.0.2&quot;&gt;
xmlns:ogc="http://www.opengis.net/ogc&quot; maxRecords="10"
         outputFormat="application/xml" outputSchema="
http://www.opengis.net/cat/csw/2.0.2&quot;
         resultType="results" service="CSW" version="2.0.2">
      <csw:Query typeNames="csw:Record">
        <csw:ElementSetName>full</csw:ElementSetName>
        <csw:Constraint version="1.1.0">
          <ogc:Filter
              xmlns:gml="http://www.opengis.net/gml&quot;&gt;
        <ogc:PropertyIsLike escape="\" singleChar="_" wildCard="%">
          <ogc:PropertyName>AnyText</ogc:PropertyName>
          <ogc:Literal>%water%</ogc:Literal>
        </ogc:PropertyIsLike>
          </ogc:Filter>
        </csw:Constraint>
      </csw:Query>
    </csw:GetRecords>
  </env:Body>
</env:Envelope>

64 threads/0s ramp period/ no error
128 threads/0s ramp period/ there will be an error in geonetwork.log :

2010-07-01 15:55:47,074 ERROR [jeeves.service] - (C) Exc : org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed
2010-07-01 15:55:47,075 DEBUG [jeeves.service] - Raised exception while executing service
<error id="error">
  <message>this IndexReader is closed</message>
  <class>AlreadyClosedException</class>
  <stack>
    <at class="org.apache.lucene.index.IndexReader" file="IndexReader.java" line="209" method="ensureOpen" />
    <at class="org.apache.lucene.index.DirectoryOwningReader" file="DirectoryOwningReader.java" line="44" method="reopen" />
    <at class="org.fao.geonet.kernel.search.LuceneIndexReaderFactory" file="LuceneIndexReaderFactory.java" line="74" method="maybeReopen" />
    <at class="org.fao.geonet.kernel.search.LuceneIndexReaderFactory" file="LuceneIndexReaderFactory.java" line="41" method="getReader" />
    <at class="org.fao.geonet.kernel.search.SearchManager" file="SearchManager.java" line="744" method="getIndexReader" />
    <at class="org.fao.geonet.kernel.search.LuceneSearcher" file="LuceneSearcher.java" line="134" method="search" />
    <at class="org.fao.geonet.services.main.Search" file="Search.java" line="146" method="exec" />
    <at class="jeeves.server.dispatchers.ServiceInfo" file="???" line="-1" method="execService" />
    <at class="jeeves.server.dispatchers.ServiceInfo" file="???" line="-1" method="execServices" />
    <at class="jeeves.server.dispatchers.ServiceManager" file="???" line="-1" method="dispatch" />
  </stack>
  <request>
    <language>en</language>
    <service>main.search.embedded</service>
  </request>
</error>

And I got the abnormal response in JMeter like that:
<?xml version="1.0" encoding="UTF-8"?>^M
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope&quot;&gt;^M
  <env:Body>^M
    <env:Fault>^M
      <env:Code>^M
        <env:Value>env:Sender</env:Value>^M
        <env:Subcode>^M
          <env:Value>NoApplicableCode</env:Value>^M
        </env:Subcode>^M
      </env:Code>^M
      <env:Reason>^M
        <env:Text xml:lang="en">org.fao.geonet.csw.common.exceptions.NoApplicableCodeEx: code=NoApplicableCode, locator=null, message=Raised exception while searching metadata : org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed</env:Text>^M
      </env:Reason>^M
      <env:Detail>^M
        <ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows&quot; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot; version="1.0.0" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd&quot;&gt;^M
          <ows:Exception exceptionCode="NoApplicableCode">^M
            <ows:ExceptionText>Raised exception while searching metadata : org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed</ows:ExceptionText>^M
          </ows:Exception>^M
        </ows:ExceptionReport>^M
      </env:Detail>^M
    </env:Fault>^M
  </env:Body>^M
</env:Envelope>^

I tested GetrecordByid with 1024 threads and it worked. The error occurred while sending "GetRecords" Request.

Can anybody tell me how to resolve this?
Thanks,

Kai
Joint Center for Intelligent Spatial Computing
703-395-2337

Hi Kai,

I suspect some additional synchronization is required on the incRef and decRef methods of the IndexReader. I'll take a look later today if no one else has done this already?

Cheers,
Simon
________________________________________
From: Kai Liu [kliu4@anonymised.com]
Sent: Friday, 2 July 2010 6:26 AM
To: jose garcia; geonetwork-devel@lists.sourceforge.net; qhuang1@anonymised.com
Subject: [GeoNetwork-devel] org.apache.lucene.store.AlreadyClosedException occurs for concurrent requests

Hi all,

I tried with JMeter using this request with the geonetwork

<?xml version="1.0" encoding="UTF-8"?>
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope&quot;&gt;
  <env:Body>
    <csw:GetRecords
xmlns:csw="http://www.opengis.net/cat/csw/2.0.2&quot;&gt;
xmlns:ogc="http://www.opengis.net/ogc&quot; maxRecords="10"
         outputFormat="application/xml" outputSchema="
http://www.opengis.net/cat/csw/2.0.2&quot;
         resultType="results" service="CSW" version="2.0.2">
      <csw:Query typeNames="csw:Record">
        <csw:ElementSetName>full</csw:ElementSetName>
        <csw:Constraint version="1.1.0">
          <ogc:Filter
              xmlns:gml="http://www.opengis.net/gml&quot;&gt;
        <ogc:PropertyIsLike escape="\" singleChar="_" wildCard="%">
          <ogc:PropertyName>AnyText</ogc:PropertyName>
          <ogc:Literal>%water%</ogc:Literal>
        </ogc:PropertyIsLike>
          </ogc:Filter>
        </csw:Constraint>
      </csw:Query>
    </csw:GetRecords>
  </env:Body>
</env:Envelope>

64 threads/0s ramp period/ no error
128 threads/0s ramp period/ there will be an error in geonetwork.log :

2010-07-01 15:55:47,074 ERROR [jeeves.service] - (C) Exc : org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed
2010-07-01 15:55:47,075 DEBUG [jeeves.service] - Raised exception while executing service
<error id="error">
  <message>this IndexReader is closed</message>
  <class>AlreadyClosedException</class>
  <stack>
    <at class="org.apache.lucene.index.IndexReader" file="IndexReader.java" line="209" method="ensureOpen" />
    <at class="org.apache.lucene.index.DirectoryOwningReader" file="DirectoryOwningReader.java" line="44" method="reopen" />
    <at class="org.fao.geonet.kernel.search.LuceneIndexReaderFactory" file="LuceneIndexReaderFactory.java" line="74" method="maybeReopen" />
    <at class="org.fao.geonet.kernel.search.LuceneIndexReaderFactory" file="LuceneIndexReaderFactory.java" line="41" method="getReader" />
    <at class="org.fao.geonet.kernel.search.SearchManager" file="SearchManager.java" line="744" method="getIndexReader" />
    <at class="org.fao.geonet.kernel.search.LuceneSearcher" file="LuceneSearcher.java" line="134" method="search" />
    <at class="org.fao.geonet.services.main.Search" file="Search.java" line="146" method="exec" />
    <at class="jeeves.server.dispatchers.ServiceInfo" file="???" line="-1" method="execService" />
    <at class="jeeves.server.dispatchers.ServiceInfo" file="???" line="-1" method="execServices" />
    <at class="jeeves.server.dispatchers.ServiceManager" file="???" line="-1" method="dispatch" />
  </stack>
  <request>
    <language>en</language>
    <service>main.search.embedded</service>
  </request>
</error>

And I got the abnormal response in JMeter like that:
<?xml version="1.0" encoding="UTF-8"?>^M
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope&quot;&gt;^M
  <env:Body>^M
    <env:Fault>^M
      <env:Code>^M
        <env:Value>env:Sender</env:Value>^M
        <env:Subcode>^M
          <env:Value>NoApplicableCode</env:Value>^M
        </env:Subcode>^M
      </env:Code>^M
      <env:Reason>^M
        <env:Text xml:lang="en">org.fao.geonet.csw.common.exceptions.NoApplicableCodeEx: code=NoApplicableCode, locator=null, message=Raised exception while searching metadata : org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed</env:Text>^M
      </env:Reason>^M
      <env:Detail>^M
        <ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows&quot; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot; version="1.0.0" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd&quot;&gt;^M
          <ows:Exception exceptionCode="NoApplicableCode">^M
            <ows:ExceptionText>Raised exception while searching metadata : org.apache.lucene.store.AlreadyClosedException: this IndexReader is closed</ows:ExceptionText>^M
          </ows:Exception>^M
        </ows:ExceptionReport>^M
      </env:Detail>^M
    </env:Fault>^M
  </env:Body>^M
</env:Envelope>^

I tested GetrecordByid with 1024 threads and it worked. The error occurred while sending "GetRecords" Request.

Can anybody tell me how to resolve this?
Thanks,

Kai
Joint Center for Intelligent Spatial Computing
703-395-2337

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork