[Geoserver-devel] execute python scripting wps

GeoServer GeoServer Developer
1

Hi

I figured out the issue but not a fix. I call WPS Execute for the python scripting hook (wps#1), the python scripting hook calls a rest service, inside the rest service there is another WPS Execute POST (wps#2) to geoserver which posts xml (ie. JTS:intersection). I believe geoserver is waiting for the initial python scripting hook wps execute to complete before the rest service execute. The queue is wps#1, wps#2 and I need wps#2 to execute before wps#1 can finish. If I point the WPS execute to another geoserver everything works fine. Can someone please explain why this is happening (threads?) and Is there a workaround?

WPS Execute → python .py → REST service → WPS Execute

We created a workflow this way so we could dynamically add WPS’s to geoserver without restarting it everytime, and the rest service can pull xml out of the database and execute the xml so we wouldn’t have to create WPS’s in python code.

Thank You
Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

From: Bessette-Halsema, Dominique E
Sent: Friday, April 05, 2013 3:44 PM
To: geoserver-devel
Subject: execute python scripting wps

Hi

First here’s some background on my project architecture.

Geoserver contains the python scripting extension

The python script under geoserver_data_dir/scripts/wps/ contains a name input, a string result, and runs a rest call to an outside rest service (ie. http://localhost:8080/restsvc/wps/execute/name)

The outside rest service then grabs a wps string from a database by name, and posts geoserver wps execute with the wps from the database to get the result.

Everything works fine if I run the rest service from a browser. The service grabs the wps and posts geoserver and returns the output string. But when I try to run everything through geoserver I don’t really get an error, but the browser keeps waiting for a response. I attached a log file that contains the logs from when I call the rest service directly and then through geoserver.

Call to run python script through geoserver:

http://localhost:8080/geoserver/wps?service=WPS&version=1.0.0&request=Execute&identifier=py:callRestSvc&datainputs=name=test4

python script rest call:

def run(name):

url=‘http://localhost:8080/restsvc/wps/execute/’

url+=name

f = urllib.urlopen(url)

return str(f.read(500))

Thank you

Dominique

2

Hi Dominique,

Can you provide the jstack output of the geoserver proess after it hangs, it will show us what all the threads are doing.

-Justin

On Tue, Apr 9, 2013 at 4:41 PM, Bessette-Halsema, Dominique E <Dominique.Bessette@anonymised.com> wrote:

Hi

I figured out the issue but not a fix. I call WPS Execute for the python scripting hook (wps#1), the python scripting hook calls a rest service, inside the rest service there is another WPS Execute POST (wps#2) to geoserver which posts xml (ie. JTS:intersection). I believe geoserver is waiting for the initial python scripting hook wps execute to complete before the rest service execute. The queue is wps#1, wps#2 and I need wps#2 to execute before wps#1 can finish. If I point the WPS execute to another geoserver everything works fine. Can someone please explain why this is happening (threads?) and Is there a workaround?

WPS Execute → python .py → REST service → WPS Execute

We created a workflow this way so we could dynamically add WPS’s to geoserver without restarting it everytime, and the rest service can pull xml out of the database and execute the xml so we wouldn’t have to create WPS’s in python code.

Thank You
Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

From: Bessette-Halsema, Dominique E
Sent: Friday, April 05, 2013 3:44 PM
To: geoserver-devel
Subject: execute python scripting wps

Hi

First here’s some background on my project architecture.

Geoserver contains the python scripting extension

The python script under geoserver_data_dir/scripts/wps/ contains a name input, a string result, and runs a rest call to an outside rest service (ie. http://localhost:8080/restsvc/wps/execute/name)

The outside rest service then grabs a wps string from a database by name, and posts geoserver wps execute with the wps from the database to get the result.

Everything works fine if I run the rest service from a browser. The service grabs the wps and posts geoserver and returns the output string. But when I try to run everything through geoserver I don’t really get an error, but the browser keeps waiting for a response. I attached a log file that contains the logs from when I call the rest service directly and then through geoserver.

Call to run python script through geoserver:

http://localhost:8080/geoserver/wps?service=WPS&version=1.0.0&request=Execute&identifier=py:callRestSvc&datainputs=name=test4

python script rest call:

def run(name):

url=‘http://localhost:8080/restsvc/wps/execute/’

url+=name

f = urllib.urlopen(url)

return str(f.read(500))

Thank you

Dominique

Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

Geoserver-devel mailing list
Geoserver-devel@anonymised.comsts.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

3

Hi!

Here’s the jstack.

Thank You

Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

Office: 619-881-2748

From: Justin Deoliveira [mailto:jdeolive@anonymised.com]
Sent: Wednesday, April 10, 2013 7:53 AM
To: Bessette-Halsema, Dominique E
Cc: geoserver-devel
Subject: Re: [Geoserver-devel] execute python scripting wps

Hi Dominique,

Can you provide the jstack output of the geoserver proess after it hangs, it will show us what all the threads are doing.

-Justin

On Tue, Apr 9, 2013 at 4:41 PM, Bessette-Halsema, Dominique E <Dominique.Bessette@anonymised.com> wrote:

Hi

I figured out the issue but not a fix. I call WPS Execute for the python scripting hook (wps#1), the python scripting hook calls a rest service, inside the rest service there is another WPS Execute POST (wps#2) to geoserver which posts xml (ie. JTS:intersection). I believe geoserver is waiting for the initial python scripting hook wps execute to complete before the rest service execute. The queue is wps#1, wps#2 and I need wps#2 to execute before wps#1 can finish. If I point the WPS execute to another geoserver everything works fine. Can someone please explain why this is happening (threads?) and Is there a workaround?

WPS Execute → python .py → REST service → WPS Execute

We created a workflow this way so we could dynamically add WPS’s to geoserver without restarting it everytime, and the rest service can pull xml out of the database and execute the xml so we wouldn’t have to create WPS’s in python code.

Thank You
Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

From: Bessette-Halsema, Dominique E
Sent: Friday, April 05, 2013 3:44 PM
To: geoserver-devel
Subject: execute python scripting wps

Hi

First here’s some background on my project architecture.

Geoserver contains the python scripting extension

The python script under geoserver_data_dir/scripts/wps/ contains a name input, a string result, and runs a rest call to an outside rest service (ie. http://localhost:8080/restsvc/wps/execute/name)

The outside rest service then grabs a wps string from a database by name, and posts geoserver wps execute with the wps from the database to get the result.

Everything works fine if I run the rest service from a browser. The service grabs the wps and posts geoserver and returns the output string. But when I try to run everything through geoserver I don’t really get an error, but the browser keeps waiting for a response. I attached a log file that contains the logs from when I call the rest service directly and then through geoserver.

Call to run python script through geoserver:

http://localhost:8080/geoserver/wps?service=WPS&version=1.0.0&request=Execute&identifier=py:callRestSvc&datainputs=name=test4

python script rest call:

def run(name):

url=‘http://localhost:8080/restsvc/wps/execute/’

url+=name

f = urllib.urlopen(url)

return str(f.read(500))

Thank you

Dominique

Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

Geoserver-devel mailing list
Geoserver-devel@anonymised.comrge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira

OpenGeo - http://opengeo.org

Enterprise support for open source geospatial.

(attachments)

jboss-jstack2.txt (81.9 KB)

4

Thanks Dominique,

My guess is contention over the thread pool used to execute proceses. When a process executes it does so in a fixed size thread pool. If the thread pool is full (all threads are currently busy executing) the process execution will wait, and block if its a synchronous execution.

So i think what has happened is deadlock, with process “A” waiting on process “B”, but “B” waiting for a spot in the thread pool which won’t become available until “A” is finished.

By default the size of the thread pool is 2 times the number of cores in the machine. So worst case is 2 threads in size which means two invocations of your process in parallel can produce a deadlock. The thread pool size is configurable so you could increase it, but that is really a bandaid solution. But it could work in conjunction with the control flow module that can limit the overall number of max requests a service can handle. So you just need to make sure that the thread pool size is greater than the max number of WPS execute requests that the server can handle.

Really what is needed is an executor for processes that does so in a time sliced fashion. However this would require a number of modifications, and has implications for how processes are authored. Perhaps Andrea, the maintainer of the wps module will have a better solution.

-Justin

On Wed, Apr 10, 2013 at 10:42 AM, Bessette-Halsema, Dominique E <Dominique.Bessette@anonymised.com> wrote:

Hi!

Here’s the jstack.

Thank You

Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

Office: 619-881-2748

From: Justin Deoliveira [mailto:jdeolive@anonymised.com]
Sent: Wednesday, April 10, 2013 7:53 AM
To: Bessette-Halsema, Dominique E
Cc: geoserver-devel
Subject: Re: [Geoserver-devel] execute python scripting wps

Hi Dominique,

Can you provide the jstack output of the geoserver proess after it hangs, it will show us what all the threads are doing.

-Justin

On Tue, Apr 9, 2013 at 4:41 PM, Bessette-Halsema, Dominique E <Dominique.Bessette@anonymised.com> wrote:

Hi

I figured out the issue but not a fix. I call WPS Execute for the python scripting hook (wps#1), the python scripting hook calls a rest service, inside the rest service there is another WPS Execute POST (wps#2) to geoserver which posts xml (ie. JTS:intersection). I believe geoserver is waiting for the initial python scripting hook wps execute to complete before the rest service execute. The queue is wps#1, wps#2 and I need wps#2 to execute before wps#1 can finish. If I point the WPS execute to another geoserver everything works fine. Can someone please explain why this is happening (threads?) and Is there a workaround?

WPS Execute → python .py → REST service → WPS Execute

We created a workflow this way so we could dynamically add WPS’s to geoserver without restarting it everytime, and the rest service can pull xml out of the database and execute the xml so we wouldn’t have to create WPS’s in python code.

Thank You
Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

From: Bessette-Halsema, Dominique E
Sent: Friday, April 05, 2013 3:44 PM
To: geoserver-devel
Subject: execute python scripting wps

Hi

First here’s some background on my project architecture.

Geoserver contains the python scripting extension

The python script under geoserver_data_dir/scripts/wps/ contains a name input, a string result, and runs a rest call to an outside rest service (ie. http://localhost:8080/restsvc/wps/execute/name)

The outside rest service then grabs a wps string from a database by name, and posts geoserver wps execute with the wps from the database to get the result.

Everything works fine if I run the rest service from a browser. The service grabs the wps and posts geoserver and returns the output string. But when I try to run everything through geoserver I don’t really get an error, but the browser keeps waiting for a response. I attached a log file that contains the logs from when I call the rest service directly and then through geoserver.

Call to run python script through geoserver:

http://localhost:8080/geoserver/wps?service=WPS&version=1.0.0&request=Execute&identifier=py:callRestSvc&datainputs=name=test4

python script rest call:

def run(name):

url=‘http://localhost:8080/restsvc/wps/execute/’

url+=name

f = urllib.urlopen(url)

return str(f.read(500))

Thank you

Dominique

Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira

OpenGeo - http://opengeo.org

Enterprise support for open source geospatial.

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.

5

Hi

I put this issue on the back burner for a while but I noticed that you can up the threads in the GeoServer WPS Services console. The default thread count was set to 1. I was not aware of this capability and upping the thread count fixed the threading issue with GeoServer the python scripting. We are continuing to stress test the python scripts to see what the thread count should be set to.

Directions

Login to the Geoserver console (admin:geoserver)

Click on WPS, under the Services section

Scroll to the bottom and change the thread counts to 10

Click save.

I was wondering why we needed to up the WPS thread count for the python scripting calls but not for direct WPS calls? We make several WPS calls a day and have not run into threading issues.

Thanks!

Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

Office: 619-881-2748

From: Bessette-Halsema, Dominique E
Sent: Wednesday, April 10, 2013 9:43 AM
To: ‘Justin Deoliveira’
Cc: geoserver-devel
Subject: RE: [Geoserver-devel] execute python scripting wps

Hi!

Here’s the jstack.

Thank You

Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

Office: 619-881-2748

From: Justin Deoliveira [mailto:jdeolive@anonymised.com]
Sent: Wednesday, April 10, 2013 7:53 AM
To: Bessette-Halsema, Dominique E
Cc: geoserver-devel
Subject: Re: [Geoserver-devel] execute python scripting wps

Hi Dominique,

Can you provide the jstack output of the geoserver proess after it hangs, it will show us what all the threads are doing.

-Justin

On Tue, Apr 9, 2013 at 4:41 PM, Bessette-Halsema, Dominique E <Dominique.Bessette@anonymised.com> wrote:

Hi

I figured out the issue but not a fix. I call WPS Execute for the python scripting hook (wps#1), the python scripting hook calls a rest service, inside the rest service there is another WPS Execute POST (wps#2) to geoserver which posts xml (ie. JTS:intersection). I believe geoserver is waiting for the initial python scripting hook wps execute to complete before the rest service execute. The queue is wps#1, wps#2 and I need wps#2 to execute before wps#1 can finish. If I point the WPS execute to another geoserver everything works fine. Can someone please explain why this is happening (threads?) and Is there a workaround?

WPS Execute → python .py → REST service → WPS Execute

We created a workflow this way so we could dynamically add WPS’s to geoserver without restarting it everytime, and the rest service can pull xml out of the database and execute the xml so we wouldn’t have to create WPS’s in python code.

Thank You
Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

From: Bessette-Halsema, Dominique E
Sent: Friday, April 05, 2013 3:44 PM
To: geoserver-devel
Subject: execute python scripting wps

Hi

First here’s some background on my project architecture.

Geoserver contains the python scripting extension

The python script under geoserver_data_dir/scripts/wps/ contains a name input, a string result, and runs a rest call to an outside rest service (ie. http://localhost:8080/restsvc/wps/execute/name)

The outside rest service then grabs a wps string from a database by name, and posts geoserver wps execute with the wps from the database to get the result.

Everything works fine if I run the rest service from a browser. The service grabs the wps and posts geoserver and returns the output string. But when I try to run everything through geoserver I don’t really get an error, but the browser keeps waiting for a response. I attached a log file that contains the logs from when I call the rest service directly and then through geoserver.

Call to run python script through geoserver:

http://localhost:8080/geoserver/wps?service=WPS&version=1.0.0&request=Execute&identifier=py:callRestSvc&datainputs=name=test4

python script rest call:

def run(name):

url=‘http://localhost:8080/restsvc/wps/execute/’

url+=name

f = urllib.urlopen(url)

return str(f.read(500))

Thank you

Dominique

Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira

OpenGeo - http://opengeo.org

Enterprise support for open source geospatial.

6

Hi Dominique,

When you say “direct wps calls” do you mean a WPS request that makes a chained call to another? If so then the answer is that GeoServer is smart enough to recognize that the call is destined for the same server and just executes it inline rather than creating an http connection back to itself to execute the sub process.

-Justin

···

On Fri, Jun 21, 2013 at 3:59 PM, Bessette-Halsema, Dominique E <Dominique.Bessette@anonymised.com> wrote:

Hi

I put this issue on the back burner for a while but I noticed that you can up the threads in the GeoServer WPS Services console. The default thread count was set to 1. I was not aware of this capability and upping the thread count fixed the threading issue with GeoServer the python scripting. We are continuing to stress test the python scripts to see what the thread count should be set to.

Directions

Login to the Geoserver console (admin:geoserver)

Click on WPS, under the Services section

Scroll to the bottom and change the thread counts to 10

Click save.

I was wondering why we needed to up the WPS thread count for the python scripting calls but not for direct WPS calls? We make several WPS calls a day and have not run into threading issues.

Thanks!

Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

Office: 619-881-2748

From: Bessette-Halsema, Dominique E
Sent: Wednesday, April 10, 2013 9:43 AM
To: ‘Justin Deoliveira’
Cc: geoserver-devel
Subject: RE: [Geoserver-devel] execute python scripting wps

Hi!

Here’s the jstack.

Thank You

Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

Office: 619-881-2748

From: Justin Deoliveira [mailto:jdeolive@anonymised.com]

Sent: Wednesday, April 10, 2013 7:53 AM
To: Bessette-Halsema, Dominique E
Cc: geoserver-devel

Subject: Re: [Geoserver-devel] execute python scripting wps

Hi Dominique,

Can you provide the jstack output of the geoserver proess after it hangs, it will show us what all the threads are doing.

-Justin

On Tue, Apr 9, 2013 at 4:41 PM, Bessette-Halsema, Dominique E <Dominique.Bessette@anonymised.com> wrote:

Hi

I figured out the issue but not a fix. I call WPS Execute for the python scripting hook (wps#1), the python scripting hook calls a rest service, inside the rest service there is another WPS Execute POST (wps#2) to geoserver which posts xml (ie. JTS:intersection). I believe geoserver is waiting for the initial python scripting hook wps execute to complete before the rest service execute. The queue is wps#1, wps#2 and I need wps#2 to execute before wps#1 can finish. If I point the WPS execute to another geoserver everything works fine. Can someone please explain why this is happening (threads?) and Is there a workaround?

WPS Execute → python .py → REST service → WPS Execute

We created a workflow this way so we could dynamically add WPS’s to geoserver without restarting it everytime, and the rest service can pull xml out of the database and execute the xml so we wouldn’t have to create WPS’s in python code.

Thank You
Dominique Bessette

Engineer, Software

General Dynamics Information Technology

Supporting Fleet Numerical Meteorology and Oceanography Center (FNMOC)

From: Bessette-Halsema, Dominique E
Sent: Friday, April 05, 2013 3:44 PM
To: geoserver-devel
Subject: execute python scripting wps

Hi

First here’s some background on my project architecture.

Geoserver contains the python scripting extension

The python script under geoserver_data_dir/scripts/wps/ contains a name input, a string result, and runs a rest call to an outside rest service (ie. http://localhost:8080/restsvc/wps/execute/name)

The outside rest service then grabs a wps string from a database by name, and posts geoserver wps execute with the wps from the database to get the result.

Everything works fine if I run the rest service from a browser. The service grabs the wps and posts geoserver and returns the output string. But when I try to run everything through geoserver I don’t really get an error, but the browser keeps waiting for a response. I attached a log file that contains the logs from when I call the rest service directly and then through geoserver.

Call to run python script through geoserver:

http://localhost:8080/geoserver/wps?service=WPS&version=1.0.0&request=Execute&identifier=py:callRestSvc&datainputs=name=test4

python script rest call:

def run(name):

url=‘http://localhost:8080/restsvc/wps/execute/’

url+=name

f = urllib.urlopen(url)

return str(f.read(500))

Thank you

Dominique

Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

–
Justin Deoliveira

OpenGeo - http://opengeo.org

Enterprise support for open source geospatial.

–
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.