I recently tried to apply a fix to the notorious memory leak that has been plaguing describe feature type for quite some time. Unfortunately as I found it breaks app-schema.
I have rolled back the patch for now and am trying to figure out why app-schema is breaking. The breakage in question is occurring because the patch I am trying to apply builds the XSDSchema object (from FeatureTypeSchemaBuilder) slightly differently. Instead of creating feature elements that directly reference the "_Feature" substitution group they now reference a clone of it.
The reason for now referencing the clone is because when you add an object to another objects substitution group it creates a back reference which causes the memory leak.
A quick hack would be to simply enable the fix for the simple feature type case, for which it seems to be no issue. But before resorting to that measure i would like to first understand the underying issue. So I am going to try to dig down in to app-schema to figure out what is occurring. I seem to remember something about substitution groups coming up a while back with the encoder but i can't quite remember.
Any help from the app-schema folks would be much appreciated
-Justin
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
Ok after some digging it does appear that when encoding complex attributes app-schema does require the substitution group to be set and a clone will not cut it.
I have modified the patch so that the sub group is kept in tact but in a way that will not continually leak memory. App schema is happy again. Will re apply soon. Going to run it through cite tests just to be extra sure.
-Justin
On 10-07-13 4:43 PM, Justin Deoliveira wrote:
Hi all,
I recently tried to apply a fix to the notorious memory leak that has
been plaguing describe feature type for quite some time. Unfortunately
as I found it breaks app-schema.
I have rolled back the patch for now and am trying to figure out why
app-schema is breaking. The breakage in question is occurring because
the patch I am trying to apply builds the XSDSchema object (from
FeatureTypeSchemaBuilder) slightly differently. Instead of creating
feature elements that directly reference the "_Feature" substitution
group they now reference a clone of it.
The reason for now referencing the clone is because when you add an
object to another objects substitution group it creates a back reference
which causes the memory leak.
A quick hack would be to simply enable the fix for the simple feature
type case, for which it seems to be no issue. But before resorting to
that measure i would like to first understand the underying issue. So I
am going to try to dig down in to app-schema to figure out what is
occurring. I seem to remember something about substitution groups coming
up a while back with the encoder but i can't quite remember.
Any help from the app-schema folks would be much appreciated
-Justin
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
I can confirm that substitution groups must be set correctly to support property types that contain feature types.
Kind regards,
Ben.
On 14/07/10 07:53, Justin Deoliveira wrote:
Ok after some digging it does appear that when encoding complex
attributes app-schema does require the substitution group to be set and
a clone will not cut it.
I have modified the patch so that the sub group is kept in tact but in a
way that will not continually leak memory. App schema is happy again.
Will re apply soon. Going to run it through cite tests just to be extra
sure.
-Justin
On 10-07-13 4:43 PM, Justin Deoliveira wrote:
Hi all,
I recently tried to apply a fix to the notorious memory leak that has
been plaguing describe feature type for quite some time. Unfortunately
as I found it breaks app-schema.
I have rolled back the patch for now and am trying to figure out why
app-schema is breaking. The breakage in question is occurring because
the patch I am trying to apply builds the XSDSchema object (from
FeatureTypeSchemaBuilder) slightly differently. Instead of creating
feature elements that directly reference the "_Feature" substitution
group they now reference a clone of it.
The reason for now referencing the clone is because when you add an
object to another objects substitution group it creates a back reference
which causes the memory leak.
A quick hack would be to simply enable the fix for the simple feature
type case, for which it seems to be no issue. But before resorting to
that measure i would like to first understand the underying issue. So I
am going to try to dig down in to app-schema to figure out what is
occurring. I seem to remember something about substitution groups coming
up a while back with the encoder but i can't quite remember.
Any help from the app-schema folks would be much appreciated
-Justin
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre
Unfortunately the way we are using substitution groups via EMF is quite flawed. _Feature lives in the gml schema... which we try to treat as immutable for the most part. But when you create an element that is in the _Feature substitution group (like we do in a DFT call), that element gets added to a list maintained by the "_Feature" element which leads to a leak. It also modifies a singleton object in a non thread safe way...
sigh... EMF/XSD... sure seemed like a good idea at the time.
On 10-07-13 7:51 PM, Ben Caradoc-Davies wrote:
Good to hear, Justin.
I can confirm that substitution groups must be set correctly to support
property types that contain feature types.
Kind regards,
Ben.
On 14/07/10 07:53, Justin Deoliveira wrote:
Ok after some digging it does appear that when encoding complex
attributes app-schema does require the substitution group to be set and
a clone will not cut it.
I have modified the patch so that the sub group is kept in tact but in a
way that will not continually leak memory. App schema is happy again.
Will re apply soon. Going to run it through cite tests just to be extra
sure.
-Justin
On 10-07-13 4:43 PM, Justin Deoliveira wrote:
Hi all,
I recently tried to apply a fix to the notorious memory leak that has
been plaguing describe feature type for quite some time. Unfortunately
as I found it breaks app-schema.
I have rolled back the patch for now and am trying to figure out why
app-schema is breaking. The breakage in question is occurring because
the patch I am trying to apply builds the XSDSchema object (from
FeatureTypeSchemaBuilder) slightly differently. Instead of creating
feature elements that directly reference the "_Feature" substitution
group they now reference a clone of it.
The reason for now referencing the clone is because when you add an
object to another objects substitution group it creates a back reference
which causes the memory leak.
A quick hack would be to simply enable the fix for the simple feature
type case, for which it seems to be no issue. But before resorting to
that measure i would like to first understand the underying issue. So I
am going to try to dig down in to app-schema to figure out what is
occurring. I seem to remember something about substitution groups coming
up a while back with the encoder but i can't quite remember.
Any help from the app-schema folks would be much appreciated
-Justin
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
Does this connect two unrelated application schemas if they both define new elements in the _Feature substitution group? Do they end up sharing a single substitution group? That would not be good.
[And are you 12 hours behind me, still working at midnight? Past your bedtime.]
On 14/07/10 11:55, Justin Deoliveira wrote:
Unfortunately the way we are using substitution groups via EMF is quite
flawed. _Feature lives in the gml schema... which we try to treat as
immutable for the most part. But when you create an element that is in
the _Feature substitution group (like we do in a DFT call), that element
gets added to a list maintained by the "_Feature" element which leads to
a leak. It also modifies a singleton object in a non thread safe way...
sigh... EMF/XSD... sure seemed like a good idea at the time.
On 10-07-13 7:51 PM, Ben Caradoc-Davies wrote:
Good to hear, Justin.
I can confirm that substitution groups must be set correctly to support
property types that contain feature types.
Kind regards,
Ben.
On 14/07/10 07:53, Justin Deoliveira wrote:
Ok after some digging it does appear that when encoding complex
attributes app-schema does require the substitution group to be set and
a clone will not cut it.
I have modified the patch so that the sub group is kept in tact but in a
way that will not continually leak memory. App schema is happy again.
Will re apply soon. Going to run it through cite tests just to be extra
sure.
-Justin
On 10-07-13 4:43 PM, Justin Deoliveira wrote:
Hi all,
I recently tried to apply a fix to the notorious memory leak that has
been plaguing describe feature type for quite some time. Unfortunately
as I found it breaks app-schema.
I have rolled back the patch for now and am trying to figure out why
app-schema is breaking. The breakage in question is occurring because
the patch I am trying to apply builds the XSDSchema object (from
FeatureTypeSchemaBuilder) slightly differently. Instead of creating
feature elements that directly reference the "_Feature" substitution
group they now reference a clone of it.
The reason for now referencing the clone is because when you add an
object to another objects substitution group it creates a back reference
which causes the memory leak.
A quick hack would be to simply enable the fix for the simple feature
type case, for which it seems to be no issue. But before resorting to
that measure i would like to first understand the underying issue. So I
am going to try to dig down in to app-schema to figure out what is
occurring. I seem to remember something about substitution groups coming
up a while back with the encoder but i can't quite remember.
Any help from the app-schema folks would be much appreciated
-Justin
--
Ben Caradoc-Davies <Ben.Caradoc-Davies@anonymised.com>
Software Engineering Team Leader
CSIRO Earth Science and Resource Engineering
Australian Resources Research Centre
On Wed, Jul 14, 2010 at 1:55 PM, Justin Deoliveira <jdeolive@anonymised.com> wrote:
Unfortunately the way we are using substitution groups via EMF is quite
flawed. _Feature lives in the gml schema... which we try to treat as
immutable for the most part. But when you create an element that is in
the _Feature substitution group (like we do in a DFT call), that element
gets added to a list maintained by the "_Feature" element which leads to
a leak. It also modifies a singleton object in a non thread safe way...
sigh... EMF/XSD... sure seemed like a good idea at the time.
Well it did hit our goal of being an non-internal representation of
XML Schema that would be around in five years time.
Been thinking about this; XSD is keeping things as singletons
expecting a schema to stay well defined. The OGC is being strange in
having an abstract schema reused on a case-by-case basis (or ever
worse cutting up the schema on a case-by-case basis and providing only
the subset used).
For the specific case of "_Feature" could we have our own list of
valid substitutions? Or can we replace the singleton with a hand
corrected instance that is thread safe and that can be cleaned up
after? The EMF library is pretty good about letting the resulting
models be hacked; so I imagine we could insert or provide an
implementation.
And yes Ben; I suspect for the web tools platform that schemas are
held in singletons to avoid reparsing them all the time.
