R: Re[2]: [Geoserver-devel] GeoServer logging subsystem - how to configure?

You can try loading a separate configuration file for the Java Logging
system, like this:
  InputStream is =
getClass().getResourceAsStream("my-logging.properties");
  LogManager.getLogManager().readConfiguration(is);

You can copy the logging.properties file from the JRE to your favorite
location
and modify it accordingly to your needs. Obviously you can load the file
from wherever
you like, not only from the classpath.

I'm using it with Tomcat 5.0.x inside JBoss 4.0.1 and it works well.
I don't know if other container configurations may prevent you
to call readConfiguration() for some security check or other reasons...

Bye
Paolo Rizzi

-----Messaggio originale-----
Da: Chris Holmes [mailto:cholmes@anonymised.com]
Inviato: venerdì 3 giugno 2005 17.16
A: Artie Konin
Cc: geoserver-devel@lists.sourceforge.net
Oggetto: Re: Re[2]: [Geoserver-devel] GeoServer logging
subsystem - how
to configure?

Hey Artie, just start sending these to geoserver-devel,
instead of just
me, ok? The list is to talk exactly about stuff like this, and if it
just goes to me it may get lost in my inbox.

Chris

Quoting Artie Konin <a-thor@anonymised.com>:

> Greetings, Chris,
>
> Friday, May 27, 2005, 1:58:18 PM, you wrote:
>
> > If you just comment out the line that starts the
log4jformater then
> it
> > should pick up your jvm levels, in the logging.properties file.
> You can
> > specify package levels and whatnot there. Perhaps we
should have a
> config
> > option to completely turn f the log4j formatter logging. The
> thought
> > behind it was to allow users to easily configure through the web
> app, but
> > it is at the price of more fine grained control for programmers.
>
> Ok, I was experimenting with different variants for a while, so
> that's why I didn't responded earlier :slight_smile: The way you suggested is
> probably the simpliest one, and it works. Hovewer, the output looks
> awfully: that two lines per message, full stack traces in most
> harmless situations and so on. Also as there is one
> java.util.loggging.LogManager instance per JVM, changing
> configuration
> file in the JRE could affect other applications running on the same
> container (as well as container itself) badly. In fact, I
> inadvertently
> blanked the whole tomcat log output two or three times trying to
> configure the loggers in the way I wanted (and I should
admit I still
> wasn't able to figure how to use my custom Formatter with this
> logging
> system). Not very satisfactory results indeed :slight_smile:
>
> > I do sort of want to use commons logging or some such, I'm not in
> love
> > with java logging...
>
> Nether do I :slight_smile: I've tried some variants of using another logging
> packages with GeoServer (not many, though - in fact there were only
> two of them - Log4j and Jakarta Commons Logging aka JCL)
and here are
> the results:
>
> 1) JCL by itself is probably not a sufficient replacement of JDK
> logging, as it is merely a wrapper over another logging packages.
> including JDK logging itself. Of course, that is if one doesn't want
> to use bundled SimpleLog class - in latter case JCL can work as
> standalone logging implementation, but capabilities of SimpleLog are
> quite limited. So JCL should be paired with another
(richier) logging
> implementation (and, probably, not that from Sun :slight_smile:
>
> 2) First I thought that as JCL is a wrapper it should provide
> seamless
> switching from one underlying implementation to another without
> rewriting the code. But it turned out that this is not the case, at
> least with GeoServer. The problem is that
> org.apache.commons.logging.Log interface which is implemented by all
> implementation wrapper classes (as well as by their own SimpleLog
> class) exposes very restricted set of methods. There are six logger
> methods: trace(), debug(), info(), warn(), error(), fatal(), each in
> two forms (one and two arguments, first is the message second is
> Throwable) and six corresponding code guards: isTraceEnabled(),
> isDebugEnabled(), ... etc. That's all :slight_smile: So your logging
code will be
> implementation-independent only if it is all about publishing log
> messages. If you want, for example, obtain logging level of specific
> logger, you will need to cast it to the implementation-specific
> wrapper class (org.apache.commons.logging.impl.Log4JLogger,
> org.apache.commons.logging.impl.Jdk14Logger etc.), get underlying
> logger with getLogger() method and then use its native methods to
> obtain the level. At this point you are stuck with specific
> implementation and wouldn't be able to change it without modifying
> the
> code :\ And this is exactly GeoServer situation: while 99% of its
> logging-related code could be switched to JCL with primitive search
> and replace operations, the remaining 1% is a tough spot :slight_smile: It will
> be implementation-dependent in any case, whether we use JCL or
> working
> directly with that implementation. This makes the usefulness of JCL
> quite questionable. (Though, I personally use JCL in my client and
> totally happy with its SimpleLog :slight_smile:
>
> 3) The only "real" third-party logging package I tried is Jakarta
> Log4j. It seems that it is quite common choice of Java developers.
> It key benefits are:
> a) GeoServer already has log4j jar in its /lib (though I
wonder where
> it is used?) - no need to add another bunch of jars, as
with Avalon
> Logkit etc.
> b) It is excitingly easy to configure comparing to JDK logging. In
> most cases a single file log4j.properties placed in /classes
> directory
> will do the job, without any programmatic initialization.
> c) One of the most important things is that log4j can (and
should) be
> configured at webapp level (that's really great, no that nasty
> interference and external configuration files as with Sun's
> logging).
> d) It is seamlessly integrates with JCL. In fact, it is possible to
> omit any explicit JCL configuration at all with log4j. I just
> placed
> commons-logging.jar in /lib together with log4j.jar and
was able to
> use JCL logging based on log4j configuration I supplied.
> e) There is a very convenient .log4j.threshold parameter in
> configuration file which sets the global logging level, which is
> applied regardless of individual loggers settings. So it is only a
> matter of changing single line in .properties file to apply
> "global"
> logging level. Of course, threshold may be also set
> programmatically
> (though right now I cannot say how to do this exactly, there are
> methods setThreshold() in LoggerRepository interface and
> implementing Hierarchy class, which seems may be retrieved with
> Logger.getLoggerRepository() method).
> f) All later versions of Log4j have similar syntax to that of JDK
> logging, for example code for obtaining the logger is exactly the
> same:
> private static final Logger LOGGER =
> Logger.getLogger("org.vfny.geoserver.global");
> The only difference that Logger class is org.apache.log4j.Logger
> and
> not java.util.logging.Logger :slight_smile:
> g) Log4j's Logger has that nice convenient level-tailored code
> guards:
> isDebugEnabled(), isErrorEnabled() etc. Writing
> if (log.isDebugEnabled()) {...}
> to me is much much simplier than
> if (LOGGER.isLoggable(Level.FINE)) {...}
> Latter even seems clumsier and often make a difference between
> guarded and unguarded logging statement :slight_smile:
>
> Well, there are some more of them, but they are less
significant, so
> now disadvantages that I see:
>
> a) The code needs to be adopted. That itself is bad enough :slight_smile:
> b) Logging levels has different names so it will need some time to
> get
> used to them (these are DEBUG, INFO, WARN, ERROR, FATAL).
> Moreover,
> they are not mapped too well with JDK's levels.
> c) There are only five different log levels in Log4j. The result is
> that
> if we use usual intuitive JDK->log4j log level mapping:
> SEVERE - ERROR
> WARNING - WARN
> INFO - INFO
> CONFIG - INFO
> FINE - DEBUG
> FINER - DEBUG
> FINEST - DEBUG
> we end up with three different debugging levels of
> java.util.logging mapped to a single DEBUG log4j level. That is
> definitely not good. The "recommended" solution here is to
> subclass org.apache.log4j.Logger and to define custom levels
> there.
> But in real life, using of such a custom levels is very
> inconvenient :\ For my testing, I avoided this problem
by patching
> log4j sources - somewhat ugly, but it works :slight_smile:
> d) The complete log4j manual is commercial. But it is as well
> possible to learn almost everything about the framework using
> only short manual, api docs and source code.
> e) Well, in any case, log4j isn't shipped with JDK and therefore not
> such ubiquitous (not a real problem, but for someone this may
> prove
> to be critical :slight_smile:
>
> 4) So, basing on the all above there are four possible variants I
> see:
>
> I) Do nothing :slight_smile: In the end, if JDK logging was good for all
> GeoServer developers for years, why break the habit and switch
> to
> another implementation
> II) Switch to JCL with log4j as underlying implementation. In this
> case modifying existing code would be somewhat harder, but the
> result is somewhat more "universal".
> III) Switch to log4j directly. Less pain (though mostly in
> automated search/replace part of the work) and might
be slightly
> faster. Why introduce another abstraction layer (JCL)
if part of
> the code will work with log4j directly?
> IV) Investigate some more logging frameworks (like Avalon Logkit)
> and possibly choose one of those :slight_smile:
>
> Personally I chose the second variant for my tests as it
seems to be
> the most complex of those :slight_smile: And, surprisingly, now I have an
> instance
> of GeoServer here up and running which uses exactly that logging
> engine, though this was achieved not without a number of ugly hacks.
> Here is a piece of tomcat log (long lines stripped):
>
> 3422 [TRACE] org.vfny.geoserver.global - FeatureType
> t_koos_shp:::t_koos_shp looking up :t_koos_shp
> 3422 [TRACE] org.vfny.geoserver.global - in string url
> 3422 [TRACE] org.vfny.geoserver.global - t_koos_shp:::t_koos_shp
> datastore found :DataStoreConfig[namespace=tambov, enabled=true, ...
> 3422 [TRACE] org.vfny.geoserver.global - in string url
> 4172 [DEBUG] org.geotools.data.arcsde -
> com.esri.sde.sdk.client.SeConnection is in place.
> 4328 [DEBUG] org.geotools.data.arcsde -
> com.esri.sde.sdk.pe.PeCoordinateSystem is in place.
> 4516 [WARN ] org.geotools.data.mysql.MySQLDataStoreFactory - Failed
> on : dbtype
> 4516 [DEBUG] org.geotools.data.mysql.MySQLDataStoreFactory -
> {namespace=http://www.tambov.ru/geoserver, ....
> 4610 [WARN ] org.geotools.data - Could not aquire Vector Product
> Format Library data store
> implementation.:java.io.FileNotFoundException: ...
> java.io.FileNotFoundException: Could not find file:
>
C:\tomcat\webapps\geoserver\data\featureTypes\t_koos_shp\t_koo
s_shp.shp\lht
> at
>
org.geotools.data.vpf.file.VPFFileStore.findFile(VPFFileStore.
java:178)
> at
>
org.geotools.data.vpf.file.VPFFileStore.getSchema(VPFFileStore
.java:88)
> at
>
org.geotools.data.vpf.file.VPFFileFactory.getFile(VPFFileFacto
ry.java:134)
> at org.geotools.data.vpf.VPFLibrary.<init>(VPFLibrary.java:167)
> at
>
org.geotools.data.vpf.VPFDataStoreFactory.create(VPFDataStoreF
actory.java:122)
>
> Note that stack trace. It seems that Log4JFormatter
currently simply
> ignores any throwables passed to it, so in normal GeoServer log this
> trace is absent. Of course, I could do this as well in the much the
> same way, but I believe that it is better change logger call in
> related geotools code section. After all, if you don't want to see
> the
> trace, then you don't pass it to the logger, and if you pass the
> throwable, then you want to see it, right?
>
> Now what I have done in order to make the things working:
> 1. Wrote simple PHP script that is traversing through GeoServer
> source tree
> and makes replacements that could be done more or less safely in
> such way. It shouldn't be too hard to write its Java analog (I
> simply was in haste, and writing such things in PHP or other
> scripting language is usually much faster). Mostly that
is changes
> of import statements, log instantiation, logger calls and
> codeguards.
> 2. Hack #1. Changed sources of log4j to add sixth log level - TRACE.
> As a positive moment now JCL and log4j levels are mapped
> one-to-one
> :slight_smile: As negative - it will be somewhat harder to upgrade to a new
> version, though it taken only a few minutes - not too much.
> BTW, in this case JCL can shield the code from being overrun by
> "non-standard" log4j calls :slight_smile: Later on both log4j and commons
> logging can be replaced with their standard variants and the code
> will not be broken (I hope).
> 3. Hack #2. Correspondingly "fixed" commons logging, so their
> Log4JLogger wrapper class now knows about that extra level.
> 4. Hack #3. Changed Log4JFormatter class. Added a new handler which
> redirects all geotools log messages to corresponding JCL loggers.
> This is perhaps the most sloppy code, but it allows to control
> geotools output from the same config file as geoserver's one.
> Anyway it can be accounted for later, if needed.
>
> I believe that something similar can be done for "pure" log4j
> installation. What is the most valuable - now I (at last) have all
> logging, including logger levels and output fomatting,
configured in
> a single configuration file - log4j.properties, which gives me
> almost
> absolute control over the process (well, in fact, I'd
change log4j a
> little bit more, as it seems to not allow specifiyng appender's
> properties in config file, the things such as output
encoding should
> be
> still configured programmatically - but that's only if
there will be
> some use of it, don't want to do fool's work :slight_smile:
>
> Ok, it is now up to you and other GeoServer developers to decide
> whether and when to change the logging subsystem. I just made sure
> it is
> practically possible and even not too hard to accomplish :slight_smile:

Cool. This is actually annoying, hearing the log4j is in
fact better.
We were originally log4j - the reason we have the log4j formatter is
becuase RobH liked how it out put things better. The java logging was
kind of imposed on us, since GeoTools switched, and we didn't have any
real good reasons not to switch. I think if we switched back it might
make sense to use the commons logging, so we could pick up the java
logging stuff from geotools. Oh yeah, we should get rid of the log4j
jar, it's there because some libraries that we used to use make use of
it. But we don't really support them any more (z39.50 stuff).

The other capability I'd like to have if/when we switch logging is to
have it write to a database. Particularly access information, as
admins would be interested in such things. I'd like logging that is
more geared to users, the types of things they want to see, not just
the developers debug statements.

>
> --
> WBR,
> Artie
>

----------------------------------------------------------
This mail sent through IMP: https://webmail.limegroup.com/

-------------------------------------------------------
This SF.Net email is sponsored by Yahoo.
Introducing Yahoo! Search Developer Network - Create apps using Yahoo!
Search APIs Find out how you can build Yahoo! directly into your own
Applications - visit
http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005
_______________________________________________
Geoserver-devel mailing list
Geoserver-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geoserver-devel

Greetings, Paolo

Monday, June 6, 2005, 1:17:38 PM, you wrote:

You can try loading a separate configuration file for the Java Logging
system, like this:
     InputStream is =
getClass().getResourceAsStream("my-logging.properties");
     LogManager.getLogManager().readConfiguration(is);

You can copy the logging.properties file from the JRE to your favorite
location and modify it accordingly to your needs. Obviously you can load
the file from wherever you like, not only from the classpath.

Yes, I already tried this approach. And, as in your case, it
basically works... But I can't say I'm to happy with the *way* it works.

First of all, I found impossible to control logger configuration from
the config file during application runtime. Simple re-reading it
doesn't affect existing loggers in any way, so it is quite impossible
to play with loggers' levels and apply the changes without restarting
the whole bloody application container (well, I guess it *might* be
possible, but I can't figure how). It shouldn't be too hard to write
some kind of watchdog, that will periodically check modification time
of the certain file, then load/parse it and change logger's parameters
programmatically when needed. But that looks like fool's work to me, as
log4j allows to do such things in a few simlple gestures.

  Again, it is obvious that JDK logging is designed to be controlled
programmatically, rather than externally via config files. That leads
to some more minor inconveniences besides the one I mentioned earlier.

  The second "interesting" issue with JDK logging is the fact that
"java.util.logging.LogManager" class is in "java" namespace and,
consequently, is usually loaded by bootstrap/system classloader, which
in turn means that in normal circumstances there is only one instance
of LogManager and one logger repository per JVM. This repository can
(and usually is) be shared among multiple web applications as well as
container itself. And special care must be taken when reloading
log configuration inside each particular webapp. For example, you
may no longer count on the repository-wide threshold (.level param in
config file), as it can be as well modified by another application
just after you load your configuration. The same applies to global
handlers. So you will have to fully set up at least the root logger(s)
of your own hierarchy. That is tolerable but still somewhat limiting.

BTW, I just found that with the default Tomcat configuration it is
pretty easy for you webapp to mess with loggers, defined in other
webapps as well as Tomcat's internal ones. And, of course, vice versa.
You never can't be sure of where you logging output is going, because
other webapp can attach any number of "rogue" handlers to your loggers
and capture your output. As a quick exercise, I was able to
completely shutdown logging of Tomcat itself and a number of its
applications by executing the following code inside a test webapp:

Enumeration<String> list = LogManager.getLogManager().getLoggerNames();
while (list.hasMoreElements()) {
     String name = list.nextElement();
     Logger.getLogger(name).setLevel(Level.OFF);
}

Of course, that is usually not a real problem, as at my own machine I
have full control over the container and web applications. But, even
knowing all the above, I accidentally messed the whole container logging
system several times after certain experiments with JDK logging
configuration file :slight_smile:

From the other hand, log4j possess none of the above issues and can
be configured and controlled both in programmatic fashion and via
config files. Besides, it is much faster, then native JDK logging
framework. Though, I admit that all this is a matter of personal
taste. May be I'm just too lazy :wink: And simplier ways are known to
deliver less experience than tougher ones :slight_smile: Anyway Java native
logging is not so bad, especially if you got iron nerves and angelic
patience :slight_smile:

--
WBR,
Artie