[GRASS5] how to minimize dependencies in GRASS binaries - with specific application to Mac OS X systems

I'm corresponding with a few list members on issues around creating binaries that are as "prerequisite free" as possible, and would like some feedback on how library dependencies work. Specifically, how to do this on a Mac OS X system, which tends to have a more restricted set of (Unix-ish) stuff installed by default than, say, a Linux system. My lack of expertise plus lack of experience (my own systems normally have the full sources and required libraries) make it hard to figure this, so I'm appealing to the list. I know that bits and pieces of the information I'm talking about are scattered throughout past mailing list traffic, but I appeal to your patience... hopefully getting this straight will lead to better information as I write up some documentation on the web pages.

Kirk Wythers and I have both made attempts at building binaries. Both of us have used the fink packaging system (essentially, a port of Debian's package tools plus a database of software ported to the Mac) to get many of GRASS' dependencies. In the end, we get a package that seems to work well if the person using it also has fink. As Michael Barton points out in the included message below, it would be nice if users did not all have to have fink installed to use the binary distribution.

We are also trying to stick to the "official" grass binary packaging system so that the end result is the same as the "official" packages provided for other platforms on the GRASS site. I have learned a lot from the generous people that have alternative binaries available now, but we'd also like to make available a system that includes installation path flexibility and no "brute-force" installation of prerequisite libraries.

In order to try to figure out the means and feasibility to make that happen, I find myself questioning my understanding of libraries and linking. I'll set out what I THINK I understand, and hope that some corrections and/or verifications would ease our task.

I think that the problem in the example depicted below was because the GNU internationalization libraries are referred to in GRASS, and on our compilation systems, it found them in the fink directories.

I will ignore for now one issue that I'm pretty sure about: this particular library is probably only needed if you were compiling in the alternate language support, which I'm pretty sure right now is only done for Russian, and that if necessary, I think we could "hide" this library from configure and GRASS would compile without internationalization support. That's all from a vague memory of past list traffic, not from research.

I think that if the compilation of this binary package had used static libraries instead of dynamic, then this problem would not have come up because the required functions would have been compiled into the GRASS binary modules. This would get rid of some of the specific issue, but would potentially make the binary distribution much bigger.

Based on these (mis?)understandings, I suppose that as long as licenses allow it, any other libraries that are missing (e.g. in this case, libintl.*.dylib) could be copied in to the binary distribution's grass library directory, the same way we now provide libgdal. As long as the number of libraries involved are reasonable, this could be the way to go.

I also know that some functions that are in standard dynamic libraries on most systems are included in "frameworks" in Mac OS X, which means that if the right flags are given at compile time, the system-provided versions can be used, and to maximize portability we would want to catch all of these to avoid packaging non-necessary extra bits. I thought that libintl might have been in the IOKit framework, but an initial search seems to suggest that this is wrong. Any corrections from Mac-savvy people most welcome.

Maybe I should stop here before I get too far ahead. How am I doing so far?

Thanks,
Scott Mitchell

P.S. Thanks again to all that have helped getting GRASS to "play nice" with Mac OS X systems in the past - in the mailing list archives that I've saved on this, Glynn Clements and Markus Neteler in particular have helped out many times...

Begin forwarded message:

On Feb 12, 2004, at 12:38 PM, Michael Barton wrote:

Scott and Kirk,

I have a student trying to install GRASS 5.3 using the binaries you and Kirk put up, packaged TclTk binaries, and NOT using fink. (I like fink a lot, but not everyone should have to install it just to use GRASS).

It turns out that these binaries have a fink dependency that needs to be removed. Here is the result of trying start GRASS 5.3 without >> fink.

[Steven-Schmichs-Computer:~] stevensc% /usr/local/bin/grass53
dyld: /usr/local/grass53/etc/lock can't open library: /sw/lib/libintl.1.dylib (
No such file or directory, errno = 2)
/usr/local/grass53/etc/Init.sh: line 145: 599 Trace/BPT trap "$ETC/lo
ck" "$lockfile" $$
Unable to properly access /Users/stevensc/.gislock5
Please notify system personel.

The student just put fink on the computer and GRASS seems to be fine.

Michael
______________________________
Michael Barton, Professor & Curator
Department of Anthropology
Arizona State University
Tempe, AZ 85287-2402
USA

------
Scott W. Mitchell Scott_Mitchell@carleton.ca
Department of Geography and Environmental Studies
Carleton University, B349 Loeb Building (Office A209)
1125 Colonel By Drive, Ottawa, ON Canada K1S 5B6
+1-613-520-2600 ext 2695 Fax: 1-613-520-4301

Scott Mitchell wrote:

I think that the problem in the example depicted below was because the
GNU internationalization libraries are referred to in GRASS, and on our
compilation systems, it found them in the fink directories.

Right; that one is a configure bug; it shouldn't be trying to detect
libintl unless you use the --with-nls switch. Note that it doesn't
actually *use* this library unless you use --with-nls, but it does get
added to the list of libraries to link against, resulting in an
unnecessary dependency.

I'll commit a fix for that.

I will ignore for now one issue that I'm pretty sure about: this
particular library is probably only needed if you were compiling in the
alternate language support, which I'm pretty sure right now is only
done for Russian,

That is correct.

I think that if the compilation of this binary package had used static
libraries instead of dynamic, then this problem would not have come up
because the required functions would have been compiled into the GRASS
binary modules.

That is also correct.

However, you can't easily specify that specific libraries should be
linked statically. You can specify that everything is linked
statically with the -static switch (at least with GNU ld; I'm not sure
about the MacOSX linker), but that is far from ideal.

The easiest way to force specific libraries to be linked statically is
to ensure that only static versions of those libraries exist on the
system.

BTW, with GNU ld, you can just put the static libraries in a separate
directory, and ensure that directory comes first in the library search
path. However, this doesn't work with the MacOSX linker. While both
linkers prefer dynamic libraries over static libraries, GNU ld gives
greater priority to the directory search path, and only uses
shared-vs-static as a "tie-breaker", whereas the MacOSX linker does it
the other way around.

This would get rid of some of the specific issue, but
would potentially make the binary distribution much bigger.

You could use static versions of "non-core" libraries which are only
used by a small number of programs (e.g. PNG, JPEG, FFTW etc) without
substantially increasing the size.

Having said that, probably the largest "bloat" issue comes from using
a static version of libgis. And if libgis was built as a shared
library, you could link that against static zlib and RPC libraries.

--
Glynn Clements <glynn.clements@virgin.net>

Scott Mitchell said:

I think that if the compilation of this binary package had used static
libraries instead of dynamic, then this problem would not have come up
because the required functions would have been compiled into the GRASS
binary modules. This would get rid of some of the specific issue, but
would potentially make the binary distribution much bigger.

Based on these (mis?)understandings, I suppose that as long as licenses
allow it, any other libraries that are missing (e.g. in this case,
libintl.*.dylib) could be copied in to the binary distribution's grass
library directory, the same way we now provide libgdal. As long as the
number of libraries involved are reasonable, this could be the way to
go.

I think that one of the arguments against including libraries into the
code is the headache this causes in terms of keeping them in a reasonable
state (up to date and bug free). Someone would have to actively follow
developments in the upstream version. Linking avoids that. This seems to
me to be a much more important reason than binary size.

Moritz