[GRASS5] GRASS 5.1 with shared libs problems

GRASS GRASS developer
1

Hi,

trying to compile GRASS 5.1 (today's CVS version) with shared libs fails
for a few modules. Maybe some Makefile experts could look into this?
(Please run 'make clean' after 'cvs up -dP' due to updated links to 5.0
source code)

To activate shared libs, change two files:

include/Make/Lib.make
-> at bottom: comment stlib, uncomment shlib

include/Make/Grass.make
-> at top: switch LIB_PREFIX and LIB_SUFFIX settings

I also tried to apply the LIBRULE line for shared libs as found in 5.0's
mk/mid.mk
without success.

Thanks in advance for help,

Markus

2

Radim Blazek wrote:

> AFAICT, Radim developed the 5.1 build system, so it might be better to
> ask him. It isn't that I'm unwilling to do anything in this area, just
> that anything which is done needs to be coordinated between the
> relevant parties.

I would not say that I have developed 5.1 build system, I just wanted
to work on grass 51, so I wrote something which enables that. I expected
that somebody who knows more about that, could improve/rewrite it, see:
http://grass.itc.it/pipermail/grass5/2002-January/001533.html

OK. For future development, there are three main options:

1. Work with the Makefiles which are already there.
2. Port the alternate build mechanism (the one in "mk") from 5.0.
3. Write a conventional build mechanism which supports out-of-place builds.

Option 1 has the disadvantage of not supporting out-of-place builds.

If you just want to compile and install GRASS, this isn't a problem.
However, it is useful for development (e.g. allowing you to keep
multiple build trees around), or for large sites (where you may wish
to build GRASS on multiple systems from a single shared source tree).

Option 2 is a bit of a hack, as the system was constrained by the need
to cooperate with the existing gmake5 mechanism. Its main problems
are:

a) the need to pass the paths to the source and build "mk" directories
to "make" when building individual subdirectories, and

b) uncertainty as to whether the [ab]use of yacc's "-b" switch (to put
the output file in another directory) is portable.

Option 3 is "the right thing", but it's also the most work.

With a conventional out-of-place build mechanism, "make" operates with
the current directory set to the build directory. Source files are
located according to make's VPATH setting, while output files are
placed in the current directory. The Makefiles live in the build
directory; these are generated from the corresponding Makefile.in
files by the configure script.

[Note: autoconf supports some of this automatically; e.g.
AC_OUTPUT(foo/bar) reads $srcdir/foo/bar.in ($srcdir is determined
from the pathname of the configure script or the --srcdir option) and
writes foo/bar relative to the current directory.]

The main complications with this are:

1. The configure script needs to obtain the list of Makefiles to be
generated (although this can be automated; you can use shell variables
in the argument to AC_OUTPUT).

2. Having configure generate lots of Makefiles via AC_OUTPUT is
potentially slow. As the actual Makefiles only need two specific
substitutions (to insert the source and build directories), it may be
better to perform this part with a dedicated script similar to the
existing mk/mkmakefiles. OTOH, Mozilla doesn't feel the need to do
this (it just passes ~1000 filenames to AC_OUTPUT).

3. Local header files won't be in the current directory, so you have
to add an explicit "-I$(SRCDIR)/$(CURDIR)" to the preprocessor flags
(and each Makefile must define CURDIR to be the relative path to the
directory from the top of the source tree).

--
Glynn Clements <glynn.clements@virgin.net>

3

On Wednesday 06 November 2002 08:14 pm, Glynn Clements wrote:

Option 1 has the disadvantage of not supporting out-of-place builds.

If you just want to compile and install GRASS, this isn't a problem.
However, it is useful for development (e.g. allowing you to keep
multiple build trees around), or for large sites (where you may wish
to build GRASS on multiple systems from a single shared source tree).

But compile more platforms in the same source tree at the same time
each in its $(ARCH) dirs should be possible?

Do you mean multiple systems with identical $(ARCH) but other differences
like versions of libraries?

Radim

4

Radim Blazek wrote:

> Option 1 has the disadvantage of not supporting out-of-place builds.
>
> If you just want to compile and install GRASS, this isn't a problem.
> However, it is useful for development (e.g. allowing you to keep
> multiple build trees around), or for large sites (where you may wish
> to build GRASS on multiple systems from a single shared source tree).

But compile more platforms in the same source tree at the same time
each in its $(ARCH) dirs should be possible?

Do you mean multiple systems with identical $(ARCH) but other differences
like versions of libraries?

There are three distinct issues which the existing (gmake5) mechanism
doesn't solve:

1. Multiple builds for the same value of $(ARCH). Either multiple
builds for the same target but with different options, or builds for
multiple targets which happen to have the same value of $(ARCH).

2. Building from a read-only source tree, e.g. a read-only NFS export
(or a writeable NFS export, but one where you don't want to have to
give write permission to everyone who might wish to build GRASS;
compiling the code and modifying the code are very distinct
operations, so the permissions should be orthogonal).

3. The fact that the existing mechanism doesn't quite work; while
object files are kept separate for different $(ARCH) settings, there
are still conflicts for Makefile, config.h, and all occurrences of
lex.yy.c, y.tab.[ch] etc.

Issue 1 could be solved by simply allowing the user to choose the
suffix. Solving issue 3 would create similar problems to those which
already apply to the "mk" mechanism.

In any case, the existing mechanism already has many of the same
problems as the "mk" mechanism; it just hides them within the gmake5
and GISGEN.sh scripts.

Both mechanisms operate with the source directory as the current
directory, and explicitly place generated files in a separate
directory.

However, the "mk" mechanism is more thorough (it handles all generated
files, not just object files), it solves issue 2 above, and the
overall mechanism is a lot simpler than gmake5 + GISGEN.sh + src/CMD/*.

The main advantages of option 3 (a conventional out-of-place build
mechanism) are that:

a) the build tree contains a complete set of specialised Makefiles, so
you can use "make -C <any directory> ..." without having to provide
any "-I" flags, and

b) GNU make has some built-in support for working this way (e.g.
vpath), so the Makefiles are cleaner (e.g. no need to explicitly use
$(OBJDIR) everywhere).

--
Glynn Clements <glynn.clements@virgin.net>

5

On Glynn Clements wrote:

OK. For future development, there are three main options:

1. Work with the Makefiles which are already there.
2. Port the alternate build mechanism (the one in "mk") from 5.0.
3. Write a conventional build mechanism which supports out-of-place
   builds.

Option 3 is "the right thing", but it's also the most work.

Considering also your explanation in another mail, I think that it
is "the right time" to do that now.

2. Having configure generate lots of Makefiles via AC_OUTPUT is
potentially slow. As the actual Makefiles only need two specific
substitutions (to insert the source and build directories), it may be
better to perform this part with a dedicated script similar to the
existing mk/mkmakefiles. OTOH, Mozilla doesn't feel the need to do
this (it just passes ~1000 filenames to AC_OUTPUT).

So we can do the same.

How to proceed now? If others agree with 3., we need somebody who can write
new build system, example Makefile.in files and some instructions how to write
Makefile.in rules. I would welcome examples especially for:
lib/db/sqlp/
lib/vector/diglib/
display/d.m/
visualization/nviz/*
/vector/v.ascii/
+ common directory, library and module

Then I can help to rewrite all other Makefiles.

Radim

6

Radim Blazek wrote:

> OK. For future development, there are three main options:
>
> 1. Work with the Makefiles which are already there.
> 2. Port the alternate build mechanism (the one in "mk") from 5.0.
> 3. Write a conventional build mechanism which supports out-of-place
> builds.

> Option 3 is "the right thing", but it's also the most work.

Considering also your explanation in another mail, I think that it
is "the right time" to do that now.

> 2. Having configure generate lots of Makefiles via AC_OUTPUT is
> potentially slow. As the actual Makefiles only need two specific
> substitutions (to insert the source and build directories), it may be
> better to perform this part with a dedicated script similar to the
> existing mk/mkmakefiles. OTOH, Mozilla doesn't feel the need to do
> this (it just passes ~1000 filenames to AC_OUTPUT).

So we can do the same.

How to proceed now? If others agree with 3., we need somebody who can write
new build system, example Makefile.in files and some instructions how to write
Makefile.in rules. I would welcome examples especially for:
lib/db/sqlp/
lib/vector/diglib/
display/d.m/
visualization/nviz/*
/vector/v.ascii/
+ common directory, library and module

Actually, I've just realised that we shouldn't actually need to
process the individual Makefiles at all; they can include the
generated .mk files using a relative path, e.g.

  CURDIR=lib/vector/diglib
  TOPDIR=../../..
  include $(TOPDIR)/mk/head.mk

  ...

  include $(TOPDIR)/mk/tail.mk

Then, only the global .mk files need to be processed; the Makefiles
can be copied verbatim. Any build-specific information can be obtained
from the included .mk files.

As a general rule, the individual Makefiles should only contain
information which is specific to that particular directory. Mostly,
this boils down to the dependencies, plus any directory-specific or
target-specific switches. As much information as possible should go
into the global .mk files (e.g. pattern rules).

The main complications are:

1. You can't easily provide a pattern rule for linking executables[1].
There are a number of common approaches to this issue; however, many
of them impose a limitation of one executable per directory, and the
ones which don't tend to be ugly.

[1] Except for the special case where an executable "foo" is made from
a single object file called "foo.o", which doesn't really help us.

2. You can't assume that there will only be one executable per
directory (e.g. i.fft/i.ifft, r.in.tiff/r.out.tiff etc); otherwise you
could just define OBJS and let the .mk files handle the rest.

Basically, the goal is to ensure that, to the greatest extent
possible, any changes to the build system only involve changing the
global .mk files and not all of the individual Makefiles. I don't mind
re-writing the core build system ad infinitum until it handles every
imaginable scenario perfectly, so long as I don't have to re-write
~1000 Makefiles every time.

I'll look into this over the next few days and get back to you. Also,
I'll replace the existing configure.in with one which is based upon
the one from 5.0.

--
Glynn Clements <glynn.clements@virgin.net>