2011/11/6 <svn_grass@osgeo.org>:
Author: hamish
Date: 2011-11-06 13:02:08 -0800 (Sun, 06 Nov 2011)
New Revision: 49122Modified:
grass/branches/develbranch_6/include/Make/Man.make
Log:
MODULE_TOPDIR refers to the source code dir, not the installed ARCH_DISTDIR. see the first line of the main Makefile.
why??? BTW, this commit breaks g.extension for installing scripts, again...
Martin
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
2011/11/6 Martin Landa <landa.martin@gmail.com>:
2011/11/6 <svn_grass@osgeo.org>:
Author: hamish
Date: 2011-11-06 13:02:08 -0800 (Sun, 06 Nov 2011)
New Revision: 49122Modified:
grass/branches/develbranch_6/include/Make/Man.make
Log:
MODULE_TOPDIR refers to the source code dir, not the installed ARCH_DISTDIR. see the first line of the main Makefile.why??? BTW, this commit breaks g.extension for installing scripts, again...
eg.
g.extension ext=g.region.point
...
GRASS_PERL=/usr/bin/perl VERSION_NUMBER=6.5.svn sh
/usr/local/src/grass6_devel/dist.i686-pc-linux-gnu/tools/g.html2man/g.html2man
/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/2593.0/dist.i686/docs/html/g.region.point.html
/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/2593.0/dist.i686/man/man1/g.region.point.1
1
sh: 0: Can't open
/usr/local/src/grass6_devel/dist.i686-pc-linux-gnu/tools/g.html2man/g.html2man
make[2]: *** [/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/2593.0/dist.i686/man/man1/g.region.point.1]
Error 127
make[2]: Leaving directory
`/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/2593.0/g.region.point'
make[1]: *** [mancmd] Error 2
make[1]: Leaving directory
`/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/2593.0/g.region.point'
make: *** [/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/2593.0/dist.i686/scripts/g.region.point]
Error 2
ERROR: Compilation failed, sorry. Please check above error messages.
Can you explain your commit a little bit more?
Martin
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
>> Author: hamish
>> Date: 2011-11-06 13:02:08 -0800 (Sun, 06 Nov 2011)
>> New Revision: 49122
>>
>> Modified:
>>
grass/branches/develbranch_6/include/Make/Man.make
>> Log:
>> MODULE_TOPDIR refers to the source code dir, not
>> the installed ARCH_DISTDIR. see the first line of the main
>> Makefile.
Martin:
> why??? BTW, this commit breaks g.extension for installing scripts,
> again...
yeah but the change before it most probably just broke *all* man page
building for a regular fresh full build(!)... (I didn't try, but it is
wrong, so "probably"..)
/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/2593.0/dist.i686/man/man1/g.region.point.1
1
sh: 0: Can't open
/usr/local/src/grass6_devel/dist.i686-pc-linux-gnu/tools/g.html2man/g.html2man
make[2]: ***
...
Can you explain your commit a little bit more?
ok, most probably I could better put in some code comments too to make it
clearer what's going on / supposed to happen--
It was all working when I fixed this all up for Ubuntu a few weeks ago; I
will recheck- maybe I had a left-over script which allowed it to work*.
But I am confident in r49122, if something is needed it is elsewhere.
(i.e. a slight adjustment in the main Makefile)
to explain:
[ see also https://trac.osgeo.org/grass/ticket/1180#comment:5 ]
commit log:
>> MODULE_TOPDIR refers to the source code dir, not
>> the installed ARCH_DISTDIR. see the first line of the main
>> Makefile.
that speaks for itself; include/Make/Man.make calls g.html2man relative
to MODULE_TOPDIR (ie source code location) not ARCH_DISTDIR (installed
build file location). the prior change had it trying to execute the
g.html2man _directory_ in the source code, not the program itself.
executing a directory doesn't work..
g.extension overrides MODULE_TOPDIR, which is why it seemed to work for
you.
in the source tree the g.html2man script is located in
tools/g.html2man/g.html2man
this is the one that SOURCE/include/Make/Man.make calls, but in the past
it was never installed in the dist.i686-pc-linux-gnu/ dir at all, so
wouldn't work for end-users with a grass-dev package installed but no
full source tree.
so as of r43748,r43749 g.html2man now gets installed into the build dir
at:
$GISBASE/tools/g.html2man
and the main Makefile edits the *install dir's* Man.make file to point it
to the altered location.
https://trac.osgeo.org/grass/changeset/43748/grass/branches/develbranch_6/Makefile
https://trac.osgeo.org/grass/changeset/43749/grass/branches/develbranch_6/Makefile
a user self-compiling outside of the source tree would run like:
src/g.addon.module$ make MODULE_TOPDIR=/path/to/grass/source/code
so in the case that it was pointed to the install dir, not the source dir,
and the $GISBASE/include/Make/ dir was installed too, it would still auto-
magically work if pointed to either source dir or install dir.
more recently this got broken by r46241, then you fixed it in r46357. (that
was 6 months ago)
summary:
source code
-----------
executable is in $src/tools/g.html2man/g.html2man
and $src/include/Make/Man.make points to above location.
installed dir ($GISBASE)
-------------
executable is in $GISBASE/tools/g.html2man
and $GISBASE/include/Make/Man.make points to above location because of
a sed hack in the main Makefile at 'make install' time*.
[*] deb/ubu pkg doesn't 'make install' so takes care of that manually
(actually there I have it installing to tools/g.html2man/g.html2man, which
would explain why it worked for my pkg build tests there in the last few
weeks)
pre-install dir ($src/dist.x86_64-unknown-linux-gnu/)
---------------
executable is in both $src/tools/g.html2man/g.html2man and
$GISBASE/tools/g.html2man
both copies of include/Make/Man.make point to tools/g.html2man/g.html2man,
usually that's ok, but because g.extension forces MODULE_TOPDIR to $GISBASE
it was looking in the wrong place. (see below)
got it? 
If g.extension can't find g.html2man, then something else has gone wrong
and will need investigation. where/when/how does it break for you?
GRASS run from the dist.x86/ dir? If so I think I see the problem:
g.extension overrides MODULE_TOPDIR there to = "$GISBASE", which has not
yet had its Man.make hacked; maybe that hack should happen at build time
(to the *dist.x86/ copy* of Man.make, not the source code copy) when the
script is installed to $GISBASE/tools/, instead of running the sed hack
at 'make install' time.. [n.b. 'sed -i' is not portable to MacOSX]
regards,
Hamish
2011/11/7 Hamish <hamish_b@yahoo.com>:
yeah but the change before it most probably just broke *all* man page
building for a regular fresh full build(!)... (I didn't try, but it is
wrong, so "probably"..)
of course I have tried that before I committed this change, nothing
was broken! BTW, similar change has been done in trunk few months ago,
nobody complained till now. As result g.extension *doesn't* work
again, as I said. No time to fix it again, I will leave it for you.
Martin
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
Hamish:
> yeah but the change before it most probably just broke *all* man page
> building for a regular fresh full build(!)... (I didn't try, but it is
> wrong, so "probably"..)
Martin:
of course I have tried that before I committed this change,
nothing was broken!
shrug. did you distclean?
$(MODULE_TOPDIR)/tools/g.html2man when MODULE_TOPDIR="." as in the case
of a regular build points to a directory not a script. can't be executed.
--unless you override the MODULE_TOPDIR setting like g.extension does.
BTW, similar change has been done in trunk few months ago,
nobody complained till now.
g.html2man.py in trunk is installed in a completely different manner
and coded to run from a different directory variable. they're not really
comparable.
As result g.extension *doesn't* work again, as I said.
only when grass was run from $source/dist.x86../. For users who had
run 'make install' or installed from a package it should have been ok.
No time to fix it again, I will leave it for you.
try r49126. (devbr6)
regards,
Hamish
Martin Landa wrote:
MODULE_TOPDIR refers to the source code dir, not the installed
ARCH_DISTDIR. see the first line of the main Makefile.
Specifically, it refers to the base directory from which the *.make
files are located. Makefiles access the *.make files via e.g.:
include $(MODULE_TOPDIR)/include/Make/Module.make
Each Makefile sets MODULE_TOPDIR itself as a relative path, but like
all make variables this can be overridden on the command line.
Unlike other directories, MODULE_TOPDIR cannot be set in *.make
because it's used to find the *.make files, so it has to be set in the
individual Makefiles or on the command line.
MODULE_TOPDIR should not be used for any other purpose. In particular,
Man.make should not be using MODULE_TOPDIR to locate g.html2man.
If g.html2man is required in order to build add-ons, then it should be
installed as part of the GRASS build process, and subsequently
referenced via $(GISBASE), as is done in 7.0.
--
Glynn Clements <glynn@gclements.plus.com>
Glynn wrote:
If g.html2man is required in order to build add-ons, then it should be
installed as part of the GRASS build process, and subsequently
referenced via $(GISBASE), as is done in 7.0.
this is the approach taken in r49126 and r49127.
hopefully everything works now, and is kosher ta boot.
Hamish
2011/11/8 Hamish <hamish_b@yahoo.com>:
Glynn wrote:
If g.html2man is required in order to build add-ons, then it should be
installed as part of the GRASS build process, and subsequently
referenced via $(GISBASE), as is done in 7.0.this is the approach taken in r49126 and r49127.
hopefully everything works now, and is kosher ta boot.
not at all
g.extension ext=i.points.auto
...
mkdir -p /home/martin/grassdata/zod2011/landamar1/.tmp/pierre/28801.0/dist.i686/man/man1
GRASS_PERL=/usr/bin/perl VERSION_NUMBER=6.5.svn sh
/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/28801.0/dist.i686/tools/g.html2man
/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/28801.0/dist.i686/docs/html/i.points.auto.html
/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/28801.0/dist.i686/man/man1/i.points.auto.1
1
sh: 0: Can't open
/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/28801.0/dist.i686/tools/g.html2man
make[2]: *** [/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/28801.0/dist.i686/man/man1/i.points.auto.1]
Error 127
make[2]: Leaving directory
`/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/28801.0/i.points.auto'
make[1]: *** [mancmd] Error 2
make[1]: Leaving directory
`/home/martin/grassdata/zod2011/landamar1/.tmp/pierre/28801.0/i.points.auto'
make: *** [cmd] Error 2
ERROR: Compilation failed, sorry. Please check above error messages.
...
g.extension still *broken*. Martin
--
Martin Landa <landa.martin gmail.com> * http://geo.fsv.cvut.cz/~landa
Hamish wrote:
> If g.html2man is required in order to build add-ons, then it should be
> installed as part of the GRASS build process, and subsequently
> referenced via $(GISBASE), as is done in 7.0.this is the approach taken in r49126 and r49127.
hopefully everything works now, and is kosher ta boot.
Man.make still uses MODULE_TOPDIR to locate g.html2man.
If it's going to use it "in place", either it should use $(GRASS_HOME)
or SRCDIR should be set in Platform.make.in. MODULE_TOPDIR isn't
intended to be an alias for SRCDIR; it's for bootstrapping, i.e. for
finding the *.make files where all of the variables are defined.
[GRASS_HOME is set to $(DSTDIR), which is the current directory when
configure is run, while $(SRCDIR) is the directory containing the
configure script. If we supported out-of-place builds, $(SRCDIR) and
$(DSTDIR) could differ.]
7.0 installs g.html2man.py to $(ARCH_DISTDIR)/tools at the start of
the build process, and the rest of the build process accesses it from
there (specifically, as $(GISBASE)/tools/g.html2man.py).
--
Glynn Clements <glynn@gclements.plus.com>
(devbr6)
Glynn:
Man.make still uses MODULE_TOPDIR to locate g.html2man.
that now gets converted to $(GISBASE) at build time for the
copy of Man.make which gets installed to $src/dist.x86..../tools/
see devbr6/Makefile and devbr6/lib/Makefile
the copy of Man.make with MODULE_TOPDIR in the source tree gets
used during standard compiles, and that works 100% fine in that
context, so I don't like to mess with it. but g.extension is
operating in another context, and that's the problematic issue.
If it's going to use it "in place", either it should use
$(GRASS_HOME) or SRCDIR should be set in Platform.make.in.
GRASS_HOME warrants some investigation. SRCDIR misses the point
of running g.extension when the source code is not present on
the end-user's computer.
[GRASS_HOME is set to $(DSTDIR), which is the current
directory when configure is run, while $(SRCDIR) is the
directory containing the configure script.
...
If we supported out-of-place builds,
$(SRCDIR) and $(DSTDIR) could differ.]
that's just the thing: g.extension is doing all these contortions
to try and support out-of-place builds (in $MAPSET/.tmp/) for
end-users who installed from a package and get their include/
grass files from a grass-dev package. they don't have the source
tree and don't necessarily have write permission* to the "real"
aka shell enviro var $GISBASE install dir.
[*] or like me don't like to install non-package-managed files
outside of /usr/local or /home, making backups and audits easier
7.0 installs g.html2man.py to $(ARCH_DISTDIR)/tools at the
start of the build process, and the rest of the build process
accesses it from there (specifically, as $(GISBASE)/tools
/g.html2man.py).
that won't help here as $(ARCH_DISTDIR) has been redirected to
$MAPSET/.tmp/, and so support files like $MAPSET/.tmp/tools/
won't exist. (Martin ran into that earlier today) thus the need
for something like $(REAL_GISBASE) to keep track of where the
real support structure lives even after the build-into dir has
be redirected to a local+temporary dir.
we're trying to eat our cake and have it too by redirecting the
build destination to one place, while retaining the rest of
the already-built stuff somewhere else. and we can almost get
away with it..
I've never really had the expectation of g.extension working
everywhere, for every module, but for unix based systems I
suspect we can realistically make it to 80%, and 90% with effort.
Hamish
Hamish wrote:
> 7.0 installs g.html2man.py to $(ARCH_DISTDIR)/tools at the
> start of the build process, and the rest of the build process
> accesses it from there (specifically, as $(GISBASE)/tools
> /g.html2man.py).that won't help here as $(ARCH_DISTDIR) has been redirected to
$MAPSET/.tmp/, and so support files like $MAPSET/.tmp/tools/
won't exist.
Right. But 7.0 doesn't access it via $(ARCH_DISTDIR)/tools, it
accesses it via $(GISBASE)/tools.
In 7.0, $(ARCH_DISTDIR) is where to put the components being built,
while $(GISBASE) is where to find the stuff that has already been
built. When building the main GRASS source tree, those will both refer
to the dist.<arch> directory, but it's intended that $(GISBASE) can
point to an installed version of GRASS if you just want to build
additional components for an existing installation.
It's also quite thorough about "installing" first then using the
installed copy via $(GISBASE), rather than accessing the source copy
directly.
The net result is that it should be possible to build almost[1] any
part of the 7.0 source tree against an installed GRASS without having
to build (or configure, or even have) any other portion of the source
tree.
[1] I can't guarantee that it's exhaustive, but it seems to handle
most cases. Headers and libraries are searched for under both
$(ARCH_DISTDIR) and $(GISBASE). Although, I've just noticed that PATH
and PYTHONPATH need similar treatment; fixed in r49159.
--
Glynn Clements <glynn@gclements.plus.com>