[GRASS-dev] GRASS 6.3.0

William Kyngesburye wrote:

> Another problem with vdigit: the C++ wrapper file
> grass6_wxvdigit_wrap.cpp
> is *not* a source file (it is built from the .i files using SWIG), and
> should not be stored in the SVN repository. It should be generated
> by SWIG
> during the build process.

Correct or not, I'vee seen it as a convention in many projects to
include pre-SWIG'd sources (ie Mapserver, Qgis, GDAL). While
developers may be OK with adding SWIG to their toolbox (though I
haven't installed it on my new Mac because I don't need it any more),
users who build from source may not have SWIG, and may not want to
install something that's not used at runtime but only for compilation.

Even if we include the file as a convenience, the Makefile should
still have the appropriate dependency information so that any changes
to the .i files are handled correctly.

--
Glynn Clements <glynn@gclements.plus.com>

Glynn Clements wrote:

> > Another problem with vdigit: the C++ wrapper file
> > grass6_wxvdigit_wrap.cpp
> > is *not* a source file (it is built from the .i files using SWIG), and
> > should not be stored in the SVN repository. It should be generated
> > by SWIG
> > during the build process.
>
> Correct or not, I'vee seen it as a convention in many projects to
> include pre-SWIG'd sources (ie Mapserver, Qgis, GDAL). While
> developers may be OK with adding SWIG to their toolbox (though I
> haven't installed it on my new Mac because I don't need it any more),
> users who build from source may not have SWIG, and may not want to
> install something that's not used at runtime but only for compilation.

Even if we include the file as a convenience, the Makefile should
still have the appropriate dependency information so that any changes
to the .i files are handled correctly.

However, this is complicated by the fact that grass6_wxvdigit.i isn't
stored in SVN, so that file gets generated, then
grass6_wxvdigit_wrap.cpp gets regenerated from that, which may make it
out of sync with the SVN version. Also, there's no guarantee that the
local timestamps (which indicate when the files were checked out)
accurately reflect the relationships between the files.

This is one reason why you don't normally keep intermediate files in a
version-control repository. E.g. we don't keep the lex/yacc-generated
files in the repository, so users need those tools to compile.

Another reason is that developers need to sync and commit the
pre-generated versions whenever they edit the underlying .i and .h
files. Also, we'll get a lot of noise in the diffs if different
developers are using different versions of SWIG (I have 1.3.31, but
the SVN version was built with 1.3.33).

Some projects keep the configure script under version control,
although that too is technically not a source file. But the situation
for the configure script is slightly different, as:

a) Neither configure nor configure.in are mentioned in the Makefiles.

b) You can't use the Makefiles until you've run configure, which
creates a chicken-and-egg situation. E.g. you can't have the top-level
Makefile generate configure from configure.in, you can't use the
configure script to detect how to run autoconf to generate the
configure script, etc.

For now, I've removed the intermediate files, so you need SWIG to
build the vdigit module from source.

Ultimately, the SVN repository exists for development. It's likely
that the inconvenience to developers is only going to increase as the
wx GUI becomes more important (e.g. as existing XDRIVER-based programs
disappear in 7.x).

If the SWIG issues are considered problematic for end users (who build
from source), it may be worth considering including the SWIG-generated
files in source tarballs.

Note that you already need SWIG if you want to build the wrappers for
the GRASS libraries (i.e. if you want to call GRASS functions directly
from Python). I would expect this to be an issue sooner or later (I
had assumed that vdigit was already doing this, but it turns out that
only the C part calls GRASS directly).

--
Glynn Clements <glynn@gclements.plus.com>

Understood, that intermediate files need to rebuilt as developers make changes.

But we should try to include them in releases. That was my main point. Hopefully it's something that could be automated.

On Mar 22, 2008, at 2:25 AM, Glynn Clements wrote:

Even if we include the file as a convenience, the Makefile should
still have the appropriate dependency information so that any changes
to the .i files are handled correctly.

However, this is complicated by the fact that grass6_wxvdigit.i isn't
stored in SVN, so that file gets generated, then
grass6_wxvdigit_wrap.cpp gets regenerated from that, which may make it
out of sync with the SVN version. Also, there's no guarantee that the
local timestamps (which indicate when the files were checked out)
accurately reflect the relationships between the files.

...

If the SWIG issues are considered problematic for end users (who build
from source), it may be worth considering including the SWIG-generated
files in source tarballs.

Note that you already need SWIG if you want to build the wrappers for
the GRASS libraries (i.e. if you want to call GRASS functions directly
from Python). I would expect this to be an issue sooner or later (I
had assumed that vdigit was already doing this, but it turns out that
only the C part calls GRASS directly).

--
Glynn Clements <glynn@gclements.plus.com>

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

"Oh, look, I seem to have fallen down a deep, dark hole. Now what does that remind me of? Ah, yes - life."

- Marvin

William:

users who build from source may not have SWIG, and may not want to
install something that's not used at runtime but only for compilation.

this sounds like a no-brainer to me. If a user wants to build from
source, then they need to install the required -dev tools. end of story.

otherwise you end up with a perpetually out-of-date situation which only
helps to both slow down development and break implementations; no point
keeping old cruft around. ./configure is our exception...

swig/python/interfaces/*.i needs cleaning up too, see the Makefile.
(that dir is partially auto-generated from grass/include/?)

back,
Hamish

      ____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Hamish wrote:

William:

users who build from source may not have SWIG, and may not want to install something that's not used at runtime but only for compilation.

this sounds like a no-brainer to me. If a user wants to build from
source, then they need to install the required -dev tools. end of story.

Hamish,

I disagree that it is a no-brainer. It is a reasonable strategy to
decide to commit some intermediate files (such as bison, flex, autoconf,
and swig output) in order to make it less complicated for developers to
build from source.

It can be very challenging for new developers to get just the right
versions of all these things for one package, much less many. For
instance, GDAL only works for a very narrow band of swig versions.
Other packages might only work with incompatible versions. If most
developers aren't actually changing the swig generated files, while
force them to wrestle with versions for all these tools?

Well, the obvious reason, as you suggest is that keeping intermediate
files in source control can result out of date versions being used
as well as "source control churn".

All I would ask is that the project make a deliberate decision on
this, not just assume the answer is cut and dried.

I, for one, have been driven away from contribution to some packages
exactly because I could never seem to get and keep just the right
versions of various things. I find this especially annoying with
autoconf.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGeo, http://osgeo.org

Frank Warmerdam wrote:

>> users who build from source may not have SWIG, and may not want to
>> install something that's not used at runtime but only for compilation.
>
> this sounds like a no-brainer to me. If a user wants to build from
> source, then they need to install the required -dev tools. end of story.

Hamish,

I disagree that it is a no-brainer. It is a reasonable strategy to
decide to commit some intermediate files (such as bison, flex, autoconf,
and swig output) in order to make it less complicated for developers to
build from source.

Committing intermediate files makes it more less complicated for end
users to build from source, but *more* complicated for *developers*.

If you just check out the source, then configure, make, make install,
having the generated files reduces the number of tools required.
However, if you want actually change any of the source files from
which the intermediate files are generated, then having the
intermediate files in SVN complicates matters, e.g. requiring you to
commit the intermediate files whenever the source files are changed.

[You also need to commit the complete chain of intermediate files if
you want to write Makefile rules which actually handle both cases.]

This causes problems if someone forgets to commit the intermediate
files, and can add a lot of noise to the change history if multiple
developers have even slightly different versions of the tools.

There's also the issue that most version control systems set local
timestamps according to checkout/revert/etc, not commit.

It can be very challenging for new developers to get just the right
versions of all these things for one package, much less many. For
instance, GDAL only works for a very narrow band of swig versions.
Other packages might only work with incompatible versions. If most
developers aren't actually changing the swig generated files, while
force them to wrestle with versions for all these tools?

Well, the obvious reason, as you suggest is that keeping intermediate
files in source control can result out of date versions being used
as well as "source control churn".

All I would ask is that the project make a deliberate decision on
this, not just assume the answer is cut and dried.

My view is that the SVN repository exists for *developers*, who will
modify source files, commit changes, perform incremental compilation,
etc. From that perspective, keeping intermediate files in the
repository is a net loss.

Keeping intermediate files in a source tarball is a different matter.
We can assume either that the user won't make changes, or that changes
will be sufficiently uncommon that requiring recompilation from
scratch (make distclean) isn't a problem. Also, any changes won't be
committed directly to SVN.

I, for one, have been driven away from contribution to some packages
exactly because I could never seem to get and keep just the right
versions of various things. I find this especially annoying with
autoconf.

I understand that problem quite well. Generally, I'll try to avoid
unnecessary depdencies altogether, but that isn't a practical option
for SWIG. We don't use automake etc (which tend to be much worse than
autoconf for version issues). By sticking with autoconf 2.13, we've
avoided the frequent updates to the later autoconf versions (Gentoo
has two separate slots for autoconf, one for 2.13 and one for all of
the newer versions).

Also, any barriers to modifying the configure script aren't
necessarily a bad thing. If someone breaks the configure script, they
break the whole of GRASS, so it's not entirely a bad thing if
developers (particularly those who don't do a lot of autoconf) need to
post configure.in diffs to the list rather than just commiting the
changes themselves.

--
Glynn Clements <glynn@gclements.plus.com>

Hi,

2008/3/14, Martin Landa <landa.martin@gmail.com>:

BTW I do not fully understood if there was consensus to create
directly releasebranch_6_4 in SVN or first to create develbranch_6 and
then releasebranch_6_4 based on develbranch_6. In any case it would be
great to start grass7 development finally:-)

I am sorry to repeat myself here...

I really don't see *any* reason why to wait with issue. Time is
passing;-) Technically there is no problem to create a new branch for
grass64. What I am not sure if to create releasebranch_6_4 directly.

1)

* create releasebranch_6_4

2)

* create develbranch_6
-> After some time create releasebranch_6_4 based on develbranch_6.

Trunk for grass7. Just to take decision and go ahead in the development:-)

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa *

Martin Landa wrote:

> BTW I do not fully understood if there was consensus to create
> directly releasebranch_6_4 in SVN or first to create develbranch_6 and
> then releasebranch_6_4 based on develbranch_6. In any case it would be
> great to start grass7 development finally:-)

I am sorry to repeat myself here...

I really don't see *any* reason why to wait with issue. Time is
passing;-) Technically there is no problem to create a new branch for
grass64. What I am not sure if to create releasebranch_6_4 directly.

1)

* create releasebranch_6_4

2)

* create develbranch_6
-> After some time create releasebranch_6_4 based on develbranch_6.

I would suggest the latter.

--
Glynn Clements <glynn@gclements.plus.com>

Hi,

FWIW I think we should wait a few more days until 6.3.0 is released
before splitting off GRASS 6/7 in the SVN. There may be no technical
reason not to do it now, but I think we should avoid any new confusion
and distractions until the current release is shipped. That and it adds
motivation for release and creates a sense of punctuated drama around the
release, which helps get it advertised. We could mention this in the
6.3.0 release announcement? (I'm meaning to work on that soon..)

very soon,
Hamish

      ____________________________________________________________________________________
Never miss a thing. Make Yahoo your home page.
http://www.yahoo.com/r/hs

On Wed, 26 Mar 2008, Glynn Clements wrote:

Martin Landa wrote:

BTW I do not fully understood if there was consensus to create
directly releasebranch_6_4 in SVN or first to create develbranch_6 and
then releasebranch_6_4 based on develbranch_6. In any case it would be
great to start grass7 development finally:-)

I am sorry to repeat myself here...

I really don't see *any* reason why to wait with issue. Time is
passing;-) Technically there is no problem to create a new branch for
grass64. What I am not sure if to create releasebranch_6_4 directly.

1)

* create releasebranch_6_4

2)

* create develbranch_6
-> After some time create releasebranch_6_4 based on develbranch_6.

I would suggest the latter.

I definitely agree. Also I think we're over-complicating things with release branches a bit at the minute. I feel they're only necessary for a stable branch that will need (in the future) to have bugfixes and nothing else backported to it. 6.3.0 isn't supposed to be a stable release (although it's starting to look like we're treating it as such) and IMHO it would have been simpler just to release it from HEAD (or develbranch_6 if it had been created first).

But this is all looking good. I can see develbranch_6 becoming one of the most stable and reliable versions of GRASS ever.

Paul