[GRASS-dev] windows qgis-grass: make grass build with vc or keep a separate qgis-grass mingw build?

Hello. The subject could rise an infinte flame, but it's just to grab
the attention!
I've digged into the windows wgi-grass world last days, and I see it's
in a stall moment.

1 - Many, in the osgeo4w dev group, is trying to make grass (built
with mingw) friend of the vc built software in the stack.
2 - Others are proposing to make a deep refactoring of grass code to
make it buildable with vc.
3 - Others (outside osgeo4w) propose to build qgis-grass with mingw
to distribute them autonomously respect to the vc stuff in osgeo4w.

Well, not a clear situation. And workforce is needed to untie it...
Could we, at least, make a roadmap?

1 - inidicate the best solution from a windows end-user point of view
2 - indicate the best solution from a developer point of view
3 - what can be done with XX money and YY time.
4 - evalauate intermediate steps to reach the optimum, but having
something useful in the meanwhile

If we reach more clearness maybe we could find contributers and
investors to boost the development.

I hope someone (users and dev) will contribute to this conversation...
Giovanni

Hi Giovanni,

On Thu, Mar 26, 2009 at 11:23 AM, G. Allegri <giohappy@gmail.com> wrote:
...

2 - Others are proposing to make a deep refactoring of grass code to
make it buildable with vc.

could you elaborate on that? What's needed, what's missing in terms
of *deep refactoring*?

Markus

could you elaborate on that? What's needed, what's missing in terms
of *deep refactoring*?

I'm not so expert to elaborate this. I've just gathered talks and
thought around MLs and chats. Someone says it shouldn't be so hard,
others say yes. I would like to know from developers (first of all
Glynn) what they think about this.

On Thu, Mar 26, 2009 at 3:08 PM, G. Allegri <giohappy@gmail.com> wrote:

could you elaborate on that? What's needed, what's missing in terms
of *deep refactoring*?

I'm not so expert to elaborate this. I've just gathered talks and
thought around MLs and chats. Someone says it shouldn't be so hard,
others say yes.

Could you please get us in contact with those saying "hard"?

Since they may have experience it would be good to get
concrete pointers where to search for the problems (since
the code base is pretty large).

Thanks
Markus

Hi Markus,

On Thu, 26. Mar 2009 at 12:51:09 +0100, Markus Neteler wrote:

On Thu, Mar 26, 2009 at 11:23 AM, G. Allegri <giohappy@gmail.com> wrote:
> 2 - Others are proposing to make a deep refactoring of grass code to
> make it buildable with vc.

could you elaborate on that? What's needed, what's missing in terms
of *deep refactoring*?

If you try to compile GRASS code with MSVC you'll run into a number of
issues. Most, if not all, are easy to fix. Nothing "deep". I did
compile some of the GRASS libraries with MSVC for QGIS at some point,
submitted patches and Glynn applied them.

But at long as we use only use C and not C++ there's just the issue of
malloc()/free() across DLL boundaries, that is an issue, if you mix
MinGW and VC built code. I'd consider those bugs - although it's
probably harmless on platforms other than Windows.

FMPOV the build system is the real issue not the codebase.

Jürgen

--
Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-20
Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50
Software Engineer D-26506 Norden http://www.norbit.de

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

Jürgen E. Fischer wrote:

FMPOV the build system is the real issue not the codebase.

The build system is a non-issue, as it doesn't affect the resulting
binaries.

Maintaining DSW/DSP files (or nmake files) isn't a realistic option,
and anything else will be "foreign" on Windows.

--
Glynn Clements <glynn@gclements.plus.com>

Hi Glynn,

On Thu, 26. Mar 2009 at 18:44:32 +0000, Glynn Clements wrote:

> FMPOV the build system is the real issue not the codebase.

The build system is a non-issue, as it doesn't affect the resulting
binaries.

Sorry, I don't see what's your point is here.

Maintaining DSW/DSP files (or nmake files) isn't a realistic option,
and anything else will be "foreign" on Windows.

cmake would work both ways. It can produce Makefiles and MSVC projects
(nmake or DSW/DSP/SLN/vcproj).

My point was simply, that if GRASS were to use cmake instead of
configure/make most part of making that happen would be on the build
system and not on the codebase itself.

Jürgen

--
Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-20
Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50
Software Engineer D-26506 Norden http://www.norbit.de

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

Jürgen E. Fischer wrote:

> > FMPOV the build system is the real issue not the codebase.

> The build system is a non-issue, as it doesn't affect the resulting
> binaries.

Sorry, I don't see what's your point is here.

My point is that you get the same binaries whether you use GNU make or
nmake, so it doesn't affect the end result, only the method by which
you get there.

> Maintaining DSW/DSP files (or nmake files) isn't a realistic option,
> and anything else will be "foreign" on Windows.

cmake would work both ways. It can produce Makefiles and MSVC projects
(nmake or DSW/DSP/SLN/vcproj).

My point was simply, that if GRASS were to use cmake instead of
configure/make most part of making that happen would be on the build
system and not on the codebase itself.

From the (inadequate) online documentation, I can't tell for sure

whether it would be suitable. At a first impression, it appears to be
too high-level. Such systems tend to either require that you use
whatever the system emits without understanding it, or that you
understand the inner workings of the system along with all of its
targets.

It might help if you explain what the problem with the existing system
is, given that GNU make is available on Windows.

--
Glynn Clements <glynn@gclements.plus.com>

I try to summarize. Everybody wish to solve the grass and qgis-grass
windows build issue in the more feasable way. Two major streams are
there:

1 - do it inside osgeo4w, using the vc built libs and apps there
(from gdal to qgis). AFAIK there are problems with linking MinGW built
things and the MSVC ones (also Marco Pasetti experience says this). Is
it 'just' a problem of memory menagement, as Jef says? Does solving
this would permit to make grass vs rest-of-the-world linkage stable?

2 - do it outside osgeo4w, continuing to build grass, qgis, gdal, etc.
with MinGW

In the first case we have to decide if we prefer to use GNU make or
nmake. Both is too much? I understand that Windows developers work
mostly with vc, so maybe we could mantain officially a GNU system, and
let Windows developers mantain, eventually, a nmake build system?
Otherwise, having all the other things built with vc (ie Qgis), we
could take for granted that on Windows we build with vc (free,
express, edition hopefully!) and stop.

Can we follow a logical order in this discussion to try to reach a
common conclusion and, possibily, a shared roadmap for our windows
developers?

On Fri, Mar 27, 2009 at 12:13 AM, G. Allegri <giohappy@gmail.com> wrote:

I try to summarize. Everybody wish to solve the grass and qgis-grass
windows build issue in the more feasable way. Two major streams are
there:

1 - do it inside osgeo4w, using the vc built libs and apps there
(from gdal to qgis). AFAIK there are problems with linking MinGW built
things and the MSVC ones (also Marco Pasetti experience says this).

I am not aware of any postings in this regard from Marco.
Please point us to that to better understand the issues he found.

2 - do it outside osgeo4w, continuing to build grass, qgis, gdal, etc.
with MinGW

Question: Did you try to update the installer from Marco? Maybe
it works out of the box with a few minor tweaks? It is in the
mswindows/ directory in the source code.

Markus

I am not aware of any postings in this regard from Marco.
Please point us to that to better understand the issues he found.

I've just asked him if he can partecipate to the discussion to explain this...

Question: Did you try to update the installer from Marco? Maybe
it works out of the box with a few minor tweaks? It is in the
mswindows/ directory in the source code.

No problems with mingw build. I've already built grass6_devel from
scratch, and it almost works (better then the one I've built against
osgeo4w libs):

1 - wxdigit and wxnviz don't work, but I know that Martin is working
on this right now
2 - Marco told me that there soudl be made some changes to update the
build system. I still have to dig in this, but there are reasons to
hope that Marco, maybe, will find some time to work on this...
3 -qgis bindings. With osgeo4w, on windows Vista, qgis crashes
whatever you try to do with the plugin: db.exe returns and makes Qgis
crash. I don't know yet if it's the same problem with memory
management or what else.

In any case the questions remain:

1 - continue to build grass with MingW?
      1a: build it against MinGW built libs (not osgeo4w)?
      1b: build it against vc built libs (osgeo4w)?

2 - build Qgis with vc (osgeo4w) and solve the bindings problems with
Grass: 1a or 1b?

On 27/03/09 10:30, G. Allegri wrote:
  > In any case the questions remain:

1 - continue to build grass with MingW?
      1a: build it against MinGW built libs (not osgeo4w)?
      1b: build it against vc built libs (osgeo4w)?

2 - build Qgis with vc (osgeo4w) and solve the bindings problems with
Grass: 1a or 1b?

IIUC what Markus and Glynn are trying to get at: these questions seem to be a bit quick without having clarified where the problem actually lies. Is it with the GRASS source code, is it with the choice of build system or is it with the osgeo4w tool chain. Jürgen says it's mainly the choice of configure/make which is at stake, Glynn seems to contest that. So before we haven't clearly identified where the problems actually lie, we cannot answer the above questions.

Moritz

Ok Markus. I'm sorry for pressing... My hope is to do not have a
broken community around this issues. I would like to come to a common
thought, and work for a coherent and clear roadmap. Probably it's me
(and the users) that don't have it clear, while it is for all of the
developers: who is doing what and in which direction?

From an old Qgis log, Frank has explained that his dream is to have

everything coherently packaged under osgeo4w, and hopefully to be able
to extract standalone builds from it (ie, qgis, grass, qgis+grass).
This viewsight probably would be the best from the users point of
view. The best would be to have osgeo4w setup, and standalone NSI
installers... but I'm talking with no experience on what this would
require, or if it's possible.

I hope the discussion will go on.

2009/3/27 Moritz Lennert <mlennert@club.worldonline.be>:

On 27/03/09 10:30, G. Allegri wrote:
> In any case the questions remain:

1 - continue to build grass with MingW?
1a: build it against MinGW built libs (not osgeo4w)?
1b: build it against vc built libs (osgeo4w)?

2 - build Qgis with vc (osgeo4w) and solve the bindings problems with
Grass: 1a or 1b?

IIUC what Markus and Glynn are trying to get at: these questions seem to be
a bit quick without having clarified where the problem actually lies. Is it
with the GRASS source code, is it with the choice of build system or is it
with the osgeo4w tool chain. Jürgen says it's mainly the choice of
configure/make which is at stake, Glynn seems to contest that. So before we
haven't clearly identified where the problems actually lie, we cannot answer
the above questions.

Moritz

Anyway, IIUC Glynn and Jurgen:

the problem on mixing MSVC and MinGW built things (at C level) is
just a problem of a clean memory management (malloc/free). So, my
question is: would changing the build system (GNU make -> cmake) solve
this? Or they're unrelated things?

2009/3/27 G. Allegri <giohappy@gmail.com>:

Ok Markus. I'm sorry for pressing... My hope is to do not have a
broken community around this issues. I would like to come to a common
thought, and work for a coherent and clear roadmap. Probably it's me
(and the users) that don't have it clear, while it is for all of the
developers: who is doing what and in which direction?

From an old Qgis log, Frank has explained that his dream is to have
everything coherently packaged under osgeo4w, and hopefully to be able
to extract standalone builds from it (ie, qgis, grass, qgis+grass).
This viewsight probably would be the best from the users point of
view. The best would be to have osgeo4w setup, and standalone NSI
installers... but I'm talking with no experience on what this would
require, or if it's possible.

I hope the discussion will go on.

2009/3/27 Moritz Lennert <mlennert@club.worldonline.be>:

On 27/03/09 10:30, G. Allegri wrote:
> In any case the questions remain:

1 - continue to build grass with MingW?
1a: build it against MinGW built libs (not osgeo4w)?
1b: build it against vc built libs (osgeo4w)?

2 - build Qgis with vc (osgeo4w) and solve the bindings problems with
Grass: 1a or 1b?

IIUC what Markus and Glynn are trying to get at: these questions seem to be
a bit quick without having clarified where the problem actually lies. Is it
with the GRASS source code, is it with the choice of build system or is it
with the osgeo4w tool chain. Jürgen says it's mainly the choice of
configure/make which is at stake, Glynn seems to contest that. So before we
haven't clearly identified where the problems actually lie, we cannot answer
the above questions.

Moritz

Hi Giovanni,

On Fri, 27. Mar 2009 at 12:36:43 +0100, G. Allegri wrote:

the problem on mixing MSVC and MinGW built things (at C level) is
just a problem of a clean memory management (malloc/free). So, my
question is: would changing the build system (GNU make -> cmake) solve
this? Or they're unrelated things?

Unrelated.

If everything is compile with the same compiler (ie. using the same
runtime library), that will make the problems disappear again.

But that's not even true for all the VC built binaries in OSGeo4W (e.g.
QGIS and Qt are build with Visual C++ 2008 Express Edition, while GDAL
and python are built with VC 7.1 - I think).

AFAICS there's no way around fixing that in GRASS for OSGeo4W.

The build system currently doesn't support VC at all. I didn't
investigate how intensive it would be to add that to the existing
Makefile, but that might be an option, too.

Jürgen

--
Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-20
Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50
Software Engineer D-26506 Norden http://www.norbit.de

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

Thanks Jef. So, fixing the memory issues in grass code isn't
avoidable. I'll go on with this (after g.proj, db binaries, v.in.ogr,
etc.), because probably this will solve the problems that users are
facing from within QGis (various crashes when using the grass plugin).
So the real limitation for the end-user, is the possibility to use the
complete wxgui (digitizer, nviz), but as Martin is working on this I
think this isn't an issue.

The build system is not strict for the end-users collegues: "I want it
to work, I don't care how they made it!". This is my boss phrase :slight_smile:

2009/3/27 Jürgen E. <jef@norbit.de>:

Hi Giovanni,

On Fri, 27. Mar 2009 at 12:36:43 +0100, G. Allegri wrote:

the problem on mixing MSVC and MinGW built things (at C level) is
just a problem of a clean memory management (malloc/free). So, my
question is: would changing the build system (GNU make -> cmake) solve
this? Or they're unrelated things?

Unrelated.

If everything is compile with the same compiler (ie. using the same
runtime library), that will make the problems disappear again.

But that's not even true for all the VC built binaries in OSGeo4W (e.g.
QGIS and Qt are build with Visual C++ 2008 Express Edition, while GDAL
and python are built with VC 7.1 - I think).

AFAICS there's no way around fixing that in GRASS for OSGeo4W.

The build system currently doesn't support VC at all. I didn't
investigate how intensive it would be to add that to the existing
Makefile, but that might be an option, too.

Jürgen

--
Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-20
Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50
Software Engineer D-26506 Norden http://www.norbit.de

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

On Fri, Mar 27, 2009 at 3:05 PM, G. Allegri <giohappy@gmail.com> wrote:

Thanks Jef. So, fixing the memory issues in grass code isn't
avoidable.

I think that several developer are willing to help here once understood
the problems which means...

I'll go on with this (after g.proj, db binaries, v.in.ogr, etc.),

... cycle through all these problems and fix one after the other.
g.proj is already done thanks to Paul.

Please send a concrete list, ideally with debug output.

Markus