[GRASS-dev] WinGRASS Wiki Hacking

Hi all,

I’m working on the WinGRASS Wiki page; I basically merged my personal GRASS To-Do list with it; please, take a quick look to check if I worked fine, thanks.

Then, I think that section 2 (Installing binary snapshots) should be removed:
I’m actually not providing binary snapshots in my WinGRASS job, and I think that no on eis doing the job. I planned to build weekly binary snapshots of both 6.4 and 7 development source code (and the installer script is already “prepared” for that), but at the moment I do’nt have enough spare time;
This said, I propose to remove this section.

What do you think about?

Regards,

Marco

On Sat, 21 Jun 2008, Marco Pasetti wrote:

Hi all,

I'm working on the WinGRASS Wiki page; I basically merged my personal GRASS To-Do list with it; please, take a quick look to check if I worked fine, thanks.

I take it we're talking about this page: http://grass.osgeo.org/wiki/WinGRASS_Current_Status

Then, I think that section 2 (Installing binary snapshots) should be removed:
I'm actually not providing binary snapshots in my WinGRASS job, and I think that no on eis doing the job. I planned to build weekly binary snapshots of both 6.4 and 7 development source code (and the installer script is already "prepared" for that), but at the moment I do'nt have enough spare time;

I don't think it should be labour intensive - it should be able to be scripted and run automatically. That said, I haven't got round to doing it either. But deleting all mention of it will discourage other people from helping with it.

This said, I propose to remove this section.

What do you think about?

I think there are a few reasons for keeping it. People might want to try old versions to see what is improved, or they might not have the bandwidth to download the monolithic installer, or they just might not like monolithic installers and prefer to run GRASS as a trial where it does not affect anything else on their system. Speaking for myself, I am always happy when I see a Windows program with a choice of downloads: monolithic
installer and zipped binaries that can be run directly and don't require installation. E.g. the TightVNC download page at http://www.tightvnc.com/download.html is a good example of a choice of Windows binaries.

Also at some stage I think we need to update the "Compiling by yourself" section to reflect that lots of the libraries are available from Gnuwin32 at http://gnuwin32.sourceforge.net/. And definitely mention that Tcl/Tk can be compiled from source and Activestate Tcl isn't a strict requirement.

"Nothing known" for XP/2000 issues sounds a little optimistic! I don't think the problem with running scripts when there are spaces in the install path has been fixed yet - has it? Off the top of my head there is also the mapset locking issue which we didn't decide on a best way to resolve yet. There are also IIRC various problems with CRLF line endings in some internal config files.

Re the FFmpeg linking problem I think Glynn said on the list that he had fixed it in SVN.

I had never heard of OSGeo4W (not sure if I like the name) - looks like it could be intersting.

P.S. Marco I'm not suggesting you fix all these things or that we need an answer to them! Just that they might be useful for us to consider.

Paul

Hi Paul,

I don't think it should be labour intensive - it should be able to be scripted and run automatically. That said, I haven't got round to doing it either. But deleting all mention of it will discourage other people from helping with it.

Yes, it's not actually a labour intensive task. I just need to do it and upload the binaries to Markus (and write down few lines in the WinGRASS readme).
I'm doing my last university exams now, so sometimes "simple things" seem to me as "time expensive", when actually they aren't (ah... student anxiety!!)

I think there are a few reasons for keeping it. People might want to try old versions to see what is improved, or they might not have the bandwidth to download the monolithic installer, or they just might not like monolithic installers and prefer to run GRASS as a trial where it does not affect anything else on their system.

actually the current WinGRASS installer just adds some entries to the registry and links to the Desktop and The StartMenu folder.
while, about the bandwith, yes, I could prepare a simple tar.gz with only the grass binaries, to use along the already distributed GRASS MSYS environment tar.gz
...we could do as follows: prepare a "complete" SVN development installer, with all included (GRASS + dependencies), and then a "simple" SVN weekly snaposhot GRASS installer that contains only the GRASS binaryes and automatically installs those binaries in the "right place" reading the needed paths into the registry.

monolithic
installer and zipped binaries that can be run directly and don't require installation

yes we could, but I don't know id we have so much space to use on the osgeo server (Markus?). This said, the installer creates 3 specific files during the installation (grass.bat, grass and .grassrc6), to fits the specif user configuration (basically the install dir path); simple binaries will not work "alone"... (even if they just need few line hacking in those files...)

Also at some stage I think we need to update the "Compiling by yourself" section to reflect that lots of the libraries are available from Gnuwin32 at http://gnuwin32.sourceforge.net/.

I'm afraid to come again on this topic, but I definetely checked that many binaries from the GNUWin32 project don't work as expected when building GRASS or other libraries. I can give you many examples: zlib, libpng, libtiff, libjpeg and others... I don't know why, but when I tried to build both GDAL and GRASS with them, they failed... while if I use my builds (from source) they compile! I'm not stubborn, nor I want to reinvent the wheel each time, but I tried... and sometimes they don't work! That's my work procedure: try prebuilt libs first! if they don't work as expected, try build them from source.... and if I say in my guide to build from source instead of use prebuilt binaries, there's actually a reason...

And definitely mention that Tcl/Tk can be compiled from source and Activestate Tcl isn't a strict requirement.

yes, I think that we should deeply modify this section. First of all I should move my building guide into /mswindows/native/ and then refer to that.

"Nothing known" for XP/2000 issues sounds a little optimistic!

yes, it's a lot optimistic! But I guess that the person who wrote that lines, intended that there are no specific issues with XP... while there are many with Windows in general...

Re the FFmpeg linking problem I think Glynn said on the list that he had fixed it in SVN.

I suppose that Glynn fixed the libavutil check in the configure, and not the gcc issue in the makefile. But I just suppose...

I had never heard of OSGeo4W (not sure if I like the name) - looks like it could be intersting.

I'm actually working with it; but there are still many opened issues about it (the main is that they use VS to build binaries)

P.S. Marco I'm not suggesting you fix all these things or that we need an answer to them! Just that they might be useful for us to consider.

Don't worry, I need this kind of mails :slight_smile:

Thanks for all; now I must come back to my books :frowning:

Marco

Marco Pasetti wrote:

> I think there are a few reasons for keeping it. People might want to try
> old versions to see what is improved, or they might not have the bandwidth
> to download the monolithic installer, or they just might not like
> monolithic installers and prefer to run GRASS as a trial where it does not
> affect anything else on their system.

actually the current WinGRASS installer just adds some entries to the
registry and links to the Desktop and The StartMenu folder.
while, about the bandwith, yes, I could prepare a simple tar.gz with only
the grass binaries, to use along the already distributed GRASS MSYS
environment tar.gz
...we could do as follows: prepare a "complete" SVN development installer,
with all included (GRASS + dependencies), and then a "simple" SVN weekly
snaposhot GRASS installer that contains only the GRASS binaryes and
automatically installs those binaries in the "right place" reading the
needed paths into the registry.

Sooner or later, someone will need to bite the bullet and produce a
dependency-based installer, probably based upon Cygwin's.

The only question is how much time gets wasted on half-baked solutions
between now and then.

BTW, it might be a good idea to actually put a link to:

  http://www.webalice.it/marco.pasetti/grass/BuildFromSource.html

in an obvious place on the main GRASS website. I.e. on:

  http://grass.osgeo.org/devel/index.php

under "Compiling GRASS", in place of the link for cross-compiling
Windows binaries on Linux.

> Also at some stage I think we need to update the "Compiling by yourself"
> section to reflect that lots of the libraries are available from Gnuwin32
> at http://gnuwin32.sourceforge.net/.

I'm afraid to come again on this topic, but I definetely checked that many
binaries from the GNUWin32 project don't work as expected when building
GRASS or other libraries. I can give you many examples: zlib, libpng,
libtiff, libjpeg and others... I don't know why, but when I tried to build
both GDAL and GRASS with them, they failed... while if I use my builds (from
source) they compile! I'm not stubborn, nor I want to reinvent the wheel
each time, but I tried... and sometimes they don't work! That's my work
procedure: try prebuilt libs first! if they don't work as expected, try
build them from source.... and if I say in my guide to build from source
instead of use prebuilt binaries, there's actually a reason...

If you post to the list details of exactly what goes wrong when you
try to use the pre-compiled libraries, we might be able to help you.

Ideally, we shouldn't be recommending that people build from source
anything for which pre-compiled binaries are available.

I appreciate that you don't have an unlimited amount of time, and just
building from source may be quicker than figuring out how to use the
binaries, but ultimately using "official" binary releases is very much
preferable.

> Re the FFmpeg linking problem I think Glynn said on the list that he had
> fixed it in SVN.

I suppose that Glynn fixed the libavutil check in the configure, and not the
gcc issue in the makefile. But I just suppose...

The OGSF Makefile didn't require any changes. The changes to configure
result in $(FFMPEGLIB) having -lavutil as well as -lavformat and
-lavcodec.

--
Glynn Clements <glynn@gclements.plus.com>

Glynn Clements wrote:

Sooner or later, someone will need to bite the bullet and produce a
dependency-based installer, probably based upon Cygwin's.

The only question is how much time gets wasted on half-baked solutions
between now and then.

Glynn,

I think that's what I'm doing already with OSGeo4W! It includes
a dependency based installer derived from the Cygwin one (but with
no dependency on cygwin runtime dlls). It is intended to provide
an integrated "production ready" windows FOSS4G solution for all
OSGeo projects, and other projects of interest.

There is already a working version available, though it is rough
around the edges, and the set of packages is still incomplete.

It includes a somewhat half-baked WinGRASS build I prepared myself
several months ago, based on the wingrass build notes at the time.
Amoung other things, it proves that a MingW built GRASS can use
a VC7.1 built GDAL.

I've been mildly disappointed in the past that Marco wasn't
interested in preparing a WinGRASS package for OSGeo4W, but
he explained that it seemed like more than he wanted to take
responsibility for at the time. I'm hopeful that someone else
from the GRASS community will take on this responsibility.

More info available at:

  http://trac.osgeo.org/osgeo4w

I hope to see some grass folks participate.

I'd add that I would love to see even independent WinGRASS
installers using as many packages as possible for dependent libraries
from OSGeo4W. I believe the OSSIM folks are already moving in this
direction.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGeo, http://osgeo.org

Frank Warmerdam wrote:

> Sooner or later, someone will need to bite the bullet and produce a
> dependency-based installer, probably based upon Cygwin's.
>
> The only question is how much time gets wasted on half-baked solutions
> between now and then.

Glynn,

I think that's what I'm doing already with OSGeo4W! It includes
a dependency based installer derived from the Cygwin one (but with
no dependency on cygwin runtime dlls). It is intended to provide
an integrated "production ready" windows FOSS4G solution for all
OSGeo projects, and other projects of interest.

There is already a working version available, though it is rough
around the edges, and the set of packages is still incomplete.

It includes a somewhat half-baked WinGRASS build I prepared myself
several months ago, based on the wingrass build notes at the time.
Amoung other things, it proves that a MingW built GRASS can use
a VC7.1 built GDAL.

I've been mildly disappointed in the past that Marco wasn't
interested in preparing a WinGRASS package for OSGeo4W, but
he explained that it seemed like more than he wanted to take
responsibility for at the time. I'm hopeful that someone else
from the GRASS community will take on this responsibility.

More info available at:

  http://trac.osgeo.org/osgeo4w

I downloaded the installer and, yes, that's exactly what I was
suggesting that GRASS should be doing. It shouldn't be necessary to
re-download dozens of third-party libraries with every GRASS snapshot
or release candidate.

It would be particularly useful if it could be extended for use by
developers as well as end users, by including the MinGW/MSys
development environment (gcc, make, flex, bison etc).

The last time I decided to put some effort into the native Windows
version, it took a good portion of 3 days to get to the point that I
could compile GRASS with the only problems being due to GRASS itself
rather than the development environment.

Many of the GRASS developers have access to a Windows system. If it
could be made so that getting to the point of being able to compile
GRASS on Windows took a few minutes rather than a few days, I think
that more of us would be willing to spend time on the native Windows
version.

One thing which I'm curious about: is it necessary to host all of the
packages on a common server? I know that the setup.ini file normally
only contains pathnames. Is that a restriction of setup.exe? Is it
something which would be easy to change? Would it be feasible to
configure the web server to use HTTP redirects to have non-OSGeo
packages (TIFF, JPEG, PNG, etc) retrieved from the original site?

--
Glynn Clements <glynn@gclements.plus.com>

Glynn Clements wrote:

I downloaded the installer and, yes, that's exactly what I was
suggesting that GRASS should be doing. It shouldn't be necessary to
re-download dozens of third-party libraries with every GRASS snapshot
or release candidate.

It would be particularly useful if it could be extended for use by
developers as well as end users, by including the MinGW/MSys
development environment (gcc, make, flex, bison etc).

Glynn,

OSGeo4W does endevour to include the stub libraries and include files
for the packages (such as libpng, etc). I hadn't contemplated including
the MingW before, but I wouldn't mind including it if someone wanted
to package it.

One thing which I'm curious about: is it necessary to host all of the
packages on a common server? I know that the setup.ini file normally
only contains pathnames. Is that a restriction of setup.exe?

I do not know a way of having packages come from different locations.

I have certainly wanted some means of "federated respositories" for
a variety of reasons. First, I would like a mechanism to support
"non-free" packages external to OSGeo that I don't feel it is right
for OSGeo to host. Second, I'd like a way of using OSGeo4W to
deliver custom content to particular clients of mine.

Unfortunately, while I have made noteworthy changes to the setup
program, I have found it a somewhat challenging environment to
work in.

> Is it

something which would be easy to change? Would it be feasible to
configure the web server to use HTTP redirects to have non-OSGeo
packages (TIFF, JPEG, PNG, etc) retrieved from the original site?

This could work in theory, but is still leaves a challenge to build
the setup.ini from these packages in different places. Currently the
setup.ini generation script makes a pass over a particular directory
tree.

I'd add that stuff like TIFF, JPEG and PNG still need to be "packaged"
as the final packages need a particular directory layout, and need
to be packaged as a .tar.bz2 file.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGeo, http://osgeo.org

Frank Warmerdam wrote:

It includes a somewhat half-baked WinGRASS build I prepared myself
several months ago, based on the wingrass build notes at the time.
Amoung other things, it proves that a MingW built GRASS can use
a VC7.1 built GDAL.

Alas, I'm finding that isn't the case. Any and all attempts to
configure GRASS using the OSGeo4W libraries fail with:

  undefined reference to `GDALOpen'

[This is after I've constructed a working gdal-config from
gdal-config.tmpl.]

I have tried both linking against the import library with:

  -L/c/OSGeo4W/lib -lgdal_i

and directly against the DLL with:

  -L/c/OSGeo4W/bin -lgdal15

I also had to remove the msvcrt.dll from the bin directory, as that
caused -L/c/OSGeo4W/bin to fail (even if you didn't actually use any
libraries) due to multiple definitions of _onexit() and atexit().

--
Glynn Clements <glynn@gclements.plus.com>

Glynn Clements wrote:

Frank Warmerdam wrote:

It includes a somewhat half-baked WinGRASS build I prepared myself
several months ago, based on the wingrass build notes at the time.
Amoung other things, it proves that a MingW built GRASS can use
a VC7.1 built GDAL.

Alas, I'm finding that isn't the case. Any and all attempts to
configure GRASS using the OSGeo4W libraries fail with:

  undefined reference to `GDALOpen'

[This is after I've constructed a working gdal-config from
gdal-config.tmpl.]

I have tried both linking against the import library with:

  -L/c/OSGeo4W/lib -lgdal_i

Glynn,

The gdal-config distributed in /c/osgeo4w/bin should return this for
gdal-config --libs:

warmerda@gdal3200% gdal-config --libs
/C/osgeo4w/lib/gdal_i.lib

I use this configure line:

./configure \
         --prefix=c:/warmerda/grassbld/bld \
         --bindir=c:/warmerda/grassbld/bld/bin \
         --with-includes=/c/warmerda/grassbld/forgrass/include \
         --with-libs=/c/warmerda/grassbld/forgrass/lib \
         --with-gdal=/c/osgeo4w/bin/gdal-config \
         --with-proj-includes=/c/osgeo4w/include \
         --with-proj-libs=/c/osgeo4w/lib \
         --with-proj-share=/c/osgeo4w/share/proj \
         --with-cxx --without-jpeg --without-tiff \
         --without-postgres --with-opengl=windows --without-fftw --without-x \
         --enable-x11=no --enable-shared=yes \
         --with-tcltk-includes=/c/tcl/include \
         --with-tcltk-libs=/c/tcl/bin

To get proj to work, I found I had to copy /c/osgeo4w/lib/proj_i.lib to
/c/osgeo4w/lib/libproj.a. If you file an OSGeo4W ticket on this, I would
be willing to make it a standard part of the proj package for OSGeo4W. But
in the meantime it is pretty easy to take the .lib stub libraries and copy
them somewhere as .a files for use.

As you may note above, I think PROJ and GDAL were the only two subcomponents
I got from OSGeo4W during the build. I would hope in the future to get more
but at this point I sort of ran out of steam.

and directly against the DLL with:

  -L/c/OSGeo4W/bin -lgdal15

I also had to remove the msvcrt.dll from the bin directory, as that
caused -L/c/OSGeo4W/bin to fail (even if you didn't actually use any
libraries) due to multiple definitions of _onexit() and atexit().

I think you are barking up the wrong tree here, and there should be
no reason to remove msvcrt.dll from the bin directory.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGeo, http://osgeo.org

Frank Warmerdam wrote:

>> It includes a somewhat half-baked WinGRASS build I prepared myself
>> several months ago, based on the wingrass build notes at the time.
>> Amoung other things, it proves that a MingW built GRASS can use
>> a VC7.1 built GDAL.
>
> Alas, I'm finding that isn't the case. Any and all attempts to
> configure GRASS using the OSGeo4W libraries fail with:
>
> undefined reference to `GDALOpen'
>
> [This is after I've constructed a working gdal-config from
> gdal-config.tmpl.]
>
> I have tried both linking against the import library with:
>
> -L/c/OSGeo4W/lib -lgdal_i

Glynn,

The gdal-config distributed in /c/osgeo4w/bin should return this for
gdal-config --libs:

warmerda@gdal3200% gdal-config --libs
/C/osgeo4w/lib/gdal_i.lib

I've tried changing it to that, with the same result:

  configure:7653: checking for GDALOpen
  configure:7679: gcc -o conftest.exe -g -I/usr/local/include -I/c/progra~1/GnuWin32/include -I/c/OSGeo4W/include -Wl,--export-dynamic,--enable-runtime-pseudo-reloc -L/usr/local/lib -L/c/progra~1/GnuWin32/lib -L/c/OSGeo4W/lib conftest.c /c/OSGeo4W/lib/gdal_i.lib -L/c/OSGeo4W/lib -lpng -lz 1>&5
  C:/DOCUME~1/glynn/LOCALS~1/Temp/ccAxbaaa.o(.text+0x2b): In function `main':
  C:/msys/1.0/home/glynn/grass/configure:7673: undefined reference to `GDALOpen'
  collect2: ld returned 1 exit status

But I've figured it out. The GDAL library was built to use "stdcall",
which isn't compatible with autoconf tests (AC_CHECK_FUNC etc).

The autoconf AC_CHECK_FUNC test doesn't use any headers, so it doesn't
know that the function uses the "stdcall" convention, and also doesn't
know that its argument list totals 8 bytes (hence the _GDALOpen@8
name).

We would have to write a custom test, using AC_TRY_LINK on a complete
test program (which includes headers). We would also need to work
around the CPL_STDCALL definition in cpl_port.h:

  #ifndef CPL_STDCALL
  #if defined(_MSC_VER) && !defined(CPL_DISABLE_STDCALL)
  # define CPL_STDCALL __stdcall
  #else
  # define CPL_STDCALL
  #endif
  #endif

which only uses __stdcall for MSVC.

Or we could just compile GDAL ourselves from source using MinGW, which
is probably the best solution. Although I would rather stick to
"official" binaries where possible, in this case it's likely to be
more trouble than it's worth.

Note that PROJ doesn't use stdcall, nor do any of the GnuWin32
libraries, nor MinGW's own libraries (obviously the import libraries
for the Windows API do, as that's what Windows' own DLLs use).

For the most part, using __stdcall is a bad idea. Modern computers
have rather more than 640KiB of RAM, so shaving a few bytes from each
function call really isn't that important any more.

To get proj to work, I found I had to copy /c/osgeo4w/lib/proj_i.lib to
/c/osgeo4w/lib/libproj.a.

I just added /c/OSGeo4W/bin to the --with-libs= argument, so it links
directly against the DLL.

> I also had to remove the msvcrt.dll from the bin directory, as that
> caused -L/c/OSGeo4W/bin to fail (even if you didn't actually use any
> libraries) due to multiple definitions of _onexit() and atexit().

I think you are barking up the wrong tree here, and there should be
no reason to remove msvcrt.dll from the bin directory.

No, that's definitely necessary if the bin directory is in the library
path (e.g. for linking directly against DLLs). Otherwise, it won't
link even the most trivial programs:

  $ cat foo.c
  int main(void)
  {
          return 0;
  }
  $ gcc foo.c
  $ gcc foo.c -L/c/osgeo4w/bin
  d000007.o(.text+0x0): multiple definition of `_onexit'
  /mingw/lib/crt2.o(.text+0x2d0):crt1.c: first defined here
  d000010.o(.text+0x0): multiple definition of `atexit'
  /mingw/lib/crt2.o(.text+0x2c0):crt1.c: first defined here
  collect2: ld returned 1 exit status
  $ mv /c/osgeo4w/bin/{,x}msvcrt.dll
  $ gcc foo.c -L/c/osgeo4w/bin
  $

That isn't necessarily a problem (msvcrt.dll should already be present
in Windows' system directory), but some people might get bitten the
same way.

Anyhow, I'd suggest that we use OSGeo4W's PROJ library, GnuWin32 for
most of the other libraries, and compile GDAL ourselves, along with
the hacked XDR library (there doesn't appear to be a standard binary
for this; GnuWin32's version of libc doesn't appear to have these
functions).

--
Glynn Clements <glynn@gclements.plus.com>

Glynn Clements wrote:

I've tried changing it to that, with the same result:

  configure:7653: checking for GDALOpen
  configure:7679: gcc -o conftest.exe -g -I/usr/local/include -I/c/progra~1/GnuWin32/include -I/c/OSGeo4W/include -Wl,--export-dynamic,--enable-runtime-pseudo-reloc -L/usr/local/lib -L/c/progra~1/GnuWin32/lib -L/c/OSGeo4W/lib conftest.c /c/OSGeo4W/lib/gdal_i.lib -L/c/OSGeo4W/lib -lpng -lz 1>&5
  C:/DOCUME~1/glynn/LOCALS~1/Temp/ccAxbaaa.o(.text+0x2b): In function `main':
  C:/msys/1.0/home/glynn/grass/configure:7673: undefined reference to `GDALOpen'
  collect2: ld returned 1 exit status

But I've figured it out. The GDAL library was built to use "stdcall",
which isn't compatible with autoconf tests (AC_CHECK_FUNC etc).

The autoconf AC_CHECK_FUNC test doesn't use any headers, so it doesn't
know that the function uses the "stdcall" convention, and also doesn't
know that its argument list totals 8 bytes (hence the _GDALOpen@8
name).

We would have to write a custom test, using AC_TRY_LINK on a complete
test program (which includes headers). We would also need to work
around the CPL_STDCALL definition in cpl_port.h:

  #ifndef CPL_STDCALL
  #if defined(_MSC_VER) && !defined(CPL_DISABLE_STDCALL)
  # define CPL_STDCALL __stdcall
  #else
  # define CPL_STDCALL
  #endif

which only uses __stdcall for MSVC.

Glynn,

Hmm, I thought I had dealt with this. The /c/osgeo4w/include/cpl_config.h
file should start with:

/* We define this here in general so that a VC++ build will publically
    declare STDCALL interfaces even if an application is built against it
    using MinGW */

#ifndef CPL_DISABLE_STDCALL
# define CPL_STDCALL __stdcall
#endif

The mingw build *should* properly respect this stdcall definition,
and it works for me. Is there any chance you are picking up some other
cpl_config.h instead of the one in /c/osgeo4w/include? Could you
confirm that the /c/osgeo4w/include/cpl_config.h on your system starts
with the stdcall stuff?

Or we could just compile GDAL ourselves from source using MinGW, which
is probably the best solution.

This is exactly what I'm trying to avoid!

Although I would rather stick to
"official" binaries where possible, in this case it's likely to be
more trouble than it's worth.

Note that PROJ doesn't use stdcall, nor do any of the GnuWin32
libraries, nor MinGW's own libraries (obviously the import libraries
for the Windows API do, as that's what Windows' own DLLs use).

For the most part, using __stdcall is a bad idea. Modern computers
have rather more than 640KiB of RAM, so shaving a few bytes from each
function call really isn't that important any more.

stdcall is used so that the GDAL functions can be more easily called
from VB6 and/or delphi as I recall. It is not done for efficiency
purposes.

To get proj to work, I found I had to copy /c/osgeo4w/lib/proj_i.lib to
/c/osgeo4w/lib/libproj.a.

I just added /c/OSGeo4W/bin to the --with-libs= argument, so it links
directly against the DLL.

I was not aware that this was possible.

I also had to remove the msvcrt.dll from the bin directory, as that
caused -L/c/OSGeo4W/bin to fail (even if you didn't actually use any
libraries) due to multiple definitions of _onexit() and atexit().

I think you are barking up the wrong tree here, and there should be
no reason to remove msvcrt.dll from the bin directory.

No, that's definitely necessary if the bin directory is in the library
path (e.g. for linking directly against DLLs). Otherwise, it won't
link even the most trivial programs:

  $ cat foo.c
  int main(void)
  {
          return 0;
  }
  $ gcc foo.c
  $ gcc foo.c -L/c/osgeo4w/bin
  d000007.o(.text+0x0): multiple definition of `_onexit'
  /mingw/lib/crt2.o(.text+0x2d0):crt1.c: first defined here
  d000010.o(.text+0x0): multiple definition of `atexit'
  /mingw/lib/crt2.o(.text+0x2c0):crt1.c: first defined here
  collect2: ld returned 1 exit status
  $ mv /c/osgeo4w/bin/{,x}msvcrt.dll
  $ gcc foo.c -L/c/osgeo4w/bin

Well, then perhaps the /bin directory should not be in the library
path!

That isn't necessarily a problem (msvcrt.dll should already be present
in Windows' system directory), but some people might get bitten the
same way.

Anyhow, I'd suggest that we use OSGeo4W's PROJ library, GnuWin32 for
most of the other libraries, and compile GDAL ourselves, along with
the hacked XDR library (there doesn't appear to be a standard binary
for this; GnuWin32's version of libc doesn't appear to have these
functions).

Ah, I mistook this email exchange to be an interest in actually packaging
wingrass for OSGeo4W. I see it is not.

I'll return to my corner and wait patiently in the hopes someone shows more
interest participating in OSGeo4W.

Regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up | Frank Warmerdam, warmerdam@pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush | President OSGeo, http://osgeo.org

Frank Warmerdam wrote:

> configure:7653: checking for GDALOpen
> configure:7679: gcc -o conftest.exe -g -I/usr/local/include -I/c/progra~1/GnuWin32/include -I/c/OSGeo4W/include -Wl,--export-dynamic,--enable-runtime-pseudo-reloc -L/usr/local/lib -L/c/progra~1/GnuWin32/lib -L/c/OSGeo4W/lib conftest.c /c/OSGeo4W/lib/gdal_i.lib -L/c/OSGeo4W/lib -lpng -lz 1>&5
> C:/DOCUME~1/glynn/LOCALS~1/Temp/ccAxbaaa.o(.text+0x2b): In function `main':
> C:/msys/1.0/home/glynn/grass/configure:7673: undefined reference to `GDALOpen'
> collect2: ld returned 1 exit status
>
> But I've figured it out. The GDAL library was built to use "stdcall",
> which isn't compatible with autoconf tests (AC_CHECK_FUNC etc).
>
> The autoconf AC_CHECK_FUNC test doesn't use any headers, so it doesn't
> know that the function uses the "stdcall" convention, and also doesn't
> know that its argument list totals 8 bytes (hence the _GDALOpen@8
> name).
>
> We would have to write a custom test, using AC_TRY_LINK on a complete
> test program (which includes headers). We would also need to work
> around the CPL_STDCALL definition in cpl_port.h:
>
> #ifndef CPL_STDCALL
> #if defined(_MSC_VER) && !defined(CPL_DISABLE_STDCALL)
> # define CPL_STDCALL __stdcall
> #else
> # define CPL_STDCALL
> #endif
> #endif
>
> which only uses __stdcall for MSVC.

Glynn,

Hmm, I thought I had dealt with this. The /c/osgeo4w/include/cpl_config.h
file should start with:

/* We define this here in general so that a VC++ build will publically
    declare STDCALL interfaces even if an application is built against it
    using MinGW */

#ifndef CPL_DISABLE_STDCALL
# define CPL_STDCALL __stdcall
#endif

Okay.

The mingw build *should* properly respect this stdcall definition,
and it works for me. Is there any chance you are picking up some other
cpl_config.h instead of the one in /c/osgeo4w/include? Could you
confirm that the /c/osgeo4w/include/cpl_config.h on your system starts
with the stdcall stuff?

It does. but that doesn't help with the configure checks, which don't
use any headers. The test program just provides a local declaration:

  char GDALOpen();

This doesn't account for the calling convention. Nor does it account
for the size of the argument list, which is appended to the name of
any functions which use the stdcall convention (as the callee cleans
up the stack, the caller must push exactly the right number of bytes,
otherwise the stack won't be restored correctly, causing a crash).

The actual test program (conftest.c) which is compiled and linked by
autoconf's AC_CHECK_FUNC macro is:

  #include "confdefs.h"
  /* System header to define __stub macros and hopefully few prototypes,
      which can conflict with char GDALOpen(); below. */
  #include <assert.h>
  /* Override any gcc2 internal prototype to avoid an error. */
  /* We use char because int might match the return type of a gcc2
      builtin and then its argument prototype would still apply. */
  char GDALOpen();
  
  int main() {
  
  /* The GNU C library defines this for functions which it implements
      to always fail with ENOSYS. Some functions are actually named
      something starting with __ and the normal name is an alias. */
  #if defined (__stub_GDALOpen) || defined (__stub___GDALOpen)
  choke me
  #else
  GDALOpen();
  #endif
  
  ; return 0; }

> Or we could just compile GDAL ourselves from source using MinGW, which
> is probably the best solution.

This is exactly what I'm trying to avoid!

I'd also like to avoid it, but I'm wondering if the hassle factor is
going to be too high. The main issue is avoiding the need for
per-platform sections in configure and elsewhere. As a general rule,
the Unix version tends to be where most of the development occurs, and
also where most of the testing occurs.

The more special cases required for Windows, the greater the chance
that the latest version won't build on Windows. It doesn't help that
the luckless user will be largely on their own, as Windows developers
tend to be scarce and relatively inexperienced, and the Unix
developers typically don't understand, can't test and/or don't
especially care.

> Although I would rather stick to
> "official" binaries where possible, in this case it's likely to be
> more trouble than it's worth.
>
> Note that PROJ doesn't use stdcall, nor do any of the GnuWin32
> libraries, nor MinGW's own libraries (obviously the import libraries
> for the Windows API do, as that's what Windows' own DLLs use).
>
> For the most part, using __stdcall is a bad idea. Modern computers
> have rather more than 640KiB of RAM, so shaving a few bytes from each
> function call really isn't that important any more.

stdcall is used so that the GDAL functions can be more easily called
from VB6 and/or delphi as I recall. It is not done for efficiency
purposes.

Okay.

>> To get proj to work, I found I had to copy /c/osgeo4w/lib/proj_i.lib to
>> /c/osgeo4w/lib/libproj.a.
>
> I just added /c/OSGeo4W/bin to the --with-libs= argument, so it links
> directly against the DLL.

I was not aware that this was possible.

It's quite common for Unix ports not to bother with import libraries
(e.g. GRASS doesn't create them; modules just link directly against
the DLLs). AFAICT, import libraries are largely an artifact of
importing symbols by ordinal (which, again, was originally done to
save a miniscule amount of space, by eliminating the need to have a
symbol table in the actual DLL).

> That isn't necessarily a problem (msvcrt.dll should already be present
> in Windows' system directory), but some people might get bitten the
> same way.
>
> Anyhow, I'd suggest that we use OSGeo4W's PROJ library, GnuWin32 for
> most of the other libraries, and compile GDAL ourselves, along with
> the hacked XDR library (there doesn't appear to be a standard binary
> for this; GnuWin32's version of libc doesn't appear to have these
> functions).

Ah, I mistook this email exchange to be an interest in actually packaging
wingrass for OSGeo4W. I see it is not.

I'll return to my corner and wait patiently in the hopes someone shows more
interest participating in OSGeo4W.

Well, I do have some interest, but right now I'm more concerned about
being able to build GRASS on Windows with a minimum of effort[1]. We
can't package what we don't have.

[1] If we can make it really easy, we might even get new volunteers
faster than the existing ones get fed up. Until we get to that point,
the number of active Windows developers at any given time is likely to
continue to oscillate between zero and one.

--
Glynn Clements <glynn@gclements.plus.com>

Frank,

On Wed, Jun 25, 2008 at 7:54 PM, Frank Warmerdam <warmerdam@pobox.com> wrote:
...

Ah, I mistook this email exchange to be an interest in actually packaging
wingrass for OSGeo4W. I see it is not.

I would not see it so negatively.
There is definitely the interest in packaging winGRASS for OSGeo4W.
The technical issues need to be worked out, this will likely take more time
than the duration of this thread (only 5 days now).

I'll return to my corner and wait patiently in the hopes someone shows more
interest participating in OSGeo4W.

To my knowledge (off list private communication) there are some folks
interested to participate in OSGeo4W. Only the entry "barrier" seems to
be too high from what I understood - at least there is some communication
issue. I feel that this could be rather easily resolved, at least I hope so.
Since we have limited resources, we should share code as much
as possible.
It is clear that we need winGRASS as standalone package (we are
already there), but it is also of high interest to integrate it into OSGeo4W
(we are not yet there).

@All involved: What about putting frustrations aside and start again?

Markus

Markus Neteler wrote:

> Ah, I mistook this email exchange to be an interest in actually packaging
> wingrass for OSGeo4W. I see it is not.

I would not see it so negatively.
There is definitely the interest in packaging winGRASS for OSGeo4W.
The technical issues need to be worked out, this will likely take more time
than the duration of this thread (only 5 days now).

> I'll return to my corner and wait patiently in the hopes someone shows more
> interest participating in OSGeo4W.

To my knowledge (off list private communication) there are some folks
interested to participate in OSGeo4W. Only the entry "barrier" seems to
be too high from what I understood - at least there is some communication
issue. I feel that this could be rather easily resolved, at least I hope so.
Since we have limited resources, we should share code as much
as possible.
It is clear that we need winGRASS as standalone package (we are
already there), but it is also of high interest to integrate it into OSGeo4W
(we are not yet there).

@All involved: What about putting frustrations aside and start again?

FWIW, my frustrations are mainly with Windows, not with anything which
Frank has said.

Unfortunately, there are conflicting imperatives. E.g.: making GDAL
use stdcall increases compatibility with various parts of the Windows
"ecosystem" (e.g. VB, Delphi) at the expense of decreasing
compatibility with Unix-oriented software (specifically, with
autoconf).

There's also the inevitable conflict between a preference for MSVC
amongst many Windows developers, and the preference for MinGW both for
Unix-compatibility and because it is free (speech *and* beer).

The stdcall issue could be fixed by avoiding AC_CHECK_FUNC in favour
of an explicit test program. We already do this for the OpenGL checks,
as the OpenGL libraries also use stdcall.

However, I'm reluctant to rely unnecessarily upon having
platform-specific cases, given the minority status of Windows amongst
GRASS users and (especially) developers, as there's always a risk that
the Windows specific parts won't get updated and/or tested regularly
enough.

There's also the issue that GRASS and GDAL share some DLLs (e.g.
zlib), and some of those DLLs have Windows-specific quirks (e.g. -lz
doesn't work with the standard Windows binaries for zlib). So, if we
use the existing GDAL binaries, we would probably have to adjust the
zlib tests as well. Having GRASS use the GnuWin32 zlib while GDAL uses
the version from zlib.net would probably cause problems.

--
Glynn Clements <glynn@gclements.plus.com>

Marco Pasetti wrote:

> Also at some stage I think we need to update the "Compiling by yourself"
> section to reflect that lots of the libraries are available from Gnuwin32
> at http://gnuwin32.sourceforge.net/.

I'm afraid to come again on this topic, but I definetely checked that many
binaries from the GNUWin32 project don't work as expected when building
GRASS or other libraries. I can give you many examples: zlib, libpng,
libtiff, libjpeg and others... I don't know why, but when I tried to build
both GDAL and GRASS with them, they failed... while if I use my builds (from
source) they compile! I'm not stubborn, nor I want to reinvent the wheel
each time, but I tried... and sometimes they don't work! That's my work
procedure: try prebuilt libs first! if they don't work as expected, try
build them from source.... and if I say in my guide to build from source
instead of use prebuilt binaries, there's actually a reason...

FWIW, I've just built GRASS (SVN head) under MSys/MinGW, and the only
libraries which I needed to compile from source were XDR, PROJ, GDAL
and Tcl/Tk[1]. For the others, I used either the project's own
binaries, or those from GnuWin32.

The resulting binaries have:

  NVIZ: yes [1]
  C++ support: yes
  FFTW support: yes [2]
  FreeType support: yes
  GDAL support: yes [3]
  JPEG support: yes
  Large File support (LFS): yes
  NLS support: yes
  OGR support: yes
  OpenGL support: yes
  PNG support: yes
  Readline support: yes
  Tcl/Tk support: yes [4]
  TIFF support: yes

And lack:

  BLAS support: no
  Cairo support: no [5]
  DWG support: no
  FFMPEG support: no
  GLw support: no
  LAPACK support: no
  Motif support: no
  MySQL support: no
  ODBC support: no
  PostgreSQL support: no
  Python support: no
  SQLite support: no
  wxWidgets support: no
  X11 support: no

[1] I had to commit a fix to NVIZ to get it to compile; the previous
change neglected to conditionalise some X11-specific code.

[2] I had to copy libfftw3-3.dll to libfftw3.dll. I suppose that we
could extend the configure check to try -lfftw3-3.

[3] For GDAL, I used its internal versions of the Zlib/TIFF/JPEG/PNG
libraries, and no other dependencies. So far as compiling GRASS is
concerned, you only need a minimal version of GDAL. You can always
replace the GDAL DLL with a more feature-rich version later.

[4] Actually, I haven't tried using a pre-compiled Tcl/Tk binary. I'm
not sure that we should recommend the ActiveState version due to the
licensing terms.

[5] The Cairo checks need pkg-config; however, after manually hacking
Platform.make, I got lib/cairodriver to compile after a minor fix
(which I have committed). I'm thinking that it may be worth adding
--with-cairo-{cflags,ldflags}= options to eliminate the requirement
for pkg-config.

Once I get over the resulting Windows fatigue, I'll look into enabling
some more features.

GLw and Motif depend upon X11; GLw is no longer used, Motif is only
used for xganim. AFAIK, BLAS/LAPACK still aren't actually used for
anything which is part of GRASS itself.

I suppose that wxPython is probably the most important omission. The
main issue here is the lack of python-config on Windows.

AFAICT, SQLite will need to be compiled from source; the only binary
package I could find only contains the DLL, not the headers.

In the meantime, can we put the binaries for XDR, PROJ, GDAL and
Tcl/Tk on the GRASS site? We really shouldn't be forcing people to
compile these themselves before they can even start trying to compile
GRASS. Especially XDR, PROJ, and GDAL, which are mandatory
dependencies.

--
Glynn Clements <glynn@gclements.plus.com>

On Fri, Jun 27, 2008 at 2:53 PM, Glynn Clements
<glynn@gclements.plus.com> wrote:

AFAIK, BLAS/LAPACK still aren't actually used for
anything which is part of GRASS itself.

Right.
It's needed only for i.spec.unmix:
http://trac.osgeo.org/grass/browser/grass-addons/imagery/i.spec.unmix

which still need some fixes to work again (the older version was based
on Meschach; the BLAS/LAPACK isn't completed at 100%).

Markus

On Fri, 27 Jun 2008, Glynn Clements wrote:

[...]

[3] For GDAL, I used its internal versions of the Zlib/TIFF/JPEG/PNG
libraries, and no other dependencies. So far as compiling GRASS is
concerned, you only need a minimal version of GDAL. You can always
replace the GDAL DLL with a more feature-rich version later.

When I compiled GDAL I used external libraries as much as possible, to keep the binary size down - but that was the only reason. For compatibility probably better to use the internal ones.

[4] Actually, I haven't tried using a pre-compiled Tcl/Tk binary. I'm
not sure that we should recommend the ActiveState version due to the
licensing terms.

I agree with that - it was quick and easy to get working before I realised how easy it was to compile Tcl/Tk on Windows anyway.

[...]

In the meantime, can we put the binaries for XDR, PROJ, GDAL and
Tcl/Tk on the GRASS site?

That would be good.

We really shouldn't be forcing people to
compile these themselves before they can even start trying to compile
GRASS. Especially XDR, PROJ, and GDAL, which are mandatory
dependencies.

On the WinGRASS wiki page there is a link to a binary package that I provided containing:
xdr-4.0-mingw32
proj-4.6.0
gdal-1.5.0
fftw-2.1.5
tcl-8.5.0
tk-8.5.0

However those versions are 5-6 months old so it would be great to get things up-to-date. And ultimately I can see us providing simple binary packages on the GRASS site, while people looking for a point & click installer could be directed towards OSGeo4W. I think that would keep server disk space requirements in check, among other things.

Paul

Glynn Clements wrote:

The stdcall issue could be fixed by avoiding AC_CHECK_FUNC in favour
of an explicit test program. We already do this for the OpenGL checks,
as the OpenGL libraries also use stdcall.

I've changed some of the checks to handle libraries which use stdcall.
The configure script now recognises the OSGeo4W versions of PROJ and
GDAL; however, linking against them fails:

gcc -shared -o /home/glynn/src/grass-svn/dist.i686-pc-mingw32/lib/libgrass_gproj.7.0.svn.dll -L/home/glynn/src/grass-svn/dist.i686-pc-mingw32/lib -Wl,--export-dynamic,--enable-runtime-pseudo-reloc -L/c/progra~1/GnuWin32/lib -L/usr/local/lib -L/c/OSGeo4W/lib OBJ.i686-pc-mingw32/get_proj.o OBJ.i686-pc-mingw32/do_proj.o OBJ.i686-pc-mingw32/convert.o OBJ.i686-pc-mingw32/datum.o OBJ.i686-pc-mingw32/ellipse.o -lgrass_gis -lgrass_datetime -lxdr -liberty -lws2_32 -lz -lintl -lproj /c/OSGeo4W/lib/gdal_i.lib && \
(cd /home/glynn/src/grass-svn/dist.i686-pc-mingw32/lib; ln -f -s libgrass_gproj.7.0.svn.dll /home/glynn/src/grass-svn/dist.i686-pc-mingw32/lib/libgrass_gproj.dll)
Cannot export .idata$4: symbol not found
Cannot export .idata$5: symbol not found
Cannot export .idata$6: symbol not found
Cannot export .text: symbol not found
Cannot export gdal15_NULL_THUNK_DATA: symbol not found
Cannot export proj_NULL_THUNK_DATA: symbol not found
collect2: ld returned 1 exit status

It may just be that we need to add some arcane compiler and/or linker
switches, but I wouldn't have a clue as to what.

On a related note, Windows' odbc32 library works (at least configure
detects it and db/driver/odbc links successfully; I haven't tried
running it).

FWIW, PROJ needed the following patch to build as a DLL:

(attachments)

proj.patch (646 Bytes)
gdal.patch (1007 Bytes)

Hi Glynn,

I'm stretched in time (electronic systems exam ongoing) , but I try to join the thread a bit;

FWIW, I've just built GRASS (SVN head) under MSys/MinGW, and the only
libraries which I needed to compile from source were XDR, PROJ, GDAL
and Tcl/Tk[1]. For the others, I used either the project's own
binaries, or those from GnuWin32.

good to know; probably something has changed since I tried the first time to use them.
in the past I had problems to build GDAL with external libs and enabling jpeg support in libtiff, using libjpeg from gnuwin32, obviously.

[1] I had to commit a fix to NVIZ to get it to compile; the previous
change neglected to conditionalise some X11-specific code.

it compiles without changes for me

[2] I had to copy libfftw3-3.dll to libfftw3.dll. I suppose that we
could extend the configure check to try -lfftw3-3.

it works as libfftw3-3 for me, but I compiled it by myself; IIRC the gnuwin32 build has been configured differently than mine... but I'm not sure about that

[4] Actually, I haven't tried using a pre-compiled Tcl/Tk binary. I'm
not sure that we should recommend the ActiveState version due to the
licensing terms.

since it's very fast to compile, also on MinGW, I would higly prefer to build from source, as I currently do

I suppose that wxPython is probably the most important omission. The
main issue here is the lack of python-config on Windows.

I didn't spent much time on it (I tried only few times to let it work without success, but I really didn't seriously try...), but I guess that the soultion would be here: http://wxconfig.googlepages.com/

In the meantime, can we put the binaries for XDR, PROJ, GDAL and
Tcl/Tk on the GRASS site? We really shouldn't be forcing people to
compile these themselves before they can even start trying to compile
GRASS. Especially XDR, PROJ, and GDAL, which are mandatory
dependencies.

I'm currently providing a MSYS build environment, with all the libs/bins I build from source for GRASS, but they are "all together"; I could provide "divided" binaries for each library, if you prefer.
On this w-e I'll send to Markus an updated version of the MSYS build env, with some upgrades and the add of the jpeg lib and support. Along with it I'll upload (to release) a new winGRASS binary release (the fourth for 6.3.0)

Regards,

Marco

Hi Paul,

When I compiled GDAL I used external libraries as much as possible, to keep the binary size down - but that was the only reason. For compatibility probably better to use the internal ones.

agree

On the WinGRASS wiki page there is a link to a binary package that I provided containing:
xdr-4.0-mingw32
proj-4.6.0
gdal-1.5.0
fftw-2.1.5
tcl-8.5.0
tk-8.5.0

However those versions are 5-6 months old so it would be great to get things up-to-date.

the current MSYS environment I provide contains:

# Flex (2.5.4a-1)
# Bison (2.1)
# Zlib (1.2.3)
# Libpng (1.2.24)
# Libtiff (3.8.2)
# Freetype (2.3.5)
# FFTW (3.1.2)
# PDCurses (3.3)
# PROJ.4 (4.6.0)
# GEOS (2.2.3)
# GSL (1.9)
# Expat (2.0.1)
# PostgreSQL (8.2.6)
# SQLite (3.5.6)
# GDAL (1.5.1) *
# Tcl/Tk (8.5.1)

* with GEOS, Expat, PostgreSQL and SQLite support enabled

The one I'm going to upload contains all the previous libs updated to last available source code release (except for flex and bison), plus the libjpeg and libtiff with jpeg support

And ultimately I can see us providing simple binary packages on the GRASS site, while people looking for a point & click installer could be directed towards OSGeo4W. I think that would keep server disk space requirements in check, among other things.

we already have a simple point & click installer!

Regards,

Marco