[GRASS5] Re: GRASS <-> GDAL circular references

On Friday 21 May 2004 18:12, Paul Kelly wrote:

> You are right, there are OGR functions and it compiled because the
> include is in gprojects.h.
> BUT, it still works.
> libgrass_gproj must not be linked against gdal.
> The linking is done when binaries are linked with libgrass_gproj.
> I made my changes to proj and compiled/linked it without gdal.
> Then I went to general/g.proj and did a make.
> It only complains about missing CPLFree, which I changed in the last
> lines of convert.c to VSIFree( *ppszDatum ); then it worked fine.
> g.proj is linked with gdal, libgrass_gproj not, so everything should be
> fine now.
> I couldn't try if g.proj works as it should, because I don't have a
> working installation of grass57.
> Please try that.

Yes it will compile OK (though maybe not on all systems, e.g. Cygwin with
shared DLLs) but that's not what the problem was: the circular dependencies
in Radim's proposal for GDAL using GRASS shared libraies are still there
and the dilemma is still valid (and I don't have the answer to that).

But it is true that gproj does not need to be (should not be) linked with
gdal library?
In general, should be EXTRA_LIBS removed from libraries' Makefiles?

Could it be gproj staticaly linked with libgdal.a?

Radim

On Mon, 24 May 2004, Radim Blazek wrote:

On Friday 21 May 2004 18:12, Paul Kelly wrote:

Yes it will compile OK (though maybe not on all systems, e.g. Cygwin with
shared DLLs) but that's not what the problem was: the circular dependencies
in Radim's proposal for GDAL using GRASS shared libraies are still there
and the dilemma is still valid (and I don't have the answer to that).

But it is true that gproj does not need to be (should not be) linked with
gdal library?
In general, should be EXTRA_LIBS removed from libraries' Makefiles?

I used to think that they could but I don't now. Markus originally put them in so the compilation with shared libraries on Mac OSX would work. And then we found a compiler option (-undefined suppress IIRC) that worked around this.

But I have since found that they need to be there when compiling with shared libraries on Cygwin (as far as I can remember it was discussed on the list that there was no way to allow undefined symbols in Cygwin DLLs).

Could it be gproj staticaly linked with libgdal.a?

I think so yes---it wouldn't make the gproj library any bigger---it only seems to check for undefined symbols there at that stage. libgdal.a would only be linked into the binary of the module that actually used it. But it
would be linked into every module and if e.g. libgdal.a was linked into every vector module it would make things rather big.

Maybe the functions in convert.c could go into a special GRASS/GDAL interface library, something like a new version of libgrass. I only noticed after I had done most of the work on GPJ_wkt_to_grass() etc. that Frank had written some slightly similar projection information conversion functions that are in libgrass but not in any version of GRASS itself, or GDAL. I suppose the projection information conversion functions are really only needed when importing or exporting data, so they should be grouped with the other functions that do that.

Paul

On Monday 24 May 2004 15:30, Paul Kelly wrote:

But I have since found that they need to be there when compiling with
shared libraries on Cygwin (as far as I can remember it was discussed on
the list that there was no way to allow undefined symbols in Cygwin DLLs).

Should we add somehow --no-undefined to make sure that it compiles on Cygwin?
It is not standard linker flag I guess. It is probably sufficient if
developers set this flag?

> Could it be gproj staticaly linked with libgdal.a?

I think so yes---it wouldn't make the gproj library any bigger---it only
seems to check for undefined symbols there at that stage. libgdal.a would
only be linked into the binary of the module that actually used it. But it
would be linked into every module and if e.g. libgdal.a was linked into
every vector module it would make things rather big.

I don't think to link modules with libgdal.a but to link libgrass_gproj.so
with libgdal.a, that means the OSR* functions will be present in
libgrass_gproj.so only, but not in modules. libgrass_gproj.so
staticaly linked with libgdal.a has only 250KB (dynamicaly linked
with libgdal.so it has 60KB).
But OSR* probably use share/gdal/ files, which should be installed
somehow somewhere?

To compile with libgdal.a we probably need a full path to the library
produced by gdal-config (new option)?

Maybe the functions in convert.c could go into a special GRASS/GDAL
interface library, something like a new version of libgrass. I only
noticed after I had done most of the work on GPJ_wkt_to_grass()
etc. that Frank had written some slightly similar projection information
conversion functions that are in libgrass but not in any version of GRASS
itself, or GDAL. I suppose the projection information conversion functions
are really only needed when importing or exporting data, so they should be
grouped with the other functions that do that.

Do you think that at least one function which produce a projection info in
a format known by GDAL OSR functions (WKT?) could be independent of GDAL
and put e.g. in libgrass_gis.so?

Radim

Hello Radim

On Tue, 25 May 2004, Radim Blazek wrote:

On Monday 24 May 2004 15:30, Paul Kelly wrote:

But I have since found that they need to be there when compiling with
shared libraries on Cygwin (as far as I can remember it was discussed on
the list that there was no way to allow undefined symbols in Cygwin DLLs).

Should we add somehow --no-undefined to make sure that it compiles on Cygwin?
It is not standard linker flag I guess. It is probably sufficient if
developers set this flag?

As far as I know I don't think that is possible for Cygwin DLLs; the dependencies on shared libraries have to be added to the linker command-line and there seems to be no way to create DLLs with undefined symbols. In 5.3 EXTRA_LIBS is called SLIBDEPS; it seems to be a fairly standard thing and I think it is correct for it to be there?

Could it be gproj staticaly linked with libgdal.a?

I think so yes---it wouldn't make the gproj library any bigger---it only
seems to check for undefined symbols there at that stage. libgdal.a would
only be linked into the binary of the module that actually used it. But it
would be linked into every module and if e.g. libgdal.a was linked into
every vector module it would make things rather big.

I don't think to link modules with libgdal.a but to link libgrass_gproj.so
with libgdal.a, that means the OSR* functions will be present in
libgrass_gproj.so only, but not in modules. libgrass_gproj.so
staticaly linked with libgdal.a has only 250KB (dynamicaly linked
with libgdal.so it has 60KB).

Well if that worked it would be an interesting idea but not very elegant I think to have a separate GDAL for GRASS statically linked in, but as I think about it more it might reduce a lot of problems.
But I don't know if it is possible to do that and make it work. When I was trying to get 5.3 to work with shared libraries on Cygwin I tried to compile libdig2 statically and then link it into a shared libvect by placing the full path to libdig2.a along with all the .o files in the linker command. It seemed to work and the file size of libgrass_vect.so increased but running nm on it none of the symbols from libdig2.a seemed to be there. Perhaps it is a simple problem but I could not get it to work. However I suppose it is much more likely to work on real Unix rather than Cygwin.

But OSR* probably use share/gdal/ files, which should be installed
somehow somewhere?

Well the ones that are relevant to datum and projection are already there in lib/proj and are copied into $GISBASE/etc/ogr_csv when compiling. GPJ_set_csv_loc() is a finder function to return the full path to that directory, and SetCSVFilenameHook( GPJ_set_csv_loc ); is used in g.proj and libgrass_gproj to tell OGR to look there for the .csv files.

To compile with libgdal.a we probably need a full path to the library
produced by gdal-config (new option)?

Yes you could maybe parse the output of gdal-config --libs with awk or sed or something and add .a to the end but it wouldn't be very elegant.

Maybe the functions in convert.c could go into a special GRASS/GDAL
interface library, something like a new version of libgrass. I only
noticed after I had done most of the work on GPJ_wkt_to_grass()
etc. that Frank had written some slightly similar projection information
conversion functions that are in libgrass but not in any version of GRASS
itself, or GDAL. I suppose the projection information conversion functions
are really only needed when importing or exporting data, so they should be
grouped with the other functions that do that.

Do you think that at least one function which produce a projection info in
a format known by GDAL OSR functions (WKT?) could be independent of GDAL
and put e.g. in libgrass_gis.so?

Well they know PROJ.4 format and that is produced independently of those functions, but to produce good quality WKT with all the information there requires more detail than is included in the PROJ.4 description so it is really best to produce the WKT inside GRASS where it can access datum.table for the EPSG datum names and other things. The OSR functions are vital for producing WKT though as it is so complicated and couldn't be done without them.

I like the idea of the functions in convert.c becoming part of the GRASS-access component of GDAL some day.but it's not clear to me at the minute what is the best way to go.

Paul

Paul Kelly wrote:

> I don't think to link modules with libgdal.a but to link libgrass_gproj.so
> with libgdal.a, that means the OSR* functions will be present in
> libgrass_gproj.so only, but not in modules. libgrass_gproj.so
> staticaly linked with libgdal.a has only 250KB (dynamicaly linked
> with libgdal.so it has 60KB).

Well if that worked it would be an interesting idea but not very elegant I
think to have a separate GDAL for GRASS statically linked in, but as I
think about it more it might reduce a lot of problems.

Linking libgrass_gproj.so against a static libgdal.a could potentially
make the resulting library dependent upon everything on which
libgdal.a depends. And that includes C++; you might even need to use
g++ to link libgrass_gproj.so.

You would also have to know what those dependencies were, and either
list them in $(SLIBDEPS) for libgrass_gproj.so, or include them in
$(LIBES) for each module which uses libgrass_gproj.so. IIRC,
gdal-config doesn't provide this information.

My guess is that using a static GDAL library will create more problems
than it solves.

--
Glynn Clements <glynn.clements@virgin.net>

On Tuesday 25 May 2004 12:09, Paul Kelly wrote:

> Should we add somehow --no-undefined to make sure that it compiles on

> Cygwin? It is not standard linker flag I guess. It is probably sufficient
> if developers set this flag?

As far as I know I don't think that is possible for Cygwin DLLs; the
dependencies on shared libraries have to be added to the linker
command-line and there seems to be no way to create DLLs with undefined
symbols. In 5.3 EXTRA_LIBS is called SLIBDEPS; it seems to be a fairly
standard thing and I think it is correct for it to be there?

ld manpage:

--no-undefined
  Normally when creating a non-symbolic shared library, undefined symbols are allowed and
  left to be resolved by the runtime loader. These options disallows such undefined symbols.

I thought, that we could use it on Linux, to ensure that it cannot happen on Cygwin.

Radim

Hi

>> Maybe the functions in convert.c could go into a special GRASS/GDAL
>> interface library, something like a new version of libgrass. I only
>> noticed after I had done most of the work on GPJ_wkt_to_grass()
>> etc. that Frank had written some slightly similar projection
>information> conversion functions that are in libgrass but not in any
>version of GRASS> itself, or GDAL. I suppose the projection information
>conversion functions> are really only needed when importing or exporting
>data, so they should be> grouped with the other functions that do that.
>
> Do you think that at least one function which produce a projection info
> in a format known by GDAL OSR functions (WKT?) could be independent of
> GDAL and put e.g. in libgrass_gis.so?

Well they know PROJ.4 format and that is produced independently of those
functions, but to produce good quality WKT with all the information there
requires more detail than is included in the PROJ.4 description so it is
really best to produce the WKT inside GRASS where it can access
datum.table for the EPSG datum names and other things. The OSR functions
are vital for producing WKT though as it is so complicated and couldn't be

done without them.

I like the idea of the functions in convert.c becoming part of the
GRASS-access component of GDAL some day.but it's not clear to me at the
minute what is the best way to go.

I would suggest (although it's a bit crucial) to move convert.c from grass
to gdal, radims stuff, so it's only build with grass-support.
A lot of the data structures and functions have to be adapted from grass
style to gdal (if possible) or normal C.
The gdal functions are used normal as it's lib internal.
The other grass functions can be used as it's linked to libgrass_gproj
already.
As I understand you, that step wouldn't harm the grass modules which use
libgrass_gproj, so it should be fine.

Frank: What is your opinion on that?

--
MfG
  Jens Oberender
------------------------------------
Die Liebe ist ein seltsam Ding.

Hello Radim

On Wed, 26 May 2004, Radim Blazek wrote:

On Tuesday 25 May 2004 12:09, Paul Kelly wrote:

Should we add somehow --no-undefined to make sure that it compiles on

Cygwin? It is not standard linker flag I guess. It is probably sufficient
if developers set this flag?

As far as I know I don't think that is possible for Cygwin DLLs; the
dependencies on shared libraries have to be added to the linker
command-line and there seems to be no way to create DLLs with undefined
symbols. In 5.3 EXTRA_LIBS is called SLIBDEPS; it seems to be a fairly
standard thing and I think it is correct for it to be there?

ld manpage:

--no-undefined
Normally when creating a non-symbolic shared library, undefined symbols are allowed and
left to be resolved by the runtime loader. These options disallows such undefined symbols.

I thought, that we could use it on Linux, to ensure that it cannot happen on Cygwin.

Ah, I think I see what you mean now. That sounds like a good idea. I thought you were suggesting to remove EXTRA_LIBS and use --no-undefined, which didn't seem to make sense to me because it meant it could never work on Cygwin (i.e. without the EXTRA_LIBS line). Though maybe EXTRA_LIBS should
be renamed SLIBDEPS. The 5.7 build system really needs slightly updated to use the aclocal.m4 from 5.3 and this could be done at the same time, but I don't really see me having time to do this soon.

Paul