Dear List,
I tried to use i.fft in GRASS 5.3 (CVS) but only get errors like:
a) Starting FFT...
fftw: BUG in executor: invalid plan
b) Starting FFT...
Speicherzugriffsfehler (segmentation fault)
Does this module work at all? Is there a special way to use it?
I installed fftw RPMs. Is this version broken?
fftw-2.1.3-1102 ?
fftw-devel-2.1.3-1102 ?
Any hint is welcome
Otto
Otto Dassau wrote:
I tried to use i.fft in GRASS 5.3 (CVS) but only get errors like:
a) Starting FFT...
fftw: BUG in executor: invalid planb) Starting FFT...
Speicherzugriffsfehler (segmentation fault)Does this module work at all?
Yes.
Is there a special way to use it?
No.
I installed fftw RPMs. Is this version broken?
fftw-2.1.3-1102 ?
fftw-devel-2.1.3-1102 ?
Maybe. The code which actually uses FFTW is:
plan = fftw2d_create_plan(
dimc, dimr,
(i_sign < 0) ? FFTW_FORWARD : FFTW_BACKWARD,
FFTW_ESTIMATE | FFTW_IN_PLACE);
fftwnd_one(plan, data, data);
fftwnd_destroy_plan(plan);
Examining the source code of FFTW 2.1.3 indicates that the above error
message always corresponds to an invalid plan type. According to the
documentation, fftw2d_create_plan() will always return either a valid
plan or NULL (and NULL would result in a segfault).
--
Glynn Clements <glynn.clements@virgin.net>
Dear list,
On Tue, 8 Jun 2004 16:21:19 +0200 Otto Dassau <otto.dassau@gmx.de>
wrote:
Dear List,
I tried to use i.fft in GRASS 5.3 (CVS) but only get errors like:
a) Starting FFT...
fftw: BUG in executor: invalid planb) Starting FFT...
Speicherzugriffsfehler (segmentation fault)Does this module work at all? Is there a special way to use it?
I installed fftw RPMs. Is this version broken?
I can second that behavior described above. (... on debian sarge)
But I figured out, that reducing the size of a window will make i.fft
work.
My original regions, where it segfaults:
g.region -p
projection: 99 (Transverse Mercator)
zone: 0
datum: potsdam
ellipsoid: bessel
north: 5956732.48
south: 5952917.12
west: 3376777.92
east: 3384823.36
nsres: 0.32
ewres: 0.31988549
rows: 11923
cols: 25151
The region where it seems to work well:
g.region -p
projection: 99 (Transverse Mercator)
zone: 0
datum: potsdam
ellipsoid: bessel
north: 5955508.16
south: 5953765.76
west: 3378823.04
east: 3380908.48
nsres: 0.32
ewres: 0.31990183
rows: 5445
cols: 6519
Does anybody run into this problem ?
THanks for any hints
Stephan Holl
--
Stephan Holl
Check headers for GnuPG Key!
http://www.gdf-hannover.de
Stephan Holl wrote:
> I tried to use i.fft in GRASS 5.3 (CVS) but only get errors like:
>
> a) Starting FFT...
> fftw: BUG in executor: invalid plan
>
> b) Starting FFT...
> Speicherzugriffsfehler (segmentation fault)
>
> Does this module work at all? Is there a special way to use it?
> I installed fftw RPMs. Is this version broken?I can second that behavior described above. (... on debian sarge)
But I figured out, that reducing the size of a window will make i.fft
work.
My original regions, where it segfaults:
rows: 11923
cols: 25151
rows: 5445
cols: 6519
11923 * 25151 * 16 bytes (2 doubles) = 4798005968
I.e. ~4.8Gb, which is likely to significantly exceed the amount of
available memory (and, on 32-bit platforms, the amount of memory which
can be allocated to a single process, regardless of how much total RAM
or swap is available).
Actually, you would need half as much again (i.e. ~7.2Gb); i.fft reads
the entire map into an array of doubles (8 bytes per cell), but FFTW
needs an array of fftw_complex values (2 doubles, 16 bytes per cell).
More generally, maps that large won't work with anything which needs
to read the entire map into memory (which includes i.fft), unless you
have a 64-bit system and a lot of RAM (swap wouldn't be much use on
something like an FFT calculation).
--
Glynn Clements <glynn.clements@virgin.net>
Hello Glynn,
thanks for pointing this out.
On Thu, 10 Jun 2004 09:45:45 +0100 Glynn Clements
<glynn.clements@virgin.net> wrote:
Stephan Holl wrote:
> > I tried to use i.fft in GRASS 5.3 (CVS) but only get errors like:
> >
> > a) Starting FFT...
> > fftw: BUG in executor: invalid plan
> >
> > b) Starting FFT...
> > Speicherzugriffsfehler (segmentation fault)
> >
> > Does this module work at all? Is there a special way to use it?
> > I installed fftw RPMs. Is this version broken?
>
> I can second that behavior described above. (... on debian sarge)
>
> But I figured out, that reducing the size of a window will make
> i.fft work.
> My original regions, where it segfaults:> rows: 11923
> cols: 25151> rows: 5445
> cols: 651911923 * 25151 * 16 bytes (2 doubles) = 4798005968
I.e. ~4.8Gb, which is likely to significantly exceed the amount of
available memory (and, on 32-bit platforms, the amount of memory which
can be allocated to a single process, regardless of how much total RAM
or swap is available).Actually, you would need half as much again (i.e. ~7.2Gb); i.fft reads
the entire map into an array of doubles (8 bytes per cell), but FFTW
needs an array of fftw_complex values (2 doubles, 16 bytes per cell).More generally, maps that large won't work with anything which needs
to read the entire map into memory (which includes i.fft), unless you
have a 64-bit system and a lot of RAM (swap wouldn't be much use on
something like an FFT calculation).
so the only possible solution would be tiling the map into smaller
pices which fit into my RAM (768MB) and calculating the fft on the small
tiles.
Thanks
Stephan Holl
--
Stephan Holl
Check headers for GnuPG Key!
http://www.gdf-hannover.de
Stephan Holl wrote:
> > But I figured out, that reducing the size of a window will make
> > i.fft work.
> > My original regions, where it segfaults:
>
> > rows: 11923
> > cols: 25151
>
> > rows: 5445
> > cols: 6519
>
> 11923 * 25151 * 16 bytes (2 doubles) = 4798005968
>
> I.e. ~4.8Gb, which is likely to significantly exceed the amount of
> available memory (and, on 32-bit platforms, the amount of memory which
> can be allocated to a single process, regardless of how much total RAM
> or swap is available).
>
> Actually, you would need half as much again (i.e. ~7.2Gb); i.fft reads
> the entire map into an array of doubles (8 bytes per cell), but FFTW
> needs an array of fftw_complex values (2 doubles, 16 bytes per cell).
Two other points which I overlooked:
1. i.fft assumes that the fft() function requires that the array
dimensions are powers of two (that was true for the old FFT routine,
but isn't true for FFTW), and allocates memory accordingly.
2. i.fft allocates space for both the real and imaginary components,
so it's actually four doubles (32 bytes) per cell.
So, you would actually need:
16384 * 32768 * 32 bytes (4 doubles) = 17179869184 (16Gb)
for the original resolution, or:
8192 * 8192 * 32 bytes (4 doubles) = 2147483648 (2Gb)
for the reduced resolution.
BTW, the reason why you need two copies is that the original fft()
function took separate real and imaginary arrays, while FFTW requires
an array of fftw_complex structs (i.e. real and imaginary components
are interleaved).
The memory requirements could be reduced to two doubles per cell
(without scaling up to a power of two) if i.fft used FFTW directly,
rather than using the fft() interface. Or to two floats per cell if
FFTW was built to use float instead of double.
> More generally, maps that large won't work with anything which needs
> to read the entire map into memory (which includes i.fft), unless you
> have a 64-bit system and a lot of RAM (swap wouldn't be much use on
> something like an FFT calculation).so the only possible solution would be tiling the map into smaller
pices which fit into my RAM (768MB) and calculating the fft on the small
tiles.
Or, if you don't need the highest frequencies, just reduce the region
resolution.
--
Glynn Clements <glynn.clements@virgin.net>
Hello Glynn,
On Thu, 10 Jun 2004 19:08:08 +0100 Glynn Clements
<glynn.clements@virgin.net> wrote:
The memory requirements could be reduced to two doubles per cell
(without scaling up to a power of two) if i.fft used FFTW directly,
rather than using the fft() interface. Or to two floats per cell if
FFTW was built to use float instead of double.
As this seems to be an option for reducing the memory-needs a little, is
this a doable task? I think it would be benefit when FFTW is used
directly instead of fft(). Correct me when I am wrong, please...
If so, how much work would that be?
> so the only possible solution would be tiling the map into smaller
> pices which fit into my RAM (768MB) and calculating the fft on the
> small tiles.Or, if you don't need the highest frequencies, just reduce the region
resolution.
Thanks for the suggestion, as I need the resolution this is no option.
Cheers
Stephan Holl
--
Stephan Holl
Check headers for GnuPG Key!
http://www.gdf-hannover.de
Stephan Holl wrote:
> The memory requirements could be reduced to two doubles per cell
> (without scaling up to a power of two) if i.fft used FFTW directly,
> rather than using the fft() interface. Or to two floats per cell if
> FFTW was built to use float instead of double.As this seems to be an option for reducing the memory-needs a little, is
this a doable task? I think it would be benefit when FFTW is used
directly instead of fft(). Correct me when I am wrong, please...If so, how much work would that be?
Not much; the fft() function itself is trivial, and much of that is
copying the data between the separate real/imaginary arrays and the
single complex array used by FFTW, which wouldn't be necessary if you
were calling FFTW directly.
Note: the fft() function is in src/libes/gmath/fft.c.
--
Glynn Clements <glynn.clements@virgin.net>
Hello Glynn,
On Fri, 11 Jun 2004 13:10:18 +0100 Glynn Clements
<glynn.clements@virgin.net> wrote:
Stephan Holl wrote:
> > The memory requirements could be reduced to two doubles per cell
> > (without scaling up to a power of two) if i.fft used FFTW
> > directly, rather than using the fft() interface. Or to two floats
> > per cell if FFTW was built to use float instead of double.
>
> As this seems to be an option for reducing the memory-needs a
> little, is this a doable task? I think it would be benefit when FFTW
> is used directly instead of fft(). Correct me when I am wrong,
> please...
>
> If so, how much work would that be?Not much; the fft() function itself is trivial, and much of that is
copying the data between the separate real/imaginary arrays and the
single complex array used by FFTW, which wouldn't be necessary if you
were calling FFTW directly.Note: the fft() function is in src/libes/gmath/fft.c.
As I am more a GIS-user than a programmer I cannot modify
src/libes/gmath/fft.c to use FFTW directly even if fft() seems trivial.
I understand whats going on there, but nothing more...
Perhaps anybody else with more programming-skills can implement this?!
--
Stephan Holl
Check headers for GnuPG Key!
http://www.gdf-hannover.de