[GRASS-dev] [bug #4864] (grass) integer division errors

this bug's URL: http://intevation.de/rt/webrt?serial_num=4864
-------------------------------------------------------------------------

Subject: integer division errors

Hi,

I've just fixed a bug in the symbol library where:

float_red = int_red / 255;

int/int -> int, so the answer was always 0.0 or 1.0. no good.

grep finds these ones too, maybe some are wrong, some are ok?

display/d.his/his.c: R = R * I / 255;
display/d.his/his.c: G = G * I / 255;
display/d.his/his.c: B = B * I / 255;
display/d.his/his.c: R = 127 + (R - 127) * S / 255 ;
display/d.his/his.c: G = 127 + (G - 127) * S / 255 ;
display/d.his/his.c: B = 127 + (B - 127) * S / 255 ;
general/g.pnmcomp/main.c: *r = (*r * c0 + *p * c1) / 255;
raster/r.his/his.c: R = R * I / 255;
raster/r.his/his.c: G = G * I / 255;
raster/r.his/his.c: B = B * I / 255;
raster/r.his/his.c: R = 127 + (R - 127) * S / 255 ;
raster/r.his/his.c: G = 127 + (G - 127) * S / 255 ;
raster/r.his/his.c: B = 127 + (B - 127) * S / 255 ;
raster/r.out.vtk/writeascii.c: fprintf(fp, "%lf %lf %lf \n", r / 255, g /
255, b / 255);
raster3d/r3.out.vtk/writeVTKData.c: fprintf(fp, "%.*f ", dp, (value / 255));
raster3d/r3.out.vtk/writeVTKData.c: fprintf(fp, "%.*f ", dp, (value / 255));
raster/r.out.tiff/r.out.tiff.c:#define SCALE(x) (((x)*((1L<<16)-1))/255)
raster3d/r3.showdspf/togif.c: val = (nshades*k)/255;

Hamish

-------------------------------------------- Managed by Request Tracker

Request Tracker wrote:

this bug's URL: http://intevation.de/rt/webrt?serial_num=4864

Subject: integer division errors

Hi,

I've just fixed a bug in the symbol library where:

float_red = int_red / 255;

int/int -> int, so the answer was always 0.0 or 1.0. no good.

grep finds these ones too, maybe some are wrong, some are ok?

They're all okay.

display/d.his/his.c: R = R * I / 255;
display/d.his/his.c: G = G * I / 255;
display/d.his/his.c: B = B * I / 255;
display/d.his/his.c: R = 127 + (R - 127) * S / 255 ;
display/d.his/his.c: G = 127 + (G - 127) * S / 255 ;
display/d.his/his.c: B = 127 + (B - 127) * S / 255 ;
general/g.pnmcomp/main.c: *r = (*r * c0 + *p * c1) / 255;
raster/r.his/his.c: R = R * I / 255;
raster/r.his/his.c: G = G * I / 255;
raster/r.his/his.c: B = B * I / 255;
raster/r.his/his.c: R = 127 + (R - 127) * S / 255 ;
raster/r.his/his.c: G = 127 + (G - 127) * S / 255 ;
raster/r.his/his.c: B = 127 + (B - 127) * S / 255 ;
raster3d/r3.showdspf/togif.c: val = (nshades*k)/255;
raster/r.out.tiff/r.out.tiff.c:#define SCALE(x) (((x)*((1L<<16)-1))/255)

These are all supposed to produce integer results.

raster/r.out.vtk/writeascii.c: fprintf(fp, "%lf %lf %lf \n", r / 255, g / 255, b / 255);
raster3d/r3.out.vtk/writeVTKData.c: fprintf(fp, "%.*f ", dp, (value / 255));
raster3d/r3.out.vtk/writeVTKData.c: fprintf(fp, "%.*f ", dp, (value / 255));

These all have FP numerators.

--
Glynn Clements <glynn@gclements.plus.com>