[GRASSLIST:8827] major data loss bug with i.points!

Hi all, I just experienced some VERY weird behavior using grass 6.1cvs. The end result is that somehow GRASS erased an entire location (several 100's of MB), without much prompting by me. Here is how it happened.

I was using i.points in an xy location when I mistakenly resized the i.points terminal window as I was entering points. The terminal window looked a little funky (mishapen i.points display), and I was presented with a grass prompt, instead of the normal i.points display. My cursor no longer responded on the monitor, though it still displayed the typical 'cross' cursor from i.points. I tried to control-c to kill i.points, but my cursor would not change. Everytime I did this the grass prompt would repeat itself without a carriage return such that it looked like:

GRASS 6.1.cvs (xy):~ > GRASS 6.1.cvs (xy):~ > GRASS 6.1.cvs (xy):~ >

rather than

GRASS 6.1.cvs (xy):~ >

Carriage returns yielded the same behavior. I decided things looked a little screwy, so I typed exit. Grass seemed to quit in a normal way (D.m disappeared along with the terminal). Upon immediately restarting Grass and firing up i.points again, I discovered that I had lost my target location! This included ALL data in the target location (a few hundred megabytes, many hours of work). I have searched thoroughly on my system, and it is all gone (not in the trash either and I haven't emptied it). My XY location is still there, but the target location has disappeared. The target files of the imagery groups all still point to the lost location. This all happened in a span of 2 minutes. Out of fear I have not attempted to recreate this scenario.

The last commands from my .bash_history file don't reveal much:
i.points
fg
exit
i.points
I tried to see if i.points was running in the background after its initial weird behavior.

I feel pretty comfortable with grass (regular user for 4 years), and this is the first time I have ever experienced anything remotely similar to this. It definitely scares me as I have a lot of data I don't care to lose. Anyone have any ideas out there about this? Developers? Is this something that has happened before with a cvs version?

I have been very happy with GRASS, but a bug like this really makes me question my use of the program. I know one should make regular backups, but this is very bothersome.

-Ian

Mac OS 10.3.9 (latest update), GRASS 6.1cvs from Lorenzo (source Oct. 22, 2005)

-------------------------------------------------------------
This message has been scanned by Postini anti-virus software.

Hi all, I just experienced some VERY weird behavior using grass
6.1cvs.

Hi Ian,

I though I'd try and add something to this thread, although I suspect
we'll never really know the cause.

  The end result is that somehow GRASS erased an entire location
(several 100's of MB), without much prompting by me. Here is how it
happened.

As far as I am aware GRASS contains no code at all anywhere in it to
intentionally remove an entire database, location, or mapset. There was
an unfortunate bug in a script that deleted someone's home directory
once when a temporary directory was not created properly ("cd $tmp ; rm
-f ${tmp}*" when the setting of $tmp failed [ouch, terribly sorry about
that]). But nothing like you describe has been reported before AFAIK.

Certainly it is a serious issue whenever hours of work and important
data is lost, at minimum it is a sobering reminder that computers are
complex beasts and random bad things do happen from time to time.

I was using i.points in an xy location when I mistakenly resized the
i.points terminal window as I was entering points. The terminal
window looked a little funky (mishapen i.points display), and I was
presented with a grass prompt, instead of the normal i.points
display. My cursor no longer responded on the monitor, though it
still displayed the typical 'cross' cursor from i.points.

if the terminal was smaller than 80x24 the i.points text may have done
weird things when it tried to draw the ascii "boxes". See bug #1107
  https://intevation.de/rt/webrt?serial_num=1107

I tried to control-c to kill i.points, but my cursor would not
change. Everytime I did this the grass prompt would repeat itself
without a carriage return such that it looked like:

GRASS 6.1.cvs (xy):~ > GRASS 6.1.cvs (xy):~ > GRASS 6.1.cvs
(xy):~ >

rather than

GRASS 6.1.cvs (xy):~ >
GRASS 6.1.cvs (xy):~ >

Carriage returns yielded the same behavior. I decided things looked a
little screwy, so I typed exit. Grass seemed to quit in a normal way
(D.m disappeared along with the terminal).

I have found that this same kind of terminal corruption has happened to
me from time to time over a ssh connection to another machine on the
network. If a lot of random crap gets sent to the display (e.g. "cat
binaryfile"), one of the million monkey strings gets interpreted as a
command for the terminal program to do something and things get weird.
Closing the terminal (or leaving it open with just "logout" from the
remote machine) gets rid of the problem. This is not a GRASS specific
error, but rather input handling issue in the terminal program (unless
of course a GRASS program spews out binary data to stdout).

The prompt losing newline capability has been pretty common to me,
especially, but not exclusively, when using the RXVT term.

I'm sure others have had the experience of pasting a large text document
to the terminal by mistake too.

Upon immediately restarting Grass and firing up i.points again, I
discovered that I had lost my target location! This included ALL
data in the target location (a few hundred megabytes, many hours of
work). I have searched thoroughly on my system, and it is all gone
(not in the trash either and I haven't emptied it). My XY location

Stuff only shows up in the trash that you have dragged there. Anything
deleted on the command prompt (MS Windows too) or by the C unlink()
command will not be saved to the trash.

is still there, but the target location has disappeared. The target
files of the imagery groups all still point to the lost location.
This all happened in a span of 2 minutes. Out of fear I have not
attempted to recreate this scenario.

seems like a valid response.

The last commands from my .bash_history file don't reveal much:
i.points
fg
exit
i.points
I tried to see if i.points was running in the background after its
initial weird behavior.

Since i.points is mentioned, we should probably raise the priority of
fixing this bug, if only as a precaution.

"i.points: error in tmp file handling"
  https://intevation.de/rt/webrt?serial_num=3798

i.points has:
imagery/i.points/main.c: unlink (tempfile1);
imagery/i.points/main.c: unlink (tempfile2);
imagery/i.points/main.c: unlink (cell_list);
imagery/i.points/main.c: unlink (group_list);
imagery/i.points/main.c: unlink (vect_list);
imagery/i.points/main.c: unlink (digit_points);
imagery/i.points/main.c: unlink (digit_results);

if the tempfile (or other) string got corrupted via this bug, it is
possible (but I'd say very unlucky) that the resultant string would be a
valid and exact pathname pointing to your mapset with the tempfile name
missing, and that is where things went wrong..? The left over garbage
might have corrupted the terminal prompt too?

I feel pretty comfortable with grass (regular user for 4 years), and
this is the first time I have ever experienced anything remotely
similar to this. It definitely scares me as I have a lot of data I
don't care to lose. Anyone have any ideas out there about this?
Developers? Is this something that has happened before with a cvs
version?

I think you are the only unlucky one so far.. hopefully the last.

I have been very happy with GRASS, but a bug like this really makes me
question my use of the program. I know one should make regular
backups, but this is very bothersome.

I think over 23 years the core code of GRASS has become pretty stable,
unfortunately sometimes new bugs do get inadvertently added and some old
bugs are only triggered under very rare circumstances. It just sucks
when you happen to be the one that finds one...

Mac OS 10.3.9 (latest update), GRASS 6.1cvs from Lorenzo (source Oct.
22, 2005)

suggestions for all:

fix the above two bugs starting with #3798. Bug 1107 may be a "won't
fix; replace the program's interactive functionality with something
better."

audit lib/init/clean_temp.c and other modules that call unlink(),
scripts that use "rm -f".

An old & important blue-water sailing rule, but I think it applies:
It is best to take action (eg backup your data) when the thought first
crosses you mind.

Hamish

On Sun, 20 Nov 2005 18:20, Hamish wrote:

> I tried to control-c to kill i.points, but my cursor would not
> change. Everytime I did this the grass prompt would repeat itself
> without a carriage return such that it looked like:
>
> GRASS 6.1.cvs (xy):~ > GRASS 6.1.cvs (xy):~ > GRASS 6.1.cvs
> (xy):~ >
>
> rather than
>
> GRASS 6.1.cvs (xy):~ >
> GRASS 6.1.cvs (xy):~ >
>
> Carriage returns yielded the same behavior. I decided things looked a
> little screwy, so I typed exit. Grass seemed to quit in a normal way
> (D.m disappeared along with the terminal).

I have found that this same kind of terminal corruption has happened to
me from time to time over a ssh connection to another machine on the
network.

The "reset" command (no arguments needed) will usually tidy up these problems.

Regards
Gordon

--

Gordon Keith
Programmer/Data Analyst
Marine Acoustics
CSIRO Marine and Atmospheric Research
http://www.cmar.csiro.au

The one thing we learn from history is that we don't learn from history.
-- Albert Einstein

Hamish wrote:

> Carriage returns yielded the same behavior. I decided things looked a
> little screwy, so I typed exit. Grass seemed to quit in a normal way
> (D.m disappeared along with the terminal).

I have found that this same kind of terminal corruption has happened to
me from time to time over a ssh connection to another machine on the
network. If a lot of random crap gets sent to the display (e.g. "cat
binaryfile"), one of the million monkey strings gets interpreted as a
command for the terminal program to do something and things get weird.
Closing the terminal (or leaving it open with just "logout" from the
remote machine) gets rid of the problem. This is not a GRASS specific
error, but rather input handling issue in the terminal program (unless
of course a GRASS program spews out binary data to stdout).

Problems with the state of the terminal itself (e.g. selecting the
line-graphics character set) can usually be fixed with the "reset"
command.

Problems with the kernel's terminal driver (e.g. CR/LF handling,
echoing etc) can often be fixed with "stty sane".

--
Glynn Clements <glynn@gclements.plus.com>