OpenMSX killed all my files!

By ro

Scribe (5061)

18-06-2023, 11:29

Accu, I am in exactly the same boat regarding wbass. cool. No hasling lwith files to get xode assembled, as all is on the machine real time in mem. Sure, short labels and limited source file capacity is evident. tho it works for me.fast and reliable.

I only use dirasdisk.to interact with external files. I emulate an HD from where I do my coding.using file exports for back up.

So, no backups?

By turbor

Hero (529)

18-06-2023, 12:17

Accumulator wrote:

Isn't it time for a additional sub-number of OpenMSX?? 17.0.19242 or 17.0.deadbeef
That means 17 and calculation of patch numbers...
To know which exact version someone is running... and which updates/patches are applied

No need for that, since it is already in there

if you run 'openmsx -v' you could get something like

openMSX 17.0
flavour: opt
components: ALSAMIDI CORE GL LASERDISC

that openMSX 17.0 tells us that you are running the official 17.0 release

If you build your own from source starting from any arbitrary non-release version you could get something like

openMSX 18.0-756-gb0b5d83ec
flavour: opt
components: ALSAMIDI CORE GL LASERDISC

which tells us you are 756 commits removed from the latest official release, and that you have build from git commit b0b5d83ec and which options you selected during build and which components the configure script found and enabled.

So with that output we can pretty much nail it down...

By Accumulator

Champion (351)

18-06-2023, 12:38

@ro, exactly!!! Do not select the home directory, if you by mistake and change drive to home dir and get that file!!! That ruined ... (the last 2 months of work)....
I have had this problem before, also using OpenMSX, with other directory, but thought was malware/virus or so....
Have setup a cheap laptop (chromebook, coreboot, linux) especially for development and coding MSX and running emulator.
Had Copied all my files,sources, gfx, and some other works from longer ago on this laptop as well.....
When I lost the data .. It would seem I have the condition of Gilles de la Tourette!! Cursing and shouting or long long time!
Now I know the reason..
A person can code and develop a lot in 2 months.... All that is gone forever, the rest on my user account as well...
Data before the last 2 months...I have... and I even make backups from backups (???!!!!!) Paranoia losing data
To say totally no backups.... made it more dramatic!! hahah!!

But is there hope I can retrieve the data from the last months???
Does OpenMSX have a 'undo directory truncate' function??

Maybe a good function to add to OpenMSX, if a directory is larger than 360720, it makes a duplicate of this dir, copies files up to 720kb and attach that directory.
Maybe even a dialog to select files upto 360/720kb, to make sure the files what made the reason to SelectThisDirAsDisk is still valid.

By DamnedAngel

Champion (286)

18-06-2023, 12:38

I could not understand from the previous posts if the problem is solved or not in version 18.
I user DirAsDisk a lot, although in a very few fixed and specific directories, but since it is not uncommon from me do to sh*tty things, it would be nice to know if the feature is now safe.

@Accumulator, could you reproduce it on v18?

By Manuel

Ascended (19678)

18-06-2023, 15:30

Accumulator wrote:

hint: put plenty of files in the home directory, MSX and non MSX, create some dirs, plemp there some files.
just run: wbass2 from a premade directory, use menu in active openmsx screeen change drive A (DirAsDisk) to ( ~ ), afterwards exit menu screen , at prompt do a dir "a:*.*" , dir "b:*.*" , and an dir "a:*.*" and load to, or save from wbass2, from home directory and I was as result almost throwing my pc to the wall!!

Yes I was using Wbass2 when this occurred, the file .asm file saving or loading (not compiled), was about 2.3Kb
I accidentally put the asm file in the home directory and in stead of copying to the right dir, I changed drive A: to the home directory...

Just checked.... Changing directory is switch A: B: A: to get results.

Alright, I can reproduce it, mostly.

prepare a large folder with all kinds of files. E.g. a copy of a homedir or whatever. It must be large and must have small files, files larger than 1kB but smaller than 720kB (in other words: several files larger than 1kB that fit on a MSX disk) and can have subdirs with such files as well. Let's call this dir /tmp/testdir. Also put in a loadable ASM file starting with letter "A" (e.g. AWBASM21.ASM") so that it will be included in the virtual disk image.
start openMSX 17.0 with wbass2 in a directory (get at https://wbsoft.home.xs4all.nl/msx/wbass2/wbass2.zip)
type in MSX: DIR "A:*.*" - this shows the WBASS2 files
type in openMSX console: diska /tmp/testdir to change to the large test dir.
type in MSX: DIR "A:*.*" - this STILL shows the WBASS2 files
type in MSX: DIR "B:*.*" - and press just Enter when MSX requests to insert a disk for drive B:. This lists several files from the testdir
type in MSX: DIR "A:*.*" - and press just Enter when MSX requests to insert a disk for drive A:. This still shows the files from the testdir
type in MSX: LOAD "AWBASM21.ASM" - this gave a disk I/O error (but the file was at least partially loaded)
at this point, there wasn't any corruption visible at all yet on the host side!
type in MSX: save "test.asm" - this gave no error

At this point, the corruption has taken place. The files that were visible on the MSX side that were larger than 1kB were truncated to 1kB. Smaller files seem to be untouched, except for their timestamps. The effect was indeed also visible in some subdirectories. But in my case (with dozens of subdirectories) only 2 were affected.
Note that not all files in the testdir were affected. Apparently only the files that were accepted to fit on the virtual disk image. So not too large files and also when the disk was full, the remaining files were not added and apparently then not affected.

The problem that the files are not visible when switching to one dir to another dir is probably the issue I talked about earlier. So that is then probably fixed in 18.0. (EDIT: I can confirm that.)

Now, let's see whether this also happens with 18.0. Will report back.

By Manuel

Ascended (19678)

18-06-2023, 15:28

Results with 18.0:

type in MSX: DIR "A:*.*" - this shows the WBASS2 files
type in openMSX console: diska /tmp/testdir to change to the large test dir.
type in MSX: DIR "A:*.*" - this now shows the testdir files
type in MSX: LOAD "AWBASM21.ASM" - this took a long time, before it gave an out of memory. But the file was partially loaded.
type in MSX: save "test.asm" - this gave a Disk full error.

No corruption in this scenario on 18.0.

Now, including the drive A/B change:

type in MSX: DIR "A:*.*" - this shows the WBASS2 files
type in openMSX console: diska /tmp/testdir to change to the large test dir.
type in MSX: DIR "A:*.*" - this shows the /tmp/testdir files
type in MSX: DIR "B:*.*" - and press just Enter when MSX requests to insert a disk for drive B:. This lists several files from the testdir
type in MSX: DIR "A:*.*" - this still shows the files from the testdir
type in MSX: LOAD "AWBASM21.ASM" - this took a long time, before it gave an out of memory. But the file was partially loaded.
type in MSX: save "test.asm" - this gave a Disk full error.

Still no corruption at all.

So, I think this issue has indeed been fixed in openMSX 18.0.

By Manuel

Ascended (19678)

18-06-2023, 16:43

So, to me, it looks like you can only get corruption if you save data to the dir-as-disk that is (too) large on openMSX 17.0 on a machine that relies on the disk change signal (like most Sony machines, but not Philips machines).

My guess is indeed that the corruption bug was introduced in openMSX 17.0 (via b0b3d6bcd123402a3c5abb8d451f66c56480d2c6) and fixed in 18.0 via 0d342a9d17eff9451561928463e3dc47d015b2af.

The files not visible bug is most likely the same as issue #1410 as mentioned before.

EDIT: I fixed that bug separately on 17.0 source code, but that still triggered it. So it's not that alone. Perhaps it's the combination of the two, but that's a bit tricky to try out separately. At least all is OK in 18.0. EDIT: confirmed that when using a machine without disk change signal (e.g. Philips NMS 8250) the issue does not occur with this fixed 17.0 binary.

By Accumulator

Champion (351)

18-06-2023, 17:29

@DamnedAngel,
I downloaded the latest release from Github, openmsx and openmsx catapult, the tar.gz files. (As the repository of Canonical and Debian stable do not supply 18.0, only in testing/unstable/sid)
Installed all dependencies, build both applications, which took some time, removed version 17.0 (openmsx, openmsx-catapult and openmsx-debugger),
installed 18.0.... and created a new directory called 'kill'
In that directory I plemped/copied random directories and files, next I opened msx-catapult and run a valid DirAsDisk directory with WBASS2...
Once running I switched the Drive A (DirAsDisk) to directory named 'kill", and surprisingly I got notifications like 'Disk Full', 'Disk Full', 'Virtual Disk Image full: xxxxxx.xxx truncated', 'Virtual Disk Image full: xxxxxx.xxx truncated', 'File too large: xxxxxxxxx.xxx', 'Virtual Disk Image full: xxxxxx.xxx truncated', 'File too large: xxxxxxxxx.xxx', 'Virtual Disk Image full: xxxxxx.xxx truncated', 'File too large: xxxxxxxxx.xxx'
Of course my first thought was: Damned, F#ck, again truncated.......
So I loaded and saved some sources made by Wbass2.

One of the proverbs of our dear Russian friends is: "Doveryai, no proveryai" or ”trust, but verify”.....

So I checked all files and sub-directories in the 'kill' directory and..... "Yippee Ki-Yay, M0th3rfuck3r"
No files truncated, no files changed, the 'truncated' message you cannot trust .. and can be deleted, or changed into 'File too large: xxxx'

This problem is resolved in 18.0, and I strongly recommend everyone should upgrade as soon as possible to 18.0!!!

By taupter

Supporter (3)

18-06-2023, 21:00

It would be nice to have 0.18 running on Android. The version available at the Play Store is still 0.14.

By turbor

Hero (529)

18-06-2023, 21:04

Those 'Virtual Disk Image full: xxxxxx.xxx truncated' Simply indicate that the file doesn't fit on the virtual 720KB floppy, so they are truncated in the MSX environemnt, not on the host OS (yet).

Problems arises when you write to the (full when created) virtual disk.
At that point the DirAsDisk code exports all/most of the MSX files that share a dir-sector or FAT-sector with the file you are writing in your MSX session. At that point those impact files are exported from the virtual disk back to your host OS but WITH THEIR TRUNCATED MSX STATE.
So as soon as you see those 'truncated warnings' you can read from the virtual floppy (and get truncated files since they didn't fit on the 720KB) but the problems arise when you write to such a floppy. Even if you made space on the virtual floppy by deleting some files , those truncated imports will still be too short in your MSX environment and thus screw up your host files when writing.