duggyfresh
Hi.
Any action that causes a database update (playing a song, using the Tag Editor, etc.) causes the Song entry to be moved to the end of the database file on the next database save. The location where the entry previously was in the file gets padded with spaces after the previous file's </Song> tag (usually, just under 1k of spaces). This leads to the file gradually increasing in size with every database update until there is more whitespace than data in the file.
The repair option in the program does not correct the issue. The only solution is to use a text editor to remove the spaces and return the file to it's correct size.
FYI, v7 did the same thing so v8 was probably not completely rewritten from the ground up. ;)
Regards,
Duggy.
To reproduce:
Create a new database.
Import a few songs.
Exit the program.
View the database file in a text editor and note the position of a particular song and the song immediately before it.
Run the program.
Play the song you noted above once (causing the Played count to be updated).
Exit the program.
View the database file again and find the updated record at the end.
Search for the song before the song played and note the string of space characters inserted after the previous song's </Song> tag.
Any action that causes a database update (playing a song, using the Tag Editor, etc.) causes the Song entry to be moved to the end of the database file on the next database save. The location where the entry previously was in the file gets padded with spaces after the previous file's </Song> tag (usually, just under 1k of spaces). This leads to the file gradually increasing in size with every database update until there is more whitespace than data in the file.
The repair option in the program does not correct the issue. The only solution is to use a text editor to remove the spaces and return the file to it's correct size.
FYI, v7 did the same thing so v8 was probably not completely rewritten from the ground up. ;)
Regards,
Duggy.
To reproduce:
Create a new database.
Import a few songs.
Exit the program.
View the database file in a text editor and note the position of a particular song and the song immediately before it.
Run the program.
Play the song you noted above once (causing the Played count to be updated).
Exit the program.
View the database file again and find the updated record at the end.
Search for the song before the song played and note the string of space characters inserted after the previous song's </Song> tag.
Posted Wed 15 Oct 14 @ 4:52 am
duggyfresh
Sorry: Windows 7 Ultimate, VDJ v8.0.2003 (happened in all previous versions, too)
Posted Wed 15 Oct 14 @ 4:55 am
Adion
If the white space area is big enough to fit new data, new data will be stored there instead of at the end at the file, so it will not continue growing indefintely.
Posted Wed 15 Oct 14 @ 5:30 am
duggyfresh
Interesting. Any idea why that decision was taken? It can't be for speed of rewriting the file because the amount of whitespace is not the same as the removed entry (causing rewriting of the whole file anyway).
As I understand it, the file is just there for storage of the data between program runs and the database is wholly in memory during operation. Can anyone explain the reason behind doing it like this?
Later: Just did a quick test with a folder of 40 files and the file was 20% whitespace after BPM analysis! That can't be right...
As I understand it, the file is just there for storage of the data between program runs and the database is wholly in memory during operation. Can anyone explain the reason behind doing it like this?
Later: Just did a quick test with a folder of 40 files and the file was 20% whitespace after BPM analysis! That can't be right...
Posted Wed 15 Oct 14 @ 6:06 am
Adion
When an item gets longer, and there's no room after it, it is replaced by whitespace, and the item is written at another part that has enough space (or the end if there's no such space)
So it is indeed done this way so that the entire file does not have to be rewritten when an item changes.
Reading is only done on startup, so it's not so much of an issue if there's 20% whitespace, it will not affect loading speed that much.
So it is indeed done this way so that the entire file does not have to be rewritten when an item changes.
Reading is only done on startup, so it's not so much of an issue if there's 20% whitespace, it will not affect loading speed that much.
Posted Wed 15 Oct 14 @ 7:46 am
duggyfresh
Ah, I understand. Thank you for the explanation. I did not realise until I tested it that VDJ writes the changes to disk after every updated record. I assumed it only wrote the file at program close. :)
It would be good to tidy up the whitespace as it closed the program, though. ;)
It would be good to tidy up the whitespace as it closed the program, though. ;)
Posted Wed 15 Oct 14 @ 8:17 am