Bookmark backups

Users and developers helping users with generic and technical Pale Moon issues on all operating systems.

Moderator: trava90

Forum rules
This board is for technical/general usage questions and troubleshooting for the Pale Moon browser only.
Technical issues and questions not related to the Pale Moon browser should be posted in other boards!
Please keep off-topic and general discussion out of this board, thank you!
Roger_Fink
Moongazer
Moongazer
Posts: 13
Joined: 2018-03-03, 15:17

Bookmark backups

Unread post by Roger_Fink » 2022-05-22, 04:47

Operating system:Win10
Browser version:31.0.0
32-bit or 64-bit browser?:64
Problem URL:
Browser theme (if not default):Past Modern revisited (latest)
Installed add-ons:n/a
Installed plugins: (about:plugins):n/a

If possible, please include the output of help->troubleshooting information (as text):
**PASTE troubleshooting information here**
I recently upgraded to the latest version of Pale Moon, 31.0.0. Apparently as a result of this, I have lost my last ten bookmark backups, which means all of them. This has never happened before and I am a long time user. The backup shown in the screenshot is the current one. I make weekly back-ups but the last one was new enough to exhibit this error. Is this a bug? Can I get them back?
Attachments
screenshot bookmarks.jpg

Roger_Fink
Moongazer
Moongazer
Posts: 13
Joined: 2018-03-03, 15:17

Re: Bookmark backups

Unread post by Roger_Fink » 2022-05-22, 07:18

Adding further to my post above:
I just did a system restore that should have reverted me back to a bookmark with an earlier date than the one shown in the screenshot above. Instead it showed the current bookmark version per the screenshot below. I've performed this operation many times, and this is not the normal result. Bookmarks are not protected data like documents or email. In fact restoring an earlier bookmark folder would be a typical intended use for system restore, though probably not anybody's first choice as a way to achieve it.
Attachments
back-up after system restore.jpg

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35477
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Bookmark backups

Unread post by Moonchild » 2022-05-22, 08:06

I'm not sure what to say other than that it seems to work fine here, and that this part of the code has not at all been changed across versions 29, 30 and 31. See attached screenshot.
If it wasn't part of your system backup (that system restore uses) and you haven't made backups of the browser profile otherwise (highly recommended) then I don't think you can get the lost files back.

If it keeps just making a single bookmark backup instead of the default (10), then please check about:config for the preference browser.bookmarks.max_backups -- it should be set to default 10.
Also, check if you have any legacy extensions that might be handling bookmarks or backups.
Attachments
bmbkup1.png
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

Blacklab
Board Warrior
Board Warrior
Posts: 1080
Joined: 2012-06-08, 12:14

Re: Bookmark backups

Unread post by Blacklab » 2022-05-22, 12:24

Just an outside possibility...

If you happen to use CCleaner then I noticed that they had added 'Bookmarks Backup' to the list of items cleaned in Firefox about 2 versions ago... and I find that whatever file paths CCleaner 'cleans' in Windows OS still affects/cleans Pale Moon too:

CCleaner.jpg
CCleaner.jpg (29.93 KiB) Viewed 1727 times

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35477
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Bookmark backups

Unread post by Moonchild » 2022-05-22, 12:27

Blacklab wrote:
2022-05-22, 12:24
If you happen to use CCleaner then I noticed that they had added 'Bookmarks Backup' to the list of items cleaned in Firefox about 2 versions ago...
Oh wow. Let's just delete people's backups out of a sense of "privacy"... way to go :P
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

Roger_Fink
Moongazer
Moongazer
Posts: 13
Joined: 2018-03-03, 15:17

Re: Bookmark backups

Unread post by Roger_Fink » 2022-05-22, 15:29

"If you happen to use CCleaner then I noticed that they had added 'Bookmarks Backup' to the list of items cleaned in Firefox about 2 versions ago... "
"Just an outside possibility..."

Not such an outside possibility. Maybe as an added security measure they could just delete the whole profile.
Avast, the C-Cleaner overlords, was the cause of the problem. They shoulda told me. I unchecked the box.

What a stupid optional "improvement". After ten automatic backups (or whatever the user sets it to in config) it cleans itself.

Blacklab
Board Warrior
Board Warrior
Posts: 1080
Joined: 2012-06-08, 12:14

Re: Bookmark backups

Unread post by Blacklab » 2022-05-22, 16:17

The really stupid thing CCleaner did was add that new 'Bookmarks Backup' option already 'ticked' and running when you updated to the new CC version... sigh! Idiots. :wtf: I mean, who reads the 'version history' updates list? :)

Always worth taking occasional backups of Bookmarks and Profile IMO...

Bookmarks Only Backup - via Library export of HTML file: Pale Moon (App Menu) button > Bookmarks > Organize Bookmarks > Library page opens > at top is 'Import and Export' dropdown menu button > select 'Export Bookmarks as HTML...' AND after choosing download location WAIT... if you have a lot of Bookmarks the export file will take a few moments to generate and appear!

Profile Backup including Bookmarks: https://www.palemoon.org/backuptool.shtml

PS. If you do use CCleaner save yourself the bother of saying 'NO' to the added options and c***ware... use their own 'Slim' version which is only available from their 'Builds' page: https://www.ccleaner.com/ccleaner/builds

Roger_Fink
Moongazer
Moongazer
Posts: 13
Joined: 2018-03-03, 15:17

Re: Bookmark backups

Unread post by Roger_Fink » 2022-05-22, 16:38

I figured that since I back up the entire profile to a flash drive once a week, I could never run into a problem I couldn't wiggle out of. Pre-checking a new item seems like a unintended mistake rather than malevolence, but given some of their past shenanigans, you never know. C-Cleaner is not all bad IMO and I don't ask it to do too much. If C-C slim can perform the tasks I need I'll try it out. Thnx.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35477
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Bookmark backups

Unread post by Moonchild » 2022-05-22, 16:48

Roger_Fink wrote:
2022-05-22, 16:38
Pre-checking a new item seems like a unintended mistake rather than malevolence
I'm 100% sure that removing backups wasn't a "mistake" per se, since the feature was added on purpose, obviously. Malevolence? probably not...
Not sure what to call it instead though. "stupid arrogance" perhaps, or "desperately trying come up with improvements when it's already doing a good job"? I mean... in what way would removing bookmark backups be in any way a good thing if they'd be leaving the places database intact otherwise?
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

van p
Astronaut
Astronaut
Posts: 592
Joined: 2015-11-19, 07:15
Location: Cincinnati, OH, U.S.A.

Re: Bookmark backups

Unread post by van p » 2022-05-23, 00:22

Roger_Fink wrote:
2022-05-22, 15:29
"If you happen to use CCleaner then I noticed that they had added 'Bookmarks Backup' to the list of items cleaned in Firefox about 2 versions ago... "
I have v5.91 and that "feature" isn't there, so apparently it's not from "about 2 versions ago," but the new v6. When I update CCleaner, I go through all the settings to make sure nothing bad is happening.

Off-topic:
I'm thinking of leaving CCleaner and going with maybe PrivaZer or BleachBit. Any advice/comments/opinions? Thanks.
Windows 10 Pro x64 v22H2 8GB i5-4570|Pale Moon v33.0.1 x64

Blacklab
Board Warrior
Board Warrior
Posts: 1080
Joined: 2012-06-08, 12:14

Re: Bookmark backups

Unread post by Blacklab » 2022-05-23, 00:33

FWIW - the CCleaner 'Bookmarks Backup' option was added in CC v5.92 issued on 11 Apr 2022 :)

CCleaner v5.92 changes (extract).jpg
CCleaner v5.92 changes (extract only)

Full CCleaner version history here: https://www.ccleaner.com/ccleaner/version-history

User avatar
tellu-white
Moon lover
Moon lover
Posts: 83
Joined: 2022-03-08, 22:02

Re: Bookmark backups

Unread post by tellu-white » 2022-05-23, 00:43

Roger_Fink wrote:
2022-05-22, 16:38
I figured that since I back up the entire profile to a flash drive once a week, I could never run into a problem I couldn't wiggle out of.
I made an add-on that uses only Javascript code to make an automatic "Bookmarks Backup" when PaleMoon is closed.

The add-on copies the files below to the Desktop, in the "Bookmarks Backup" folder:

places.sqlite
places.sqlite-shm
places.sqlite-wal

The "Bookmarks Backup" folder is created by the add-on when PaleMoon is first closed.

It seems to work well. You can download it here:

https://www.mediafire.com/file/qn264s72 ... e.zip/file

Try it and see if it's ok for you.
backup_bookmarks_on_palemoon_close.png
backup_bookmarks_on_palemoon_close.png (23.26 KiB) Viewed 1510 times

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35477
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Bookmark backups

Unread post by Moonchild » 2022-05-23, 07:29

tellu-white wrote:
2022-05-23, 00:43
I made an add-on that uses only Javascript code to make an automatic "Bookmarks Backup" when PaleMoon is closed.
I'm afraid you've gone about it the wrong way.
  • places.sqlite contains a lot more than just bookmarks.
  • You're making copies of an sqlite database in a "dirty" state. This may not be recoverable (-shm and -wal are db "log" files used to update the final database when closed)
Pale Moon (mind the space in the name, please) already makes bookmark backups, so why reinvent the wheel?
There's also browser.bookmarks.autoExportHTML to export bookmarks to HTML automatically, which may be useful.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
tellu-white
Moon lover
Moon lover
Posts: 83
Joined: 2022-03-08, 22:02

Re: Bookmark backups

Unread post by tellu-white » 2022-05-23, 13:53

Moonchild wrote:
2022-05-23, 07:29
I'm afraid you've gone about it the wrong way.
By what I will say next, I do not intend to contradict you. Your knowledge of the subject is far greater than mine. For me, making add-ons is a hobby (which started from a desire to solve some problems). Besides, I'm of an age where people are walking their grandchildren in the park, so I don't expect a bright future in programming :) With that said, I'm going to present below the findings I've made, over time.

I've been using Pale Moon for several years now. Before that I used Firefox, also for many years. During this time, the number of "Bookmarks" kept growing until I ended up with the "places.sqlite" file now being 120 MB. Probably this is the reason why, for some time, the process "palemoon.exe" persists for a long time (in "Task Manager / Processes"), and it is frequent cases where this situation ends with the error message "Pale Moon web browser has stopped working".
pale_moon_web_browser_has_stopped_working.png
pale_moon_web_browser_has_stopped_working.png (17.09 KiB) Viewed 1329 times
Through tests, I noticed that this problem disappears permanently if I delete the "places.sqlite" file (120 MB), which is replaced by Pale Moon with a 10 MB file (without Bookmarks).

When I close Pale Moon after the error message "Pale Moon web browser has stopped working", all three "places.sqlite" files remain in the "Profiles" folder («sqlite database in a "dirty" state»). Although this situation has been repeated many times, I have never had any "Bookmark" missing.

The add-on I made saves the three "places.sqlite" files as soon as I press Pale Moon's "Close" button, before they are accessed by Pale Moon «(-shm and -wal are db "log" files used to update the final database when closed)». I found this visually, in a "file manager" that does not automatically sort modified (or added) files. This led me to assume that Pale Moon accesses these files only after the code in the add-on has already been executed: ".observe = function(event){...}". In this case, these files are not corrupted and Pale Moon should treat them as it treats them if it closes with the error "Pale Moon web browser has stopped working" (in which case I have never had any "Bookmark" missing). I have also tested this assumption ("no file corruption") and so far it has not been refuted.

I repeat what I said at the beginning of the post: I do not intend to contradict you. On the other hand, as long as the add-on doesn't corrupt the "places.sqlite" files, an additional method to mitigate the risk of losing Bookmarks doesn't hurt, especially since it is automated, being executed every time I close Pale Moon - no need to worry that I forgot to backup manually :)

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35477
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Bookmark backups

Unread post by Moonchild » 2022-05-23, 14:17

Maybe the fact that you've been accessing the files without a lock while still opened has been the reason your browser has been unstable.
By the way, 120MB is not a problem for sqlite, at all; it can handle much larger databases. However, if you have data corruption in it, then that might be a different story altogether.

You may want to go in and go to help -> troubleshooting information and find the database integrity check and run that.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
tellu-white
Moon lover
Moon lover
Posts: 83
Joined: 2022-03-08, 22:02

Re: Bookmark backups

Unread post by tellu-white » 2022-05-23, 16:16

Thank you for your reply and suggestion.
Moonchild wrote:
2022-05-23, 14:17
Maybe the fact that you've been accessing the files without a lock while still opened has been the reason your browser has been unstable.
Pale Moon had the same problem ("Pale Moon web browser has stopped working") before making this add-on. From this I conclude that it is not this add-on that created the problem. Since I have many (too many :) ) add-ons installed (Extensions + CustomButtons), I'll have to see if there is another add-on (or several) that causes this error. This will take some time.

However, it is interesting that this problem disappears if I delete the "places.sqlite" file (120 MB), which is replaced by Pale Moon with a 10 MB file (without Bookmarks). Wouldn't this problem persist if it was generated by an add-on?!

The database seems to be OK:
database_integrity_check.png
database_integrity_check.png (5.58 KiB) Viewed 1288 times

Code: Select all

Integrity 	

> Integrity check
+ The database is sane
> Coherence check
+ The database is coherent
> Orphans expiration
+ Database cleaned up
> Vacuum
Initial database size is 122880 KiB
+ The database has been vacuumed
Final database size is 122880 KiB
> Statistics
Database size is 122880 KiB
user_version is 35
page_size is 32768
cache_size is -2048
journal_mode is wal
synchronous is 1
History can store a maximum of 110008 unique pages
Table moz_places has 36949 records
Table moz_historyvisits has 19 records
Table moz_inputhistory has 1 records
Table moz_hosts has 7926 records
Table moz_bookmarks has 77361 records
Table moz_keywords has 0 records
Table sqlite_sequence has 0 records
Table moz_favicons has 8069 records
Table moz_anno_attributes has 6 records
Table moz_annos has 2 records
Table moz_items_annos has 1449 records
Table sqlite_stat1 has 15 records
...

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35477
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Bookmark backups

Unread post by Moonchild » 2022-05-23, 17:40

tellu-white wrote:
2022-05-23, 16:16
Pale Moon had the same problem ("Pale Moon web browser has stopped working") before making this add-on. From this I conclude that it is not this add-on that created the problem. Since I have many (too many :) ) add-ons installed (Extensions + CustomButtons), I'll have to see if there is another add-on (or several) that causes this error. This will take some time.

However, it is interesting that this problem disappears if I delete the "places.sqlite" file (120 MB), which is replaced by Pale Moon with a 10 MB file (without Bookmarks). Wouldn't this problem persist if it was generated by an add-on?!
It depends on how old your places database is. If you carried it forward from Firefox or many incarnations of Pale Moon with installed/uninstalled extensions, and especially if it was copied across different operating systems, then it's quite possible there is corrupt content in it making the browser crash, even if the database has no structural issues (which is what the integrity check can deal with and would be a quick, non-invasive fix). In that case I suggest killing it, recreating an empty one and then restoring a bookmark backup from inside the "organize bookmarks" window.

Note: you can easily recreate corrupt structure or content by manipulating files that are "in use" so I still strongly discourage making "backups" in the way you're doing with the extension. as said it is possible those backups won't even be restorable, to begin with, or have corrupt data, if the files were copied in the middle of an update.
If you want to interact with the places database, then you should do so through the sqlite API. In the case of bookmarks you can even use a much simpler interface as well from within the browser to make backups.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
tellu-white
Moon lover
Moon lover
Posts: 83
Joined: 2022-03-08, 22:02

Re: Bookmark backups

Unread post by tellu-white » 2022-10-24, 21:27

tellu-white wrote:
2022-05-23, 13:53
for some time, the process "palemoon.exe" persists for a long time (in "Task Manager / Processes"), and it is frequent cases where this situation ends with the error message "Pale Moon web browser has stopped working".
tellu-white wrote:
2022-05-23, 13:53
I noticed that this problem disappears permanently if I delete the "places.sqlite" file (120 MB), which is replaced by Pale Moon with a 10 MB file (without Bookmarks).
After my last post here it happened more and more often that Pale Moon closed with the error message "Pale Moon web browser has stopped working" (finally, almost every time I closed the browser). I didn't want to resort to creating a new database (i.e. not having the old bookmarks loaded in Pale Moon) so I looked for other solutions. So, I found a code in the head_common.js file:

Pale Moon source code\UXP-PM28.8.3_Release\toolkit\components\places\tests\head_common.js

- code that I inserted in the Backup Bookmarks on Pale Moon Close add-on before the code that backs up the places.sqlite files. This way the problem you mentioned disappeared:
Moonchild wrote:
2022-05-23, 07:29
You're making copies of an sqlite database in a "dirty" state.
Moonchild wrote:
2022-05-23, 17:40
you can easily recreate corrupt structure or content by manipulating files that are "in use" so I still strongly discourage making "backups" in the way you're doing with the extension.
Moonchild wrote:
-shm and -wal are temporary files for open databases. They will be folded back into the main .sqlite upon shutdown.

https://forum.palemoon.org/viewtopic.php?f=3&t=24336#p189050
Remarks:

The temporary files places.sqlite-shm and places.sqlite-wal have already been merged into the main places.sqlite file (and then deleted) by the code taken from head_common.js - BEFORE executing the code that saves the places.sqlite file. I tested this by keeping the code that also saves the places.sqlite-shm and places.sqlite-wal files (using "try ... catch"). The result: only the places.sqlite file will be found in the desktop backup folder, but NOT the places.sqlite-shm and places.sqlite-wal files.

I've been testing this new version of the Backup Bookmarks on Palemoon Close add-on for more than two weeks now, and since then it has never (ever) happened that Pale Moon closes with the error message "Pale Moon web browser has stopped working" (and I haven't lost any bookmarks during this period). Also, the database remained OK (uncorrupted):
The database is sane - The database is coherent - Database cleaned up - The database has been vacuumed.png
The database is sane - The database is coherent - Database cleaned up - The database has been vacuumed.png (5.67 KiB) Viewed 864 times
I would be grateful if you could tell me what you think about this solution. Below is the code from the head_common.js file:

Code: Select all


waitForConnectionClosed(aCallback);

function waitForConnectionClosed(aCallback)
{
	promiseTopicObserved("places-connection-closed").then(aCallback);
	shutdownPlaces();
}

function aCallback()
{
	// alert("Places Connection Closed");
}

function promiseTopicObserved(aTopic)
{
	return new Promise(resolve => {
		Services.obs.addObserver(function observe(aObsSubject, aObsTopic, aObsData) {
			Services.obs.removeObserver(observe, aObsTopic);
			resolve([aObsSubject, aObsData]);
		}, aTopic, false);
	});
}

function shutdownPlaces() {
	// alert("shutdownPlaces: starting");
	let promise = new Promise(resolve => {
		Services.obs.addObserver(resolve, "places-connection-closed", false);
	});
	let hs = PlacesUtils.history.QueryInterface(Ci.nsIObserver);
	hs.observe(null, "profile-change-teardown", null);
	// alert("shutdownPlaces: sent profile-change-teardown");
	hs.observe(null, "test-simulate-places-shutdown", null);
	// alert("shutdownPlaces: sent test-simulate-places-shutdown");
	return promise.then(() => {
		alert("shutdownPlaces: complete");
	});
}

Here is the new version of the Backup Bookmarks on Palemoon Close add-on ( Version II ) - where one can see all the code, as well:

https://www.mediafire.com/file/d6sf2b64n96y38s/Backup_Bookmarks_on_Pale_Moon_Close_Version_II.zip/file

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35477
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Bookmark backups

Unread post by Moonchild » 2022-10-24, 21:33

Completely shutting down the places database in a running browser? Not a good idea either.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
tellu-white
Moon lover
Moon lover
Posts: 83
Joined: 2022-03-08, 22:02

Re: Bookmark backups

Unread post by tellu-white » 2022-10-24, 21:46

Moonchild wrote:
2022-10-24, 21:33
Completely shutting down the places database in a running browser? Not a good idea either.
Thank you for your reply, even though its content made me blue :)

Locked