Bookmark backups

[tellu-white]

Completely shutting down the places database in a running browser? Not a good idea either.

Following your reply, I have uninstalled the "Backup Bookmarks on Palemoon Close" add-on. Immediately after that, Pale Moon restarted to close with the error message "Pale Moon web browser has stopped working". This behavior continued almost every time I closed the browser, so I decided to reinstall the add-on. Since then Pale Moon has NEVER closed with an error message, I haven't lost any bookmarks and the database has remained OK (uncorrupted).

As I said before, I don't want to resort to creating a new database (i.e. not having the old bookmarks loaded in Pale Moon), a solution loved by Pale Moon but not by me

By this post I don't want to contradict you (you said that "shutting down the places database in a running browse is not a good idea either"), but the situation described above leaves me without an answer to the following issues:

1. What exactly in the code inserted in the add-on makes Pale Moon avoid the problem that appears when closing the program and thus close without error?

2. The inserted code runs only after pressing the X button to close the browser, at which time, however, the database will be closed by Pale Moon itself, so I don't understand why the code taken from the "head_common.js" file would be harmful and why it turns out, on the contrary, to have a positive effect (error avoidance) on the code used by Pale Moon to close the database ?!

Remarks:

a. If I delete the "places.sqlite" file (120 MB), it is replaced by Pale Moon with a 10 MB file (without Bookmarks). In this case, Pale Moon closes INSTANTLY after pressing the X button (no error, obviously).

b. Both if the "Backup Bookmarks on Palemoon Close" add-on is not installed and if it is installed, the 120 MB "places.sqlite" file makes the browser closing time about one minute. The only difference between the two cases is the occurrence or avoidance of the "Pale Moon web browser has stopped working" error.

I would be grateful if you would give me a more elaborate explanation of the aspects mentioned above.

[Moonchild]

The behaviour you're seeing is because you have corrupted your places database with the extension (a considerable risk I indicated).
Removing the sqlite database and letting the browser create a new one and then re-importing your bookmarks from html (or potentially a json bookmark backup) is probably the only way you can restore sanity at this point. It's obvious that something in the structure of the database (probably one or more of the indices) is causing an application crash on shutdown. Manually shutting down the places database on window close may avoid this crash since the shutdown code will notice the places db has already been shut down and thus skipping operations that would normally take place (and are desired), but that doesn't mean it actually fixes anything. Quite the opposite, it just avoids a symptom.

[Kris_88]

tellu-white,
As Moonchild mentioned, closing the places db manually is a bad idea. Why not just intercept "places-connection-closed" notification and then make your things?

Code: Select all

Components.utils.import("resource://gre/modules/Services.jsm");

function EXTPBK_onPlacesClosed(aSubject, aTopic, aData) {
  Services.obs.removeObserver(EXTPBK_onPlacesClosed, "places-connection-closed");

  var dat = new Date();
  var sdate = dat.getFullYear() + 
             ('0' + (dat.getMonth()+1)).slice(-2) +
             ('0' + dat.getDate()).slice(-2) + '_'+
             ('0' + dat.getHours()).slice(-2) + 
             ('0' + dat.getMinutes()).slice(-2) + 
             ('0' + dat.getSeconds()).slice(-2);

  var SrcFile = Services.dirsvc.get('ProfD', Components.interfaces.nsIFile);
  SrcFile.append('places.sqlite');
  SrcFile.copyTo(null, 'places_' + sdate + '.bkp')
};

function EXTPBK_Init() {
  Services.obs.addObserver(EXTPBK_onPlacesClosed, "places-connection-closed", false);
};

EXTPBK_Init();

[tellu-white]

Why not just intercept "places-connection-closed" notification and then make your things?

It doesn't work!

I uninstalled the "Backup Bookmarks on Palemoon Close" add-on and then used the code you provided. It worked until I restarted Windows and deleted the Pale Moon Cache, then it didn't work anymore: Pale Moon restarted to close with the error message "Pale Moon web browser has stopped working" and your code didn't backup the "places.sqlite" file anymore.

Important!

When Pale Moon closes with this error message, the "places.sqlite-shm" and "places.sqlite-wal" files are no longer deleted, indicating that the database is no longer closed.

**********

The behaviour you're seeing is because you have corrupted your places database with the extension (a considerable risk I indicated).

As I said in a previous comment:

Pale Moon had the same problem ("Pale Moon web browser has stopped working") before making this add-on. From this I conclude that it is not this add-on that created the problem.

**********

Removing the sqlite database and letting the browser create a new one and then re-importing your bookmarks from html (or potentially a json bookmark backup) is probably the only way you can restore sanity at this point.

I tried the two options you suggested, after previously uninstalling the "Backup Bookmarks on Palemoon Close" add-on.

1. I started with "Export Bookmarks to HTML". It took about five minutes to complete this procedure. Pale Moon produced the 170 MB file "bookmarks.html":

01 Backup html.jpg

2. The next step was to make backup bookmarks in "json" format. This procedure failed. After about a minute, the procedure ended with no message and no result:

02 Backup json.jpg

03 Backup json.jpg

As you can see, the procedure ended without producing a "json" file.

I repeated this step three times (with the same result), then gave up.

3. Next I deleted the database, after which Pale Moon created a new one. After restarting the browser, I did "Import Bookmarks from HTML":

04 Import Bookmarks from HTML.png

05 Import Bookmarks from HTML.png

After more than ten minutes, during which time I received seven messages like the one in the picture above, Pale Moon created the following files:

• places.sqlite
• places.sqlite-shm
• places.sqlite-wal

The "places.sqlite" file was the same size (120 MB) as the one from which the "bookmarks.html" file was created, but a 116 MB "places.sqlite-wal" file was created:

06 Import Bookmarks from HTML.png

07 Import Bookmarks from HTML.png

From this point on Pale Moon started to close again with the error message "Pale Moon web browser has stopped working", so it didn't close the database anymore and the 116 MB file "places.sqlite-wal" remained intact. This seems to have caused another problem: every time I tried to "scroll" the bookmarks in Pale Moon, I had to wait a while until the bookmarks scrolling unblocked.

Manually shutting down the places database on window close may avoid this crash ... but that doesn't mean it actually fixes anything. Quite the opposite, it just avoids a symptom.

In the end, I'm back to avoiding the problem, not solving it, which doesn't seem to work if I don't give up using the old bookmarks. I went back to the old database (the old "places.sqlite") and to the " Backup Bookmarks on Palemoon Close" add-on. The scrolling problem disappeared as well as the error message when closing Pale Moon. As I said before, I don't want to resort to creating a new database, i.e. not having the old bookmarks loaded in Pale Moon, so as long as my add-on works, i.e. as long as Pale Moon closes without error and merges the "places.sqlite-shm" and "places.sqlite-wal" files into the main "places.sqlite" file, as long as I don't lose any bookmarks and as long as the "Places Database - Verify Integrity" test shows me that the database is OK (uncorrupted), for me it is less important that this solution "doesn't fixes anything, it just avoids a symptom".

[tellu-white]

The next step was to make backup bookmarks in "json" format. This procedure failed. After about a minute, the procedure ended with no message and no result

I wanted to check if this problem occurred because of other installed add-ons. So, I tried to backup bookmarks in "json" format from Pale Moon Portable, without installing any add-ons in it.

First I replaced the "places.sqlite" created by Pale Moon Portable with the 120 MB one, which is causing me problems, then I proceeded with the backup procedure. This procedure also failed in Pale Moon Portable. Here too, after about a minute, the procedure ended without any message and no result (again the procedure did not create a "json" file). This time, however, I also opened the "Error Console" and here the error message in the picture below appeared:

01.png

02.png

The error message:

Code: Select all

Timestamp: 10/29/2022 16:02:07
Error: A promise chain failed to handle a rejection. Did you forget to '.catch', or did you forget to 'return'?
See https://developer.mozilla.org/Mozilla/JavaScript_code_modules/Promise.jsm/Promise

Date: Sat Oct 29 2022 16:01:37 GMT+0300 (GTB Standard Time)
Full Message: out of memory
Full Stack: JS frame :: resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js :: register :: line 193
JS frame :: resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js :: completePromise :: line 714
JS frame :: resource://gre/modules/Task.jsm :: TaskImpl_handleException :: line 455
JS frame :: resource://gre/modules/Task.jsm :: TaskImpl_run :: line 330
JS frame :: resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js :: process :: line 931
JS frame :: resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js :: walkerLoop :: line 812
JS frame :: resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js :: scheduleWalkerLoop/< :: line 746
Source Code:
0

Is it possible that this error is also related to Pale Moon's failure to close this problem database?

[Moonchild]

Code: Select all

Full Message: out of memory

probably means there's a serious structural issue in the database that the places code can't recover from, resulting in it "spinning its wheels" until it runs out of memory.

Is it possible that this error is also related to Pale Moon's failure to close this problem database?

Very likely.

[Goodydino]

Why do you need an extension to backup bookmarks? You can do that manually from the Organize Bookmarks window. I do that when I make changes to the bookmarks.

[Moonchild]

Why do you need an extension to backup bookmarks?

They don't need one. But they are trying to do so anyway to suit their needs. unfortunately it seems that in the process of tinkering on live data, things have gone awry.
The browser makes compressed bookmark backups by default and those could simply be used or copied out (storing the backups in a safe place). There's nothing wrong with just using the tried&tested routines that are built right into the browser.
Heck, you could even make a point of running the browser from a batch program that automatically copies the backups the browser itself makes to a safe backup location after the browser shuts down. That would be a hell of a lot safer than trying to do something from within a running browser.

[tellu-white]

Removing the sqlite database and letting the browser create a new one and then re-importing your bookmarks from html (or potentially a json bookmark backup) is probably the only way you can restore sanity at this point. It's obvious that something in the structure of the database (probably one or more of the indices) is causing an application crash on shutdown.

After the unsuccessful attempt to restore the database from the "bookmarks.html" file, I tried again, this time in several steps:

1. I exported five kookmark folders (the largest/loaded ones) in "html" format - using an add-on I made. Example:

01.png

2. I deleted the five folders already exported in "html" format:

02.png

3. I continued with "Export Bookmarks to HTML":

03.png

4. I tested (again) a "backup" in "json" format. This time the procedure succeeded (in the absence of the five large folders - with many bookmarks in them).

NOTE: I didn't use the "json" format to restore the database because this option leads to the loss of some ICONs (see the test below).

Before "restore":

04.png

After "restore":

05.png

5. Out of curiosity, I checked the structure of the resulting "html" files after saving the bookmarks and found no problems: their structure was identical to that of a test "html" file (obtained with "Export Bookmarks to HTML" from a freshly unzipped Pale Moon Portable):

06.png

6. I continued with "Import Bookmarks from HTML". This time everything went without problems and at the end I got a 60 MB "places.sqlite" file.

07.png

7. Next I imported the bookmarks from the five "html" files exported at the beginning, again without problems. In the end I got a "places.sqlite" file of 120 MB, the same size as the original one (the one I tried to rebuild):

08.png

Again, this new attempt was useless. The above procedure did not solve the problem! Although I let Pale Moon create a new database by importing (valid) "html" files, it started to close again with the error message "Pale Moon web browser has stopped working".

Finally, I did another test: in an empty database (created by Pale Moon) I imported the largest "html" file of the five exported at the beginning. This time Pale Moon closed without error. It seems that for Pale Moon "smaller is better"

Conclusion: with each new test, the mystery of this database gets deeper

[Figueroa]

Have you tried the FEBE extension?
https://addons.palemoon.org/addon/febe/

It appears to do exactly what you need. Backup your bookmarks every time you shutdown Pale Moon.

febe.jpg

[Kerebron]

tellu-white, could it be that some of your bookmarks contain a set of characters that mess with the database structure? I don't know if it's possible to "sabotage" a database structure from within a database record in Pale Moon.

[tellu-white]

Have you tried the FEBE extension?
https://addons.palemoon.org/addon/febe/
It appears to do exactly what you need. Backup your bookmarks every time you shutdown Pale Moon.

It doesn't work!

I uninstalled the "Backup Bookmarks on Palemoon Close" add-on and then used the FEBE extension. I have tried several settings in conjunction with the "Backup when browser is closed" option:

1. "User-defined backups" with setting "Backup on shutdown grace period (seconds)" = 10 seconds (Default).

2. "User-defined backups" with the setting "Backup on shutdown grace period (seconds)" = 0 seconds.

3. "User-defined backups" with the setting "Backup on shutdown grace period (seconds)" = 120 seconds.

3.a. Without "Post entries and exit".

3.b. With "Post entries and exit".

Regardless of the variant used, at the end Pale Moon still closed with the error "Pale Moon web browser has stopped working", did not close the database and left the files "places.sqlite-shm" and "places.sqlite-wal" undeleted (see last picture).

Screenshots of version 3.b.:

01.png

02.png

03.png

04.png

05.png

Pale Moon in "Task Manager" (inactive) before closing it:

06.png

Pale Moon after clicking the X (close) button:

07.png

08.png

09.png

10.png