importing bookmarks

Suggestions and feature requests for the Pale Moon browser

Moderator: satrow

person45
Hobby Astronomer
Hobby Astronomer
Posts: 25
Joined: 2017-10-20, 07:00

importing bookmarks

Unread post by person45 » 2019-08-21, 18:50

1. Pale Moon could not read a places.sqlite file, and then Pale Moon renamed it to places.sqlite.corrupt.

2. Chrome gave an option to import from Internet Explorer or Firefox. I moved places.sqlite to the Firefox profile. When I imported the places.sqlite file to Chrome, it loaded quickly. Chrome was then able to export an 11.2 mb html backup instantaneously.

3. This html backup could then be imported back to Pale Moon, but it took a long time with CPU usage. I even got an alert that a bookmark overlay script was busy or not responding.

4. Eventually the 11.2 mb html backup was turned into a 20 mb places.sqlite file by Pale Moon. This was a fix for the corrupted places.sqlite issue.

This whole process was unpleasant. Can you make Pale Moon just get it right the first time? I know bookmarks function is low on the priority list, but there is a clear difference in speed and capability. This is an area where improvement can be made. Is it easy to just copy whatever Chrome is doing here?

Also when I open the bookmarks menu for the first time during a session, I notice it takes a few seconds to open up on my physical hard drive. It feels like Pale Moon is retrieving the entire list before displaying. Would it be more efficient and faster if Pale Moon loaded only the first 30 entries quickly? The other entries can be retrieved as needed as I scroll down the list of bookmarks. (It's much faster on an ssd, so I understand if this is not considered to be an issue.)
Last edited by person45 on 2019-08-21, 19:46, edited 2 times in total.

User avatar
therube
Board Warrior
Board Warrior
Posts: 1076
Joined: 2018-06-08, 17:02

Re: importing bookmarks

Unread post by therube » 2019-08-21, 19:09

(While you may be able to throw around places.sqlite between different browsers & browser versions, including going from newer to older & or different, you can't really expect "compatibility".

If I had the need to move places.sqlite from 1 to another, I would certainly try it, but I would not be perturbed if it failed, or if it only worked to a point. Many times I may be glad that it worked - to a point.)


Where did the particular places.sqlite file originate from?
And what versions of FF (Quantum or Legacy?) / Chrome are we talking?


Different "import" methods will give differing import speeds.
Being able to read the .sqlite directly, is just that.
Importing certainly can be time consuming as that data has to be parsed & then written out to a .sqlite. And there can be time differences between .html & .json too.

And you happen to have a rather large (compared to myself) set of bookmarks (&/or other histories in your places.sqlite). (The size of a .sqlite is not particularly indicative of how much content it actually contains as they are generally created with a rather large initial size, then tend to grow by set increments.)

person45
Hobby Astronomer
Hobby Astronomer
Posts: 25
Joined: 2017-10-20, 07:00

Re: importing bookmarks

Unread post by person45 » 2019-08-21, 19:38

therube wrote:
2019-08-21, 19:09
they are generally created with a rather large initial size, then tend to grow by set increments.)
I appreciate the fast response, but I already found this information about the places.sqlite size.

https://support.mozilla.org/en-US/questions/969977
"The minimum file size of the places.sqlite file is 10 MB and Firefox increases this file in 10 MB chunks when necessary (its size is never reduced when deleting history or bookmarks). This is done for performance issues caused by fragmentation."

I know it's preferable not to use the places.sqlite file directly, but I think I had a problem with my physical hard drive, and my older Pale Moon wasn't functioning properly.

This still doesn't excuse the fact that the backup html file that Chrome created took such a long time to transfer. Also I find it strange when Chrome is able to read the "corrupted" places.sqlite file while Pale Moon cannot.

person45
Hobby Astronomer
Hobby Astronomer
Posts: 25
Joined: 2017-10-20, 07:00

Re: importing bookmarks

Unread post by person45 » 2019-08-21, 19:54

therube wrote:
2019-08-21, 19:09
If I had the need to move places.sqlite from 1 to another, I would certainly try it, but I would not be perturbed if it failed, or if it only worked to a point. Many times I may be glad that it worked - to a point.)
For reference, this is where I found the fix.

https://support.mozilla.org/en-US/questions/969977
"Then I tried to import all browsing history from Chrome, and guess what!

All browsing history is back in the non corrupted brand new data base.
Earlier I had mentioned that from chrome I was able to import browsing history from firefox , even though the data base of firefox was corrupt, chrome was successful in importing all of my firefox's history, so later in the new profile , all I had to do was reverse the process."

I'm not the only person reporting a compatibility issue with places.sqlite. I hope I don't come across as a whining person. I want Pale Moon to be the best it can be, and sometimes we can look to other browsers for ideas for improvement.

User avatar
Tomaso
Board Warrior
Board Warrior
Posts: 1294
Joined: 2015-07-23, 16:09
Location: Norway

Re: importing bookmarks

Unread post by Tomaso » 2019-08-21, 20:19

person45 wrote:
2019-08-21, 18:50
3. This html backup could then be imported back to Pale Moon, but it took a long time with CPU usage. I even got an alert that a bookmark overlay script was busy or not responding.
This is a very old and known Mozilla bug.
Here's my issue report @ GitHub:
https://github.com/MoonchildProductions/UXP/issues/505/
It was actually closed yesterday, for "being stale".

person45
Hobby Astronomer
Hobby Astronomer
Posts: 25
Joined: 2017-10-20, 07:00

Re: importing bookmarks

Unread post by person45 » 2019-08-21, 20:23

Tomaso wrote:
2019-08-21, 20:19
This is a very old and known Mozilla bug.
I just tested restoring a json backup, and it still takes a long time with steady CPU usage and increased memory usage.

I know this is unimportant as long as it works eventually, but this is a small area where improvement would be very helpful.

Maybe this issue only affects users who have a big bookmarks "library" of links that were collected over the years.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 24451
Joined: 2011-08-28, 17:27
Location: 58°2'16"N 14°58'31"E
Contact:

Re: importing bookmarks

Unread post by Moonchild » 2019-08-21, 21:17

Tomaso wrote:
2019-08-21, 20:19
It was actually closed yesterday, for "being stale".
Obviously nobody had any interest in doing anything for this to improve it. Feel free to reopen it if you've done some work that should be evaluated.
"If you want to build a better world for yourself, you have to be willing to build one for everybody." -- Coyote Osborne
Image

person45
Hobby Astronomer
Hobby Astronomer
Posts: 25
Joined: 2017-10-20, 07:00

Re: importing bookmarks

Unread post by person45 » 2019-08-22, 17:55

Moonchild wrote:
2019-08-21, 21:17
Obviously nobody had any interest in doing anything for this to improve it. Feel free to reopen it if you've done some work that should be evaluated.
That's understandable. Browsing is much more important than bookmarking.

It's a little sad that Google can afford to just pay off a recent college graduate to take care of little features while the main work under the hood can be done by senior engineers.

https://www.theguardian.com/technology/ ... erage-2018
The average pay was over $200,000 in the UK.

It would be a good idea to try to increase market share if maybe on first use Pale Moon would automatically scan for other browsers and offer to import bookmarks with one click of a button. Making the migration process easy and fast from Firefox, Opera, Safari, Chrome, or Edge would be helpful. That's probably why Chrome has such good compatibility with even a corrupt places.sqlite.

http://kb.mozillazine.org/Locked_or_dam ... ces.sqlite
"It is also possible that Firefox was not closed properly, as might happen when your computer loses power with Firefox running, which can cause a corrupt places.sqlite file."

I'm not quite sure what happened, but maybe I had a computer crash or power outage which caused it.

person45
Hobby Astronomer
Hobby Astronomer
Posts: 25
Joined: 2017-10-20, 07:00

Re: importing bookmarks

Unread post by person45 » 2019-08-22, 21:34

https://github.com/MoonchildProductions ... ssues/587/
Lorienna wrote, "importing large amounts of bookmarks from HTM files will result in PM hanging, and potentialy giving a "Unresponsive script" error message I have a lot of bookmarks too, and so far haven't seen the "Unresponsive script" error on PM, but it shows up on FF quite often, particularly on more recent releases. However, the HTML importing process takes rather long time indeed,"

tomaso wrote, "my tests now indicate that the issue first seems to have appeared between Firefox v23.0a1 (2013-05-13) and v23.0 Final."

It seems there's some kind of issue introduced in the year 2013.

User avatar
Tomaso
Board Warrior
Board Warrior
Posts: 1294
Joined: 2015-07-23, 16:09
Location: Norway

Re: importing bookmarks

Unread post by Tomaso » 2019-08-23, 10:34

person45 wrote:
2019-08-22, 21:34
Lorienna wrote, "importing large amounts of bookmarks from HTM files will result in PM hanging, and potentialy giving a "Unresponsive script" error message
Actually, I wrote that.
Lorienna simply quoted it. :)

person45
Hobby Astronomer
Hobby Astronomer
Posts: 25
Joined: 2017-10-20, 07:00

Re: importing bookmarks

Unread post by person45 » 2019-09-06, 02:53

I know a way to test this.

Generate a list of URLs that don't exist:
http://bookmarktest.com/1
http://bookmarktest.com/2
http://bookmarktest.com/3

Until you get a 5mb html file. Then go to Show All Bookmarks>Import and Backup>Import Bookmarks from HTML...

Compare the time it takes to create the places.sqlite file with how fast Chrome or Opera does it.

https://support.mozilla.org/en-US/questions/1100618
"Firefox seems to using JavaScript code to import an HTML file and that is always slower than using C++ code and if this JavaScript code runs into the check for hangs (max_chrome_script_run_time) then you get a slow script alert. Importing a large amount of bookmarks can take considerable time."

"I did notice a considerable difference between importing in Opera and Firefox when all went well : roughly 1 second and 1 minute for the same file."

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 24451
Joined: 2011-08-28, 17:27
Location: 58°2'16"N 14°58'31"E
Contact:

Re: importing bookmarks

Unread post by Moonchild » 2019-09-06, 07:21

Maybe a separate tool should be created to import large amounts of bookmarks into places.sqlite that provides a more interactive experience.
"If you want to build a better world for yourself, you have to be willing to build one for everybody." -- Coyote Osborne
Image

Goodydino
Astronaut
Astronaut
Posts: 507
Joined: 2017-10-10, 21:20

Re: importing bookmarks

Unread post by Goodydino » 2019-09-15, 20:37

person45 wrote:
2019-08-21, 20:23
I just tested restoring a json backup, and it still takes a long time with steady CPU usage and increased memory usage.

I know this is unimportant as long as it works eventually, but this is a small area where improvement would be very helpful.

Maybe this issue only affects users who have a big bookmarks "library" of links that were collected over the years.
How many thousand bookmarks do you have? I just restored my bookmarks after backing them up and it took less than a second. I backup bookmarks to a folder outside of the profile as .json.

Post Reply