importing bookmarks
Forum rules
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific bugs, git, the repositories, etc.
This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Please post issues with specific websites, extensions, etc. in the relevant boards for those topics.
Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific bugs, git, the repositories, etc.
This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Please post issues with specific websites, extensions, etc. in the relevant boards for those topics.
Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
importing bookmarks
1. Pale Moon could not read a places.sqlite file, and then Pale Moon renamed it to places.sqlite.corrupt.
2. Chrome gave an option to import from Internet Explorer or Firefox. I moved places.sqlite to the Firefox profile. When I imported the places.sqlite file to Chrome, it loaded quickly. Chrome was then able to export an 11.2 mb html backup instantaneously.
3. This html backup could then be imported back to Pale Moon, but it took a long time with CPU usage. I even got an alert that a bookmark overlay script was busy or not responding.
4. Eventually the 11.2 mb html backup was turned into a 20 mb places.sqlite file by Pale Moon. This was a fix for the corrupted places.sqlite issue.
This whole process was unpleasant. Can you make Pale Moon just get it right the first time? I know bookmarks function is low on the priority list, but there is a clear difference in speed and capability. This is an area where improvement can be made. Is it easy to just copy whatever Chrome is doing here?
Also when I open the bookmarks menu for the first time during a session, I notice it takes a few seconds to open up on my physical hard drive. It feels like Pale Moon is retrieving the entire list before displaying. Would it be more efficient and faster if Pale Moon loaded only the first 30 entries quickly? The other entries can be retrieved as needed as I scroll down the list of bookmarks. (It's much faster on an ssd, so I understand if this is not considered to be an issue.)
2. Chrome gave an option to import from Internet Explorer or Firefox. I moved places.sqlite to the Firefox profile. When I imported the places.sqlite file to Chrome, it loaded quickly. Chrome was then able to export an 11.2 mb html backup instantaneously.
3. This html backup could then be imported back to Pale Moon, but it took a long time with CPU usage. I even got an alert that a bookmark overlay script was busy or not responding.
4. Eventually the 11.2 mb html backup was turned into a 20 mb places.sqlite file by Pale Moon. This was a fix for the corrupted places.sqlite issue.
This whole process was unpleasant. Can you make Pale Moon just get it right the first time? I know bookmarks function is low on the priority list, but there is a clear difference in speed and capability. This is an area where improvement can be made. Is it easy to just copy whatever Chrome is doing here?
Also when I open the bookmarks menu for the first time during a session, I notice it takes a few seconds to open up on my physical hard drive. It feels like Pale Moon is retrieving the entire list before displaying. Would it be more efficient and faster if Pale Moon loaded only the first 30 entries quickly? The other entries can be retrieved as needed as I scroll down the list of bookmarks. (It's much faster on an ssd, so I understand if this is not considered to be an issue.)
Last edited by person45 on 2019-08-21, 19:46, edited 2 times in total.
Re: importing bookmarks
(While you may be able to throw around places.sqlite between different browsers & browser versions, including going from newer to older & or different, you can't really expect "compatibility".
If I had the need to move places.sqlite from 1 to another, I would certainly try it, but I would not be perturbed if it failed, or if it only worked to a point. Many times I may be glad that it worked - to a point.)
Where did the particular places.sqlite file originate from?
And what versions of FF (Quantum or Legacy?) / Chrome are we talking?
Different "import" methods will give differing import speeds.
Being able to read the .sqlite directly, is just that.
Importing certainly can be time consuming as that data has to be parsed & then written out to a .sqlite. And there can be time differences between .html & .json too.
And you happen to have a rather large (compared to myself) set of bookmarks (&/or other histories in your places.sqlite). (The size of a .sqlite is not particularly indicative of how much content it actually contains as they are generally created with a rather large initial size, then tend to grow by set increments.)
If I had the need to move places.sqlite from 1 to another, I would certainly try it, but I would not be perturbed if it failed, or if it only worked to a point. Many times I may be glad that it worked - to a point.)
Where did the particular places.sqlite file originate from?
And what versions of FF (Quantum or Legacy?) / Chrome are we talking?
Different "import" methods will give differing import speeds.
Being able to read the .sqlite directly, is just that.
Importing certainly can be time consuming as that data has to be parsed & then written out to a .sqlite. And there can be time differences between .html & .json too.
And you happen to have a rather large (compared to myself) set of bookmarks (&/or other histories in your places.sqlite). (The size of a .sqlite is not particularly indicative of how much content it actually contains as they are generally created with a rather large initial size, then tend to grow by set increments.)
Re: importing bookmarks
I appreciate the fast response, but I already found this information about the places.sqlite size.
https://support.mozilla.org/en-US/questions/969977
"The minimum file size of the places.sqlite file is 10 MB and Firefox increases this file in 10 MB chunks when necessary (its size is never reduced when deleting history or bookmarks). This is done for performance issues caused by fragmentation."
I know it's preferable not to use the places.sqlite file directly, but I think I had a problem with my physical hard drive, and my older Pale Moon wasn't functioning properly.
This still doesn't excuse the fact that the backup html file that Chrome created took such a long time to transfer. Also I find it strange when Chrome is able to read the "corrupted" places.sqlite file while Pale Moon cannot.
Re: importing bookmarks
For reference, this is where I found the fix.
https://support.mozilla.org/en-US/questions/969977
"Then I tried to import all browsing history from Chrome, and guess what!
All browsing history is back in the non corrupted brand new data base.
Earlier I had mentioned that from chrome I was able to import browsing history from firefox , even though the data base of firefox was corrupt, chrome was successful in importing all of my firefox's history, so later in the new profile , all I had to do was reverse the process."
I'm not the only person reporting a compatibility issue with places.sqlite. I hope I don't come across as a whining person. I want Pale Moon to be the best it can be, and sometimes we can look to other browsers for ideas for improvement.
Re: importing bookmarks
This is a very old and known Mozilla bug.
Here's my issue report @ GitHub:
https://github.com/MoonchildProductions/UXP/issues/505/
It was actually closed yesterday, for "being stale".
Re: importing bookmarks
I just tested restoring a json backup, and it still takes a long time with steady CPU usage and increased memory usage.
I know this is unimportant as long as it works eventually, but this is a small area where improvement would be very helpful.
Maybe this issue only affects users who have a big bookmarks "library" of links that were collected over the years.
Re: importing bookmarks
Obviously nobody had any interest in doing anything for this to improve it. Feel free to reopen it if you've done some work that should be evaluated.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
Re: importing bookmarks
That's understandable. Browsing is much more important than bookmarking.
It's a little sad that Google can afford to just pay off a recent college graduate to take care of little features while the main work under the hood can be done by senior engineers.
https://www.theguardian.com/technology/ ... erage-2018
The average pay was over $200,000 in the UK.
It would be a good idea to try to increase market share if maybe on first use Pale Moon would automatically scan for other browsers and offer to import bookmarks with one click of a button. Making the migration process easy and fast from Firefox, Opera, Safari, Chrome, or Edge would be helpful. That's probably why Chrome has such good compatibility with even a corrupt places.sqlite.
http://kb.mozillazine.org/Locked_or_dam ... ces.sqlite
"It is also possible that Firefox was not closed properly, as might happen when your computer loses power with Firefox running, which can cause a corrupt places.sqlite file."
I'm not quite sure what happened, but maybe I had a computer crash or power outage which caused it.
Re: importing bookmarks
https://github.com/MoonchildProductions ... ssues/587/
Lorienna wrote, "importing large amounts of bookmarks from HTM files will result in PM hanging, and potentialy giving a "Unresponsive script" error message I have a lot of bookmarks too, and so far haven't seen the "Unresponsive script" error on PM, but it shows up on FF quite often, particularly on more recent releases. However, the HTML importing process takes rather long time indeed,"
tomaso wrote, "my tests now indicate that the issue first seems to have appeared between Firefox v23.0a1 (2013-05-13) and v23.0 Final."
It seems there's some kind of issue introduced in the year 2013.
Lorienna wrote, "importing large amounts of bookmarks from HTM files will result in PM hanging, and potentialy giving a "Unresponsive script" error message I have a lot of bookmarks too, and so far haven't seen the "Unresponsive script" error on PM, but it shows up on FF quite often, particularly on more recent releases. However, the HTML importing process takes rather long time indeed,"
tomaso wrote, "my tests now indicate that the issue first seems to have appeared between Firefox v23.0a1 (2013-05-13) and v23.0 Final."
It seems there's some kind of issue introduced in the year 2013.
Re: importing bookmarks
I know a way to test this.
Generate a list of URLs that don't exist:
http://bookmarktest.com/1
http://bookmarktest.com/2
http://bookmarktest.com/3
Until you get a 5mb html file. Then go to Show All Bookmarks>Import and Backup>Import Bookmarks from HTML...
Compare the time it takes to create the places.sqlite file with how fast Chrome or Opera does it.
https://support.mozilla.org/en-US/questions/1100618
"Firefox seems to using JavaScript code to import an HTML file and that is always slower than using C++ code and if this JavaScript code runs into the check for hangs (max_chrome_script_run_time) then you get a slow script alert. Importing a large amount of bookmarks can take considerable time."
"I did notice a considerable difference between importing in Opera and Firefox when all went well : roughly 1 second and 1 minute for the same file."
Generate a list of URLs that don't exist:
http://bookmarktest.com/1
http://bookmarktest.com/2
http://bookmarktest.com/3
Until you get a 5mb html file. Then go to Show All Bookmarks>Import and Backup>Import Bookmarks from HTML...
Compare the time it takes to create the places.sqlite file with how fast Chrome or Opera does it.
https://support.mozilla.org/en-US/questions/1100618
"Firefox seems to using JavaScript code to import an HTML file and that is always slower than using C++ code and if this JavaScript code runs into the check for hangs (max_chrome_script_run_time) then you get a slow script alert. Importing a large amount of bookmarks can take considerable time."
"I did notice a considerable difference between importing in Opera and Firefox when all went well : roughly 1 second and 1 minute for the same file."
Re: importing bookmarks
Maybe a separate tool should be created to import large amounts of bookmarks into places.sqlite that provides a more interactive experience.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
Re: importing bookmarks
How many thousand bookmarks do you have? I just restored my bookmarks after backing them up and it took less than a second. I backup bookmarks to a folder outside of the profile as .json.person45 wrote: ↑2019-08-21, 20:23I just tested restoring a json backup, and it still takes a long time with steady CPU usage and increased memory usage.
I know this is unimportant as long as it works eventually, but this is a small area where improvement would be very helpful.
Maybe this issue only affects users who have a big bookmarks "library" of links that were collected over the years.
Re: importing bookmarks
I don't know, but if I bookmark 5 per day over 6 years, then that's about 11,000 without much effort. It just steadily grows over time.
I realize this affects only a small percentage of users, so I can accept that it is a low-priority issue.
It's not the Pale Moon developers' fault. The "cor-el" guy at the firefox forum said that it was a javascript issue. Mozilla may have messed it up in the year 2013. Tomaso wrote, "my tests now indicate that the issue first seems to have appeared between Firefox v23.0a1 (2013-05-13) and v23.0 Final." If it's possible to revert back to the old way, then maybe that would be a much easier fix.
Re: importing bookmarks
More importantly, how often do you revisit the sites you've bookmarked? The visit date/visit count fields in the bookmark manager should be instructive. For comparison, my bookmarks backup from del.icio.us over a 12 year period is just under 1500.
"One hosts to look them up, one DNS to find them and in the darkness BIND them."
Linux Mint 21 Xfce x64 on HP i5-5200 laptop, 12 GB RAM.
AutoPageColor|PermissionsPlus|PMPlayer|Pure URL|RecordRewind|TextFX
Linux Mint 21 Xfce x64 on HP i5-5200 laptop, 12 GB RAM.
AutoPageColor|PermissionsPlus|PMPlayer|Pure URL|RecordRewind|TextFX
-
- Moon Magic practitioner
- Posts: 2986
- Joined: 2015-09-26, 04:51
- Location: U.S.
Re: importing bookmarks
Off-topic:
See How to count bookmarks without extensions and external tools.
(originally posted here)
See How to count bookmarks without extensions and external tools.
(originally posted here)
Re: importing bookmarks
That's a good point. Most of my bookmarks are a collection of sites of interest, research, or information. I guess I use it more like an archive.
This is another reason why it's a low-priority issue.
But the question is why would Mozilla use this slower, javascript method in the 1st place? Is there any benefit to it?
Re: importing bookmarks
If you never check the validity of the bookmarks, you are likely to have at least a few dozen invalid ones. Web pages are not eternal.