urlclassifier3.sqlite vanishing act

[dallas7]

On my tower and laptop systems both Win7hpSP1x64 & Pale Moon Portable 19.0.2-x64:

Every week I zip up my profile folders and store them on a NAS partition I use for backups. To save space I delete the relatively huge urlclassifier3.sqlite file and after the zip restore it from the Recycle Bin. Or if I forget the latter action, the sqlite file has always rebuilt itself as it should.

Recently I've noticed that file no longer exists. I wish I could say exactly when that started, but it's been three, maybe four, weeks. In it's place is the urlclassifier.pset file.

I have for the longest time had an ini file where
browser.safebrowsing.enabled=true
browser.safebrowsing.malware.enabled=true
but even when set to false and then checking both "Block report..." in Options > Security the sqlite file fails to build.

Even if I delete the pset file, it rebuilds (regardless of the status of the block options) which, as you know, is the default action when blocking is disabled. When enabled, an sqlite file should complete its download within an hour, tops.

Did something change? Maybe with 19.0.2??

[dallas7]

No change in 20.0.1.

Anyone??

[Blacklab]

Last year I chanced across an article about two Mozilla devs praised for massively reducing the size of the Google Safe Browsing database file that Firefox needed to download - maddeningly I have never been able find it since - the gist of which (flaky memory permitting) suggested that the database could be a fraction of the current size, possibly even small enough to be downloaded at browser start? Not sure if this ever got as far as being implemented?

Also this very curious post by Mozilla dev Justin Dolske tacked on the end of old bug #499362 "Switching between major releases resets urlclassifier3.sqlite" (essentially about speed of rebuilding database in Firefox 3).... but added almost 3 years after the previous comment - assume while changing Bug status to Resolved Invalid?

The sqlite DB is gone now.

Pity there is no further explanation, no link, and no mention on his blog. Everything else found on-line still talks about deleting and rebuilding the urlclassifier3.sqlite file e.g. Kioskeas.net's Optimize Firefox (completely) dated March 2013 - the contents of which, I hasten to say, I am not recommending! (EDIT: 2 below would seem to make Kioskea's page, and many similar articles, obsolete information? The change to a new safebrowsing store seems to have been kept rather quiet? )

EDIT 1: This might be the article I remembered? Garf's blog "New SafeBrowsing backend" which links to bug #673470 which has comments from Nicholas Nethercote whose MemShrink Project blog is often an interesting, if rather technical read and contains a few links to more information.

EDIT 2: @dallas7 Looks like your "Did something change? Maybe with 19.0.2??" was correct! The second of the four "further development" bugs mentioned in Bug 673470 Comment #146 is probably the cause of your "sqlite vanishing act" - specifically bug #723153 "Remove old safebrowsing store from profiles"

bug #673470 introduces the new safebrowsing store. We should remove the old store when users aren't expected to switch to a Firefox version using the old one any more..... bug #673470 landed in Nightlies for Firefox 17, so scheduling this for Firefox 19 or 20.

[dallas7]

Thanks for all that great detail.

You got me looking at my profile folder again and lo and behold the urlclassifier.pset file is absent and there is now a a 154 bytes urlclassifierkey3.txt file. The pset file, to the best of my recollection, is the one built when the two Block options are disabled. So, something went different since my OP last month.

Also there is a safebrowsing folder with a dozen files worth about 3.5 MB. Today is the first time I noticed that folder - so much for my powers of observation. Not to mention my now embarrassing "No change..." posting yesterday.

I can conclude, then, SafeBrowsing is functional and with far more efficiency than previously implemented within the context of local resources.

In looking for more info on that safebrowsing folder, I can't tell exactly when that first showed up - there are some discussions dating back early 2012. I know that as late as February of this year, if I forgot to restore the ~40 MB urlclassifier3.sqlite after my backup, a new one got built.

Further, I would think such an improvement would have been announced by Mozilla with brass bands and fireworks. I subscribe to about a dozen tech news feeds (gHacks, Raymond.CC, 404 Tech, etc.) and don't recall ever reading about it. Something like that should have been extolled with great fanfare and kudos from the bloggers and pundits.

Oh well. At least now we have Australis (aka MozillaChrome) to look forward to...

Curiosity satisfied. GMKB (grey matter knowledge base) updated.

Cheers.