scripted cache clearing

Users and developers helping users with generic and technical Pale Moon issues on all operating systems.

Moderator: trava90

Forum rules
This board is for technical/general usage questions and troubleshooting for the Pale Moon browser only.
Technical issues and questions not related to the Pale Moon browser should be posted in other boards!
Please keep off-topic and general discussion out of this board, thank you!
Lew Rockwell Fan

scripted cache clearing

Unread post by Lew Rockwell Fan » 2018-03-18, 17:08

Is there any reason not to clear cache by simply

Code: Select all

rm -rf "~/.cache/moonchild productions/pale moon/randomasciistring.profilename/*"
?
Or, alternately, using

Code: Select all

find -type f
to rm just the "regular" files, leaving the directory tree intact? Or some combination of similar commands if some directories must be left intact and others can be removed?

I've tried 3 extensions for this, and judging by the recursive size of the directory and content, they don't work. Menu: Tools-preferences-advanced-network-"clear now" button (the first one) seems to work, so that validates at least part of my test.

But is that test sufficient? Somewhere in one of the descriptions of relevant extensions I read reference to clearing cache in RAM as well as on rust. I presume the button buried deep in the Tools menu does this as well. But other than browsing a while, checking RAM usage, clicking the button, and checking RAM usage again, I don't know how to confirm or test this. Is there any way to script that action?

Basically, I want code to duplicate the action of that button.
Last edited by Lew Rockwell Fan on 2018-03-18, 22:26, edited 3 times in total.

terranigma

Re: scripted cache clearing

Unread post by terranigma » 2018-03-18, 19:01

You can define an user mountable tmpfs folder for cache. Makes your caching as fast as your memory.. volatile as your memory either.
Like this (/etc/fstab):

Code: Select all

tmpfs /home/terra/.cache/moonchild\040productions tmpfs rw,nodev,relatime,size=250M,mode=755,user,nofail,uid=1000,gid=100 0 0
Adjust path and uid,gid options for your user.
\040 is not typo, a white space character.

$ umount .cache/moonchild\ productions
$ mount .cache/moonchild\ productions

makes all data clear.
Last edited by terranigma on 2018-03-18, 23:41, edited 1 time in total.

Lew Rockwell Fan

Re: scripted cache clearing

Unread post by Lew Rockwell Fan » 2018-03-18, 21:49

terranigma wrote:You can define an user mountable tmpfs folder for cache. . . .

Code: Select all

tmpfs /home/terra/.cache/moonchild\040productions tmpfs rw,nodev,relatime,size=250M,mode=755,user,nofail,uid=1000,gid=100 0 0
. . .
$ umount .cache/moonchild\ productions
$ mount .cache/moonchild\ productions

makes all data clear.
Thank you. I take it you're doing this & it works for you? My RAM/hard-drive-space ratio is not high enough to make that exact course attractive but it is illuminating. That would seem to be logically equivalent to:

Code: Select all

rm -rf "~/.cache/moonchild productions/*"
Right? So, if you're getting away with that, then surely my more limited

Code: Select all

rm -rf "~/.cache/moonchild productions/pale moon/randomasciistring.profilename/*"
wouldn't cause any catastrophe, right? Unless some text file further up in "~/.cache/moonchild productions/pale moon/" is supposed to be keeping track of what's in there, in which case maybe it is all or nothing.

But there is still the question of "Is this the full equivalent of "Tools-preferences-advanced-network-first 'clear now' button"? One of the extensions I read about claimed to "clear cache on disk AND in RAM". I interpreted that to mean that a default installation had some of both. Maybe that's a Firefox-only thing, or maybe I misunderstood it. Maybe they just meant their extension still worked if you had moved the cache to a tmpfs. Thoughts, anyone?

I don't have a problem with speed of access to cache on drive. I'm just looking for an easier way to clear cache for the occasional site that is not working and that suggests clearing cache as a fix. The only one like that I use at the moment is protonmail, and it only happens now and then, but clearing cache does work, so I'd just like to make it easy to do. By default, it's 5 clicks in a gui, each click preceded by a couple of seconds of a wetware process like "What is it I click next - oh yeah, network - now where is it?". If the rm approach works, or yours if I were willing to give it the RAM, I can reduce it to 2 clicks, 1 if I turn it into a button.

Lew Rockwell Fan

Re: scripted cache clearing

Unread post by Lew Rockwell Fan » 2018-03-18, 23:12

Update:

I had the syntax wrong:

Code: Select all

rm -rf "~/.cache/moonchild productions/*"
does NOT work for me. At all. Noting is deleted. Had to change 2 things.

- * doesn't expand in this circumstance. Bash tries to delete a file named '*'. OK, I've seen that before. Never can remember when this (globbing I believe it is called) works and when some more esoteric form of regex is needed. YMMV. This is on an ext4 fs, which may be relevant - I'm not sure.

- And, more surprising to me, '~' doesn't expand either. I had to give the canonical path thus:
rm -rf "/home/user/.cache/moonchild productions"

That worked. Palemoon did not crash. It did not complain. It continued to work normally. The internet did not break. Nukes were not launched. PM did not immediately create a new cache. It did when I restarted it. I may experiment further to understand that better. I have not yet tested deleting cache for only the current profile, but I assume that will work if this does.

My only remaining question is "Is that all the clear cache button does?" Is there any such thing as "cache in RAM" in a default setup that I need to do something about?

Anyway, I've got this on a menu (2 clicks) so the next time I see a site that isn't working under circumstances where "clear cache" is the suggested fix, I'll find out it this will do the trick.

terranigma

Re: scripted cache clearing

Unread post by terranigma » 2018-03-18, 23:38

It is just cache. Directory structure is rebuilt if it doesn't exist. That's all.
If you're using SSD, using tmpfs as cache is a good thing for your SSD's health due to the high amount of writes/removes are eliminated. Ram size matters of course.
Last edited by terranigma on 2018-03-18, 23:40, edited 2 times in total.

Lew Rockwell Fan

Re: scripted cache clearing

Unread post by Lew Rockwell Fan » 2018-03-19, 00:51

Trying to understand how this compares to what the buttons deep in the menu do, I found some things that seemed weird enough to make me distrust the spacefm info dialog applied against the directory. So I fell back on a terminal and made comments when I did something with PM. Can't explain it more succintly than just copying the content of the terminal verbatim:

- - - - the following copied from terminal - - - -

$ du -sbh '/home/me/.cache/moonchild productions'
16M /home/me/.cache/moonchild productions

$ nohup palemoon &
[1] 17965

$ nohup: ignoring input and appending output to 'nohup.out'
[1]+ Done nohup palemoon

$ du -sbh '/home/me/.cache/moonchild productions'
16M /home/me/.cache/moonchild productions

$ # opened in tabs a bunch of sites not recently visited

$ du -sbh '/home/me/.cache/moonchild productions'
21M /home/me/.cache/moonchild productions

$ # opened one more tab. Closed a tab that wasn't finishing loading.

$ du -sbh '/home/me/.cache/moonchild productions'
21M /home/me/.cache/moonchild productions

$ # tools, prefs, advanced, net: clicked BOTH 'Clear Now' buttons

$ # gui claimed both 'web content cache' & 'applications cache' to have 0 bytes

$ du -sbh '/home/me/.cache/moonchild productions'
16M /home/me/.cache/moonchild productions

$ # That's bigger than the zero I remember meeting

$ # Closed all but one tab open to a small local text file

$ # tools, prefs, advanced, net: STILL claims 0 bytes in both places.

$ # Clicked both 'Clear Now' buttons

$ du -sbh '/home/me/.cache/moonchild productions'
16M /home/me/.cache/moonchild productions

$ # Looks like some sort of irreducible minimum - but when I explore the directory with spacefm, I find all sorts of images I recognize from websites, none of which are open now.

$ # shut down pale moon.

$ du -sbh '/home/me/.cache/moonchild productions'
15M /home/me/.cache/moonchild productions

$ nohup palemoon &
[1] 21863

$ nohup: ignoring input and appending output to 'nohup.out'

$ # opens back up with just the 1 tab open to small local text file

$ du -sbh '/home/me/.cache/moonchild productions'
318K /home/me/.cache/moonchild productions

$ # Better, Apparently it waits until it restarts to actually delete the files,But there is STILL one clearly identifialble thumbnail that looks like a screen cap of a stackoverflow page.

- - - - end copy from terminal - - - -
I thought I knew what those buttons did. Apparently not. The obvious interpretation is that deleting the directory is just plain better than the cache clearing from the built in buttons, which seem to be reporting inaccurately, not deleting anything immediately, and not deleting some stuff that should be, even on shutting down and restarting. That seems implausible so I don't know what to make of it. I suspect I may be misunderstanding some fundamental definitions.

As a practical matter for me, I'll keep my rm menu entry. Next time I encounter the 'clear the browser cache' suggestion to fix something broken, I'll see if it suffices. It's certainly easier than hunting down the button(s) deep in the tools menu. I would like to understand why those buttons do less than I expected though, and particularly why they didn't delete the stackoverflow thumbnail even after restarting, but there is much of the universe I don't understand & so far it hasn't killed me.

Thanks for your input.
Last edited by Lew Rockwell Fan on 2018-03-19, 01:01, edited 4 times in total.

Lew Rockwell Fan

Re: scripted cache clearing

Unread post by Lew Rockwell Fan » 2018-03-19, 03:54

OK, after more testing, I can add:

Code: Select all

rm -r f "/home/user/.cache/moonchild productions"
deletes the tree as expected, but but stops caching altogether & PM does NOT resume caching until you stop and restart it.

However

Code: Select all

find "/home/user/.cache/moonchild productions" -type f -exec rm -f {} \;
removes all the files from the directory tree leaving less than 100 kb of directory structure amd PM continues caching normally, which would seem to be both more desirable and closer to the built in clear-cache function. Better in fact, since it actually works and gets rid of all the thumbnails & archives of saved scripts. It seems to me I've stumbled on a bug since shouldn't the built in function do this? Anybody see any problem in leaving a bunch of empty directories?

UPDATE: This zaps cookies too. In at least one place in the PM ui (History, Clear Recent History with a check boxed list following) "cache" is clearly used in a sense that excludes cookies (and for that matter some of the other stuff in the "cache" directory). I would exclude them or deal with them separately if I could figure out which file(s) they were in.

I think in the stuff I'm reading that kind of talks all around the subject, the word "cache" is not being used with consistent meaning. So I'm now bloody confused. Frequently you see advice along the line of "If such and such doesn't work, try clearing your browser cache." So what exactly, in terms of actual files, does that refer to? Everything in the /home/user/.cache/whatever_browser directory or some subset thereof?

It doesn't come up so often that I instantly remember the exact steps to clear cache (and apparently there are 2 precedures, assuming the one that starts in the History menu is equivalent to the one that starts in the Tools menu), but it does come up often enough to make me want to put it in a menu where it is easy to find. To do that I need a command.

What exactly does the "History, Clear Recent History, everything, check ONLY cache with everything else unchecked, Clear Now" delete? Is it the same thing that "Tools, prefs, advanced, net, Cached web content - Clear Now" deletes? And is either or both or neither what people have in mind when they suggest "clearing the cache" as a possible fix for a site that isn't working?
Last edited by Lew Rockwell Fan on 2018-03-19, 07:21, edited 2 times in total.

Lew Rockwell Fan

Re: scripted cache clearing

Unread post by Lew Rockwell Fan » 2018-03-20, 06:10

SOLVED, completely I think.
OK, I think I finally figured it out, if anybody is interested:

If you click:

"History" on the menu bar
"Clear Recent History"
select "Everything" on the drop down menu at the top
check "Cache"
uncheck everything else
click "Clear Now"

What it does in the cache directory for the profile "~/.cache/moonchild productions/pale moon/randomlookingstring.profilename" appears to be:

cache2/entries/ is emptied. Looks like (based on size of files) some of the files are being renamed & moved to cache2/doomed/ but most apparently are deleted.

cache2/index is deleted, not revised.

_CACHE_CLEAN_ contains a single byte & isn't touched. A config file maybe?
These directories and their contents are not touched:
thumbnails/
Cache/
OfflineCache/ - the sole file in this is index.sqlite which is not modified. Cookies maybe?
startupCache/

Some files in cache2/doomed/ aren't affected. Some are deleted. Some appear to have been moved and renamed from files in cache2/entries/

That's all.
= = = = = = = = = = =
None of the buttons that are supposed to put this function on a toolbar did anything like that for me.

Code: Select all

find "/home/user/.cache/moonchild productions" -type f -exec rm -f {} \;
is pretty good, but it deletes a lot more, notably including cookies. Think of it as "Nuke cache, cookies, and all their kin." It is much better than just deleting the whole cache directory because PM immediately resumes it's normal caching, and cookie baking, etc. I'll keep it on my menu. There is a place for nukes.

But I found something that does replicate the exact function of deleting cache through the history menu while making it a lot easier to find & has additional benefits:

https://addons.mozilla.org/en-US/firefo ... src=search

Works great for me. I configured it as a button that drops down a menu. And one of the preconfigured entries is clear cache. There is a panel mode too, that didn't interest me & I didn't test. As far as I can tell this is doing EXACTLY the same thing as the History,etc. action.

For each of the 3 native functions (1 through the history menu & 2 through the tools menu) and for prefbar, I made before and after summaries of files, directiories, sizes, and mod times using find with file and stat and so on. Then I generated lists of files that stayed the same, changed, disappeared, and were created with comm and sort. The second button in the tools menu was a red herring. It does something else altogether. I haven't studied the comparisons for tools,prefs,advanced,net,cached-web-content,clear-now as closely yet, but it appears to be about the same as the one under the history menu.
Last edited by Lew Rockwell Fan on 2018-03-20, 06:35, edited 2 times in total.

User avatar
adesh
Board Warrior
Board Warrior
Posts: 1277
Joined: 2017-06-06, 07:38

Re: scripted cache clearing

Unread post by adesh » 2018-03-20, 11:47

Lew Rockwell Fan wrote:is pretty good, but it deletes a lot more, notably including cookies. Think of it as "Nuke cache, cookies, and all their kin."
No, that does not delete cookies. Cookies along with most precious cargo reside in "~/.moonchild productions/pale moon/" separated info profile specific directories.

Lew Rockwell Fan

Re: scripted cache clearing

Unread post by Lew Rockwell Fan » 2018-03-21, 05:29

Ah, thank you. You are correct. I just tested it again. At some point in my testing my cookies disappeared & I thought that was why. I must have done something else and lost track of it. Sorry 'bout that. Well, find & delete all the files sure is a nice and simple approach. I wonder the significance of the stuff the history menu approach does not delete is. Other than my imaginary loss of cookies I haven't noticed any impairment of function from the more drastic find & delete.

Walter Dnes
Astronaut
Astronaut
Posts: 650
Joined: 2015-07-30, 20:29
Location: Vaughan, ON, Canada

Re: scripted cache clearing

Unread post by Walter Dnes » 2018-03-21, 08:29

See https://support.mozilla.org/en-US/kb/profiles-where-firefox-stores-user-data for detailed info on what is stored where. Pale Moon is a fork of Firefox.
There's a right way
There's a wrong way
And then there's my way

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35481
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: scripted cache clearing

Unread post by Moonchild » 2018-03-21, 14:40

Although I encourage people to set their browser up the way they prefer, I have to wonder why you're going this route.

Disk cache is there for a good reason. It's *supposed* to be somewhat permanent storage. Putting the disk cache in volatile RAM pretty much defeats the point of having it in the first place.
Also, the wear on an SSD would not be nearly as much as you'd think; cached items are generally not rotated out very fast. Actually clearing the cache every times would cause a lot of wear though because it would be re-filled every time, so you'd achieve the opposite if you would do this on an SSD.

If you really don't want the cache to be used, you can simply disable the disk cache altogether, and increase memory cache size. With the advent of cache v2 (Tycho+), memory cache is, in fact, efficiently used as opposed to the previous version which used memory cache more like a buffer than an actual cache.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

Lew Rockwell Fan

Re: scripted cache clearing

Unread post by Lew Rockwell Fan » 2018-03-22, 03:03

Thanks for the link, Walter. That looks interesting. Thanks for your input, Moonchild. I start stuff like this for practical reasons, then it turns into a puzzle that I simply will NOT let defeat me.
Moonchild wrote: . . . I have to wonder why you're going this route. . . .
I'm not sure which "you" & which "this" you mean. For me, like I said, I occasionally run into a site that isn't working correctly, where the admins suggest trying clearing cache. The last time, maybe a month ago, it was protonmail, and clearing cache did work. For a week or so, I had to do it every time I used that account, despite only having one account there.

It doesn't happen often, but when it does, it is annoying to have to poke around deep in a multi-layered gui, looking for something I know is there somewhere, button this, tab that, check this, uncheck that, click the other. I'm not a real gooey guy, that's all. It is much easier for me to quickly find and conveniently use it, if I put in a static, hand editable, hierarchical menu (like the ones typical of *box window managers). This is probably partly a matter of bad eyesight, partly a matter of taste, and partly a matter of brain wiring. My objective is merely to add an additional, & to me more convenient, ui for a particular function, not change the inner workings, nor the usage.
Moonchild wrote: . . . you can simply disable the disk cache altogether, and increase memory cache size. With the advent of cache v2 (Tycho+), memory cache is, in fact, efficiently used as opposed to the previous version which used memory cache more like a buffer than an actual cache.
Interesting. Hadn't read about that. Not practical for me, since my hardware resource ratios are pretty normal, & I tinker and reboot a lot anyway. But for somebody with scads of RAM and long uptimes that would make a lot of sense.

Some people might keep cache in RAM for security reasons. They probably should consider using a live-ROM system anyway and make the issue moot.

Anyway, I have 2 pretty good solutions now, the conservative Prefbar one which seems to be identical to the native function, and the scorced-earth find command. Both only 2 clicks. I'll study Walter's link to see what else my find is killing.

User avatar
adesh
Board Warrior
Board Warrior
Posts: 1277
Joined: 2017-06-06, 07:38

Re: scripted cache clearing

Unread post by adesh » 2018-03-22, 04:30

Lew Rockwell Fan wrote:I occasionally run into a site that isn't working correctly, where the admins suggest trying clearing cache. The last time, maybe a month ago, it was protonmail, and clearing cache did work. For a week or so, I had to do it every time I used that account, despite only having one account there.
As an alternative you can bypass cache for these sites by reloading the page using Ctrl+Shift+R key combination. It will update the cache also. Much better.
BTW, for me, if I face cache issues frequently with a site, it would raise serious questions about the design and implementation of the site.

Lew Rockwell Fan

Re: scripted cache clearing

Unread post by Lew Rockwell Fan » 2018-03-25, 03:34

adesh wrote:As an alternative you can bypass cache for these sites by reloading . . .
/me facepalms. Shoot. That simple? I should have thought of trying that. Next time I will. Thanks.

Locked