PM never releases memory?

Users and developers helping users with generic and technical Pale Moon issues on all operating systems.

Moderator: trava90

Forum rules
This board is for technical/general usage questions and troubleshooting for the Pale Moon browser only.
Technical issues and questions not related to the Pale Moon browser should be posted in other boards!
Please keep off-topic and general discussion out of this board, thank you!
yellowfinch

PM never releases memory?

Unread post by yellowfinch » 2019-07-19, 16:33

Am I the only one observing this behaviour?

Despite using almost exclusively a Private Window in PM for everything I do on the Web, I find that the amount of memory used by PM always increases with usage and never goes back down. Invoking the GC and/or CC functions brings it down by only a negligible amount.

While it starts at approximately 2.1GB, after a couple of weeks PM is often using half of my 12GB of RAM.

I have been observing this annoying behaviour from the first version of PM that I ever used (27.0.3) through all the subsequent ones (currently 28.5.2).

I have set browser.sessionstore.max_tabs_undo and browser.sessionstore.max_windows_undo to zero but it made no difference with respect to memory usage.

More context:

I tend to leave apps (including PM) running indefinitely, shutting them down only when I’m about to reboot the computer, which I normally do at most once a month. In the browser, I have a core of only 10 to 12 tabs that remain open permanently and that I reload once a day or more. Of course, I open and close many other tabs each day. I have been using my computers this way since the mid-90’s with Windows and since 2017 with Linux, which is when I started using PM. I never observed a memory-hogging problem with Firefox in Windows.

For a couple of years, I was running PM without disk caching (i.e. “Override...” and then setting the maximum disk cache space to zero) because that is how I had been using Firefox in Windows for almost then years and getting excellent results.

However, almost six months ago, I changed PM to let it use any amount of disk cache it wants. That hasn’t made any difference to the phenomenon I described at the top. In fact, I’ve been surprised at how little gets written to the disk cache folder, but what I find there makes sense given the tabs that are open.

So it seems that the memory bloating is not caused by true caching but rather simply by PM hanging on to everything that ever gets “loaded” when accessing Web pages and never releasing it, even when the possibility to undo the closing of tabs and the “session restore” feature are not requested and presumed to be turned off.

New Tobin Paradigm

Re: PM never releases memory?

Unread post by New Tobin Paradigm » 2019-07-19, 16:44

Maybe you should close the application every once in a while and not run it for weeks. Either that or download some more ram.

PalleP
Fanatic
Fanatic
Posts: 167
Joined: 2018-02-14, 17:36

Re: PM never releases memory?

Unread post by PalleP » 2019-07-19, 19:07

No, you are not the only one observing this behaviour, and it is not a Linux only problem.
I have Win7 32bit and therefore app. 3GB ram. PM starts with using app. 300mb and gets irresponsive when it passes app. 1GB PM. PM must then be force terminated, although there is still app. 1GB free memory.
I normally only have a few tabs open and PM will then run for some hours. When I compare prices, I will often open 10-15 tabs rapidly, and PM will completely choke by this. It is very annoying and has been a problem for me since v.28 was introduced, although one single 28 version did not seem to have this problem. I don't remember which one it was but something was done with the garbage collection.
If I try to free memory using about:memory it will, as you write, only have negligible effect. It may reduce the memory usage with only one or two hundred MB, and will never come even close to the 300GB PM started using.
So, here is how I open 10-15 tabs (ridiculous and annoying):
1) Find the product, by search engine or Ebay.
2) Open the first 10-15 results in new tabs.
3) Wait until PM is completely chocked.
4) Force terminate PM
5) Reopen PM
6) Watch the pages that was remembered by PM before it chocked.

vannilla
Moon Magic practitioner
Moon Magic practitioner
Posts: 2183
Joined: 2018-05-05, 13:29

Re: PM never releases memory?

Unread post by vannilla » 2019-07-19, 20:48

I don't know the internals of Pale Moon memory management, but I remember reading some time ago an article about memory usage in applications written in the Go language (I can't find the link right now, sorry): in short, when an application requires more memory, the Go runtime requests some more from the OS but never releases it even after the garbage collector is run. This means that if at some point the application had to allocate a gigabyte of memory, the application would hold onto that gigabyte forever (things might have changed since that article was written.)
It might be possible that Pale Moon does the same thing, requesting memory from the OS but never releasing it despite the garbage collector doing its job.
So yeah, if this is really the case then the best way to reduce memory usage is to close your applications more often than "before reboot".

yami_

Re: PM never releases memory?

Unread post by yami_ » 2019-07-19, 21:37

Off-topic:
vannilla wrote:
2019-07-19, 20:48
when an application requires more memory, the Go runtime requests some more from the OS but never releases it even after the garbage collector is run
Go runtime will release unused memory pages, if those pages are no longer needed. Writing any non-trivial Go application would be impossible otherwise.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35473
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: PM never releases memory?

Unread post by Moonchild » 2019-07-19, 22:48

I run Pale Moon for very long sessions myself, and don't experience issues with memory not being released.
If there was a systematic issue with memory management, then you would notice this in a very short amount of time because web browsing allocates and releases a LOT of memory, constantly. Every image in every page for example has to be decoded, sent to a drawing layer, and composited in the page.

Now, there is a different issue which can occur and that is that pages themselves don't release memory. Especially if you keep tabs open for a long time that have dynamically updated content then there's a good chance that your memory consumption is caused by pages not releasing memory -- that is however not something the browser can force because the browser is doing exactly what it is told to do in that case.

If people are curious: the memory allocator used by Pale Moon is a specifically-patched version of jemalloc. It's patched to solve some sec issues and to tune it to typical use in a web browser.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

yellowfinch

Re: PM never releases memory?

Unread post by yellowfinch » 2019-07-20, 00:51

Moonchild wrote:
2019-07-19, 22:48
Especially if you keep tabs open for a long time that have dynamically updated content then there's a good chance that your memory consumption is caused by pages not releasing memory
In that case, shouldn't I see the memory usage go down substantially when I close the offending tabs? That's not what I'm observing.

99HVYSK
Apollo supporter
Apollo supporter
Posts: 42
Joined: 2018-11-20, 12:21

Re: PM never releases memory?

Unread post by 99HVYSK » 2019-07-20, 05:23

yellowfinch wrote:
2019-07-20, 00:51
In that case, shouldn't I see the memory usage go down substantially when I close the offending tabs? That's not what I'm observing.
at least for Gmail is not the case, in some occasion, close tab will make it ghost, which will eventually doom palemoon. although palemoon get several fix to address the issue, but it couldn't resolve it completely (at least for my experience). since the day i ditched Gmail, I had never kill palemoon for memory drain issue.

check about:memory see if you get any ghost window.
Pale Moon v29, pentadactyl, eMatrix, Arch Linux, twelve years old ThinkPad X200

User avatar
adesh
Board Warrior
Board Warrior
Posts: 1277
Joined: 2017-06-06, 07:38

Re: PM never releases memory?

Unread post by adesh » 2019-07-20, 06:21

Moonchild wrote:
2019-07-19, 22:48
I run Pale Moon for very long sessions myself, and don't experience issues with memory not being released.
I think the issue is with private browsing mode. Pale Moon uses RAM/tmpfs (instead of disk) for caching web content in PB mode and it keeps piling up as you keep browsing. This usage doesn't decrease even if you close (a lot of) tabs. It's only after _last_ private window is closed that the memory is released.

PalleP
Fanatic
Fanatic
Posts: 167
Joined: 2018-02-14, 17:36

Re: PM never releases memory?

Unread post by PalleP » 2019-07-20, 07:45

99HVYSK wrote:
2019-07-20, 05:23
yellowfinch wrote:
2019-07-20, 00:51
In that case, shouldn't I see the memory usage go down substantially when I close the offending tabs? That's not what I'm observing.
at least for Gmail is not the case, in some occasion, close tab will make it ghost, which will eventually doom palemoon. although palemoon get several fix to address the issue, but it couldn't resolve it completely (at least for my experience). since the day i ditched Gmail, I had never kill palemoon for memory drain issue.

check about:memory see if you get any ghost window.
I would also expect unloaded pages to release memory no matter how ram hungry they are. Gmail is a potential culprit because of the ghost window, but I have not observed other Google services to produce ghost windows.
Using sat view in Google maps will rather quickly choke my PM, and if I manage to close it before PM gets totally frozen, the memory will not be released, not even when using the tools in about:memory. The only way to make PM work again is to restart or terminate PM. The restart function will not work when PM is about to choke, the confirmation window pops up and after "yes" is pressed, nothing happens even if patience lets me wait ½ an hour.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35473
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: PM never releases memory?

Unread post by Moonchild » 2019-07-20, 10:45

adesh wrote:
2019-07-20, 06:21
I think the issue is with private browsing mode. Pale Moon uses RAM/tmpfs (instead of disk) for caching web content in PB mode and it keeps piling up as you keep browsing. This usage doesn't decrease even if you close (a lot of) tabs. It's only after _last_ private window is closed that the memory is released.
Private browsing mode isn't necessarily designed to be used for exteremely long sessions. Even "permanent PB mode" (setting it to never remember history) was only designed to be used in setups where you would have relatively limited sessions (maybe a couple hours max), e.g. library computer use. If you are using that mode as per normal in long-standing sessions then you are likely not entirely understanding what PB is for and what it does and doesn't do. PB mode only keeps your -local- computer use private, meaning nothing will be saved locally of your activities in your session. It does not make you private on the web or even to your local network's administration. Its only potential side-effect benefit for the web is that it doesn't retain information across sessions that you might not know about. If you run long sessions in PB mode, you lose even that side-effect benefit because the session itself will act exactly like a non-private session otherwise (with some specific setups in website frameworks potentially not working as a negative). There really is no point in trying to use it that way, and of course with the necessary temporary storage in memory required to be able to run a browsing session that way you will indeed see a slow increase there over time. All of that is as-expected.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

PalleP
Fanatic
Fanatic
Posts: 167
Joined: 2018-02-14, 17:36

Re: PM never releases memory?

Unread post by PalleP » 2019-07-20, 20:00

I don't use private browsing but have the problem anyway.

New Tobin Paradigm

Re: PM never releases memory?

Unread post by New Tobin Paradigm » 2019-07-20, 20:12

PalleP wrote:
2019-07-20, 20:00
I don't use private browsing but have the problem anyway.
Uhh huh, uhh huh *nods then walks away*.

PalleP
Fanatic
Fanatic
Posts: 167
Joined: 2018-02-14, 17:36

Re: PM never releases memory?

Unread post by PalleP » 2019-07-20, 20:52

New Tobin Paradigm wrote:
2019-07-20, 21:13
Uhh huh, uhh huh *nods then walks away*.
Is this supposed to be helpful?

I have now since v.28 was introduced tried to get help with the choking PM, but there is no progress. Maybe this helpful reply will do the job :roll:
Last edited by PalleP on 2019-07-21, 07:09, edited 1 time in total.

New Tobin Paradigm

Re: PM never releases memory?

Unread post by New Tobin Paradigm » 2019-07-20, 21:13

PalleP wrote:
2019-07-20, 20:52
I have now since v.28 was introduced tried to get help with the choking PM, but there is no progress. Maybe this helpful reply will do the job :roll:
A lot of things HAVE been resolved. Implying they haven't been is just silly. Maybe it is your computer or configuration. Have you considered that? I mean what is your basis uhh based on? Seen the same thing happen on other systems?

Off-topic:
Now if you excuse me, we are celebrating a great achievement in my channel and I don't want to be distracted from it.

yellowfinch

Re: PM never releases memory?

Unread post by yellowfinch » 2019-07-20, 21:59

Moonchild wrote:
2019-07-20, 10:45
you are likely not entirely understanding what PB is for and what it does and doesn't do.
I do understand what PB does and my reasons for using it have nothing to do with privacy. It prevents my hard drive from being littered with histories that I don't need. It also reduces wear on that drive, by limiting the amount of activity on it. That's why I've been using it for years and it works great. (I've only recently changed my computer after using the same laptop for 15 years. And it was running 24/7!)

From my point of view, the only drawback of PB is the memory hogging. But now that the issue is confirmed, it won't bug me as much.

Although...

I still don't understand why the “necessary temporary storage in memory required to be able to run a browsing session that way” doesn't get released when I close tabs.

New Tobin Paradigm

Re: PM never releases memory?

Unread post by New Tobin Paradigm » 2019-07-20, 22:06

Because Private Browsing is on a per-window basis NOT per-tab. So you won't release some stuff until the specific Private Browsing window is closed. So, you may want to de-emphasize your tabbed browsing and return to old school window'd browsing. That would fit your specific requirements better.

Of course, you could just have the application clear your private data on shutdown and use the thing normally.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35473
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: PM never releases memory?

Unread post by Moonchild » 2019-07-21, 01:10

It prevents my hard drive from being littered with histories that I don't need. It also reduces wear on that drive, by limiting the amount of activity on it.
Am I right in assuming you are thinking that normal browser use will cause disproportionate wear on your drive? And that you are using an SSD? And that you still think that SSDs will fail in a short amount of time unless you treat them like something super fragile? Unless you're using one of the very first generation SSDs with an uncached "dumb" controller on it, there's really no reason to treat it any different than a platter drive (aside from defragging which should be done much less often and with specialized algorithms).

Browser history is stored in an sqlite database (along with your bookmarks and a number of other things) called the places database. This database will be accessed in normal use whether you are in private browsing mode or not, so thinking it will "reduce wear" by not writing history is a bit of a fallacy.
yellowfinch wrote:
2019-07-20, 21:59
I still don't understand why the “necessary temporary storage in memory required to be able to run a browsing session that way” doesn't get released when I close tabs.
As Tobin already pointed out it's per window. All your tabs in a private browsing window will be using the same shared pool of resources which, by design, isn't released until you end your PB -session- (meaning you close the window).
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

PalleP
Fanatic
Fanatic
Posts: 167
Joined: 2018-02-14, 17:36

Re: PM never releases memory?

Unread post by PalleP » 2019-07-21, 07:05

New Tobin Paradigm wrote:
2019-07-20, 21:13

Maybe it is your computer or configuration. Have you considered that?

Yes, of course I have. I find it hard to blame my PC, knowing that my hundreds of other programs runs troublefree.
I have even tried to install other browsers with extensions just for test (eg. Iridium and Firefox). They also run troublefree. Am I still supposed to believe that PM is not the culprit?
Unfortunately I don't like the other browsers interface, that is why I still after all this time and frustration try to make PM behave acceptable, instead of just using another browser.

New Tobin Paradigm wrote:
2019-07-20, 21:13

Seen the same thing happen on other systems?

Yes, I have a very similar PC and it does exactly the same. My other pc's are labtops and are only briefly used for browsing, so I won't notice the bug. I only use 32bit Win7.

Win7 Ultimate 32 bit
Dell XPS 420 with Q9450 cpu and 4GB RAM.
M$E AV
Disabling extensions does not help.

User avatar
adesh
Board Warrior
Board Warrior
Posts: 1277
Joined: 2017-06-06, 07:38

Re: PM never releases memory?

Unread post by adesh » 2019-07-21, 09:34

PalleP wrote:
2019-07-21, 07:05
Disabling extensions does not help.
Try with a vanilla profile.

What are the browser settings you have tweaked? It'd be good if you publish your "troubleshooting info" here so somebody can point out the faulty configuration. See the stickies for that.

Locked