Pale Moon User Interface Freezes
Forum rules
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific bugs, git, the repositories, etc.
This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Please post issues with specific websites, extensions, etc. in the relevant boards for those topics.
Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific bugs, git, the repositories, etc.
This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Please post issues with specific websites, extensions, etc. in the relevant boards for those topics.
Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
-
- Keeps coming back
- Posts: 940
- Joined: 2021-01-26, 11:18
Re: Pale Moon User Interface Freezes
As far as I understand, the problem is not with jemalloc. There is a leak of memory blocks that are not java objects directly. That is, these could be some kind of auxiliary dynamic structures.
-
- Hobby Astronomer
- Posts: 22
- Joined: 2023-07-12, 00:14
Re: Pale Moon User Interface Freezes
I'm just trying to help with the analysis...
I'd suggest trying another malloc, if it helps... But please have it compatible at least with Windows 7...
For a test case - open four or more webpages and reload first, second, third, fourth, and again first, second, fourth, load another... Some heavyweight as today's WordPress... This way they are allocated in between each other, and releasing memory of one keeps the pages held by the other... If you repeatedly reload just one webpage at the end, it may properly release all, but that is not a real browsing pattern probably of many users...
Even in such case it should reuse the existing chunks and arenas and not acquire new and new ones...
πα½
The excess pages belong to mozglue.dll (i.e. various malloc() and free(), also used by some xul.dll objects...), not to mozjs.dll where is GC code... GC (jsgc.cpp and Allocator.cpp) probably uses its own pages allocated by VirtualAlloc elsewhere than the pages of mozglue.dll ... (just an assumption by studying the source code...)athenian200 wrote: ↑2024-02-11, 16:20... some assumptions made by the GC to reflect the fact that we're using something else ...
For a test case - open four or more webpages and reload first, second, third, fourth, and again first, second, fourth, load another... Some heavyweight as today's WordPress... This way they are allocated in between each other, and releasing memory of one keeps the pages held by the other... If you repeatedly reload just one webpage at the end, it may properly release all, but that is not a real browsing pattern probably of many users...
Even in such case it should reuse the existing chunks and arenas and not acquire new and new ones...
πα½
-
- Contributing developer
- Posts: 1537
- Joined: 2018-10-28, 19:56
- Location: Georgia
Re: Pale Moon User Interface Freezes
We split a lot of components out of xul.dll that were merged into it by Mozilla in the past. And also Mozilla had some weird sandboxing code we got rid of very early on. I don't think mozglue itself is something we've touched very much, though. As far as I know, though, there shouldn't really be anything in mozglue dealing with memory allocation, unless some third-party DLLs are doing their own memory allocation and it's causing problems.P_A_Semi wrote: ↑2024-02-11, 18:51The excess pages belong to mozglue.dll (i.e. various malloc() and free(), also used by some xul.dll objects...), not to mozjs.dll where is GC code... GC (jsgc.cpp and Allocator.cpp) probably uses its own pages allocated by VirtualAlloc elsewhere than the pages of mozglue.dll ... (just an assumption by studying the source code...)
https://repo.palemoon.org/MoonchildProd ... er/mozglue
From what I can tell, everything we've done to mozglue is basically code clean up, changes to DLL blocklists, and maybe adding a couple utility functions. EDIT: There is a very small amount of code touching memory in there, but it all seems to be behind the MOZ_LINKER flag and is related to opening third-party libraries from what I can tell.
So, the issue you're talking about... is it anything we introduced, or is it something Mozilla did before our fork point? You keep mentioning Firefox 45... was the bug you're talking about with memory allocation introduced in Firefox 45 and we still have it? If the issue first appeared in Firefox 45, we would have to go back and look at what Mozilla changed then, and see if we can figure out how they messed up.
I gotta admit, I don't think we have the skills needed to pin down exactly where this is happening in our codebase on the basis of what's happening in DLL files. We don't really look at the codebase in terms of what the final binaries are doing, we usually just see the source code that generates them, and rarely look at low-level stuff like jemalloc or mozglue. I've seen a few other programs in hex editors that way, but not ours really. All we can do is maybe work with a regression window to figure out what caused the problem to begin with, in order to figure out what component it might be in.
"The Athenians, however, represent the unity of these opposites; in them, mind or spirit has emerged from the Theban subjectivity without losing itself in the Spartan objectivity of ethical life. With the Athenians, the rights of the State and of the individual found as perfect a union as was possible at all at the level of the Greek spirit." -- Hegel's philosophy of Mind
-
- Add-ons Team
- Posts: 582
- Joined: 2017-01-14, 02:40
- Location: Philippines
Re: Pale Moon User Interface Freezes
Off-topic:
Speaking of ghost windows, I've recently found out that there is a way to manually unlink them via the memory manager. I tried exposing it to about:memory and it seems to get rid of the ghost windows, but as the comment in the code says, "This is only for debugging leaks, and can cause bad things to happen if called."
-
- Keeps coming back
- Posts: 940
- Joined: 2021-01-26, 11:18
Re: Pale Moon User Interface Freezes
-
- Hobby Astronomer
- Posts: 22
- Joined: 2023-07-12, 00:14
Re: Pale Moon User Interface Freezes
While I remember time, when Netscape Navigator was renamed to Mozilla, after Mozilla I've been using only Firefox 3, 19 and 45, only upgrading when it was unsustainable to use the old version... (And I'm not going to use any newer Firefox version, since it does not support saving MHT files, which I crucially need... Great thanks for the Palemoon browser, which is now more "Firefox" than the new Firefox itself... I keep using Firefox 45 only marginally for administrative tasks and few forums where it remembers the login I don't, doing all news reading in Palemoon...)athenian200 wrote: ↑2024-02-11, 20:48... is it anything we introduced, or is it something Mozilla did before our fork point? You keep mentioning Firefox 45...
From what I can tell, the Firefox 45 has the same inefficient jemalloc as Palemoon, just it endures a little more, not any much more, probably because some objects were smaller...??
To further illustrate the problem, the attachments are from about:memory shortly before the crash, when Palemoon started to have repaint problems...
There were 554 Mb "reserved", i.e. decommited memory, wasting address space... (while it's right to decommit unused memory, so that it spares physical memory, but it must be reused, when new memory is needed, otherwise address space clogs, as there are only 2Gb available in 32-bit version...)
On the last line of second attachment, it shows there was only 13 Mb max "contiguous" space left, all rest of address space clogged...
Also the difference between heap-allocated and heap-mapped shows a lot of memory wasting, not reusing free blocks properly...
(this is from Palemoon version 32.2.1 but I doubt it would differ in newer release...)
πα½
You do not have the required permissions to view the files attached to this post.
-
- Hobby Astronomer
- Posts: 22
- Joined: 2023-07-12, 00:14
Re: Pale Moon User Interface Freezes
I offer you a "Contribution" to solve this problem (at least temporarily before you replace malloc), or to show you the difference...
I've written replacement Heap memory manager, revectoring all relevant functions from mozglue.dll to handle them little faster and substantially more efficiently...
MozMem.zip can be downloaded from my website...
User "installation" is described there in Readme.txt file: unpack, backup api-ms-win-crt-heap-l1-1-0.dll in Bin\Palemoon folder, and replace it with my version, and restart Palemoon...
Just like that...
It is only available for 32-bit Windows version of Palemoon...
I know you don't use the heap functions from that DLL much, but when the DLL is loaded, it revectors all relevant functions in mozglue.dll :
malloc,realloc,free,calloc,recalloc,_aligned_malloc,malloc_usable_size,posix_memalign (and by that also moz_xmalloc,strdup etc depending on the lower-level ones...)
Palemoon is noticeably (a little, I think like 10% but I don't know how to measure that) faster with this, especially the memory intensive tasks, it endures a lot more (about half more), before it crashes from out-of-memory, while it uses more memory (while the original crashed with 800Mb of memory reported by Windows TaskManager, this one started to have repaint problems with 1300Mb memory reported, and a lot later... this solves problems only at the heap level, but not the leaks themselves... but memory is reused around the leaks well...)
Tested with Palemoon 32.2.1 on Windows 7 32-bit, and on somehow outdated Windows 10 64-bit with that same 32-bit Palemoon...
It doesn't touch GC blocks, it just revectors all jemalloc usage...
(There is a programming tool included, that dumps block usage summary...)
There are still some known problems:
- about:memory does not report this new heap, and it shows negative value of heap-unclassified... (I could also reimplement jemalloc_stats() ...)
- during startup, your debugger may report it causes exceptions, while it searches for kernel32.dll ... if you consider that a problem, I could rewrite it a little to avoid this... normal user does not notice that, I don't know if your msvc debugger would report it...?
Since due to hardware failure of my development computer last summer, I don't have any compiler here on Notebook beside NASM assembler, and my scripting language... No linker, no real debugger beside my TaskView to observe memory and stacks... I took memory manager from one of my recent programs, that has been written 80% in assembler and 20% in Delphi Pascal, and rewritten all the rest into assembler... About a week time, more than 50 hours programming and observing, not counting the time I read news in Palemoon to test it - just for this port... (The original version I reused was few weeks of work and debugging etc...) The DLL has 15kb when compiled, the source is over 300kb in assembler...
Potentially I could rewrite it to an opaque function MemMgr* InitMmAlloc(HANDLE hKernel32), that would get GetModuleHandleA('kernel32.dll'), and return struct with functions malloc,realloc,free, ... etc... You would properly #ifdef that to 32-bit windows version...
Or you can use this DLL as it is... Any user can use that even without your action...
The assembler source is included with instructions, how to compile it, and where to download nasm version needed, from SourceForge... (my code is not on SourceForge or git or elsewhere...)
The small blocks are thread-safe even without locking, the middle (above 256 bytes) and large blocks use EnterCriticalSection locking... Small blocks are aligned on their size, at least 16 byte alignment, the middle blocks are aligned on 8-byte boundaries, just _aligned_malloc aligns them on 32 bytes, and large blocks are page-aligned (4k)... The middle blocks use same in-page format as Kernel memory in Windows XP... Since the memory format is shared with my program, my tool TaskView auto-detects memory block boundaries when watching memory - TaskView while it runs also on 64-bit windows, but it can view only memory and annotated stacks of 32-bit processes...
Potentially I could add tagging middle and large blocks (i.e. over 256 bytes) with the address calling malloc() to aid hunting the leaks, since there is spare dword (uint32) available... It is not possible with smaller blocks at the heap level, as they are tightly packed...
I assume you would probably not use it, when you find another malloc replacement (because I won't provide unix or win64 versions), since the old version of jemalloc you use is unsustainable...
(but if you decide so, you may redistribute it or modify it as you like... and you may request some modification from me...)
πα½
I've written replacement Heap memory manager, revectoring all relevant functions from mozglue.dll to handle them little faster and substantially more efficiently...
MozMem.zip can be downloaded from my website...
User "installation" is described there in Readme.txt file: unpack, backup api-ms-win-crt-heap-l1-1-0.dll in Bin\Palemoon folder, and replace it with my version, and restart Palemoon...
Just like that...
It is only available for 32-bit Windows version of Palemoon...
I know you don't use the heap functions from that DLL much, but when the DLL is loaded, it revectors all relevant functions in mozglue.dll :
malloc,realloc,free,calloc,recalloc,_aligned_malloc,malloc_usable_size,posix_memalign (and by that also moz_xmalloc,strdup etc depending on the lower-level ones...)
Palemoon is noticeably (a little, I think like 10% but I don't know how to measure that) faster with this, especially the memory intensive tasks, it endures a lot more (about half more), before it crashes from out-of-memory, while it uses more memory (while the original crashed with 800Mb of memory reported by Windows TaskManager, this one started to have repaint problems with 1300Mb memory reported, and a lot later... this solves problems only at the heap level, but not the leaks themselves... but memory is reused around the leaks well...)
Tested with Palemoon 32.2.1 on Windows 7 32-bit, and on somehow outdated Windows 10 64-bit with that same 32-bit Palemoon...
It doesn't touch GC blocks, it just revectors all jemalloc usage...
(There is a programming tool included, that dumps block usage summary...)
There are still some known problems:
- about:memory does not report this new heap, and it shows negative value of heap-unclassified... (I could also reimplement jemalloc_stats() ...)
- during startup, your debugger may report it causes exceptions, while it searches for kernel32.dll ... if you consider that a problem, I could rewrite it a little to avoid this... normal user does not notice that, I don't know if your msvc debugger would report it...?
Since due to hardware failure of my development computer last summer, I don't have any compiler here on Notebook beside NASM assembler, and my scripting language... No linker, no real debugger beside my TaskView to observe memory and stacks... I took memory manager from one of my recent programs, that has been written 80% in assembler and 20% in Delphi Pascal, and rewritten all the rest into assembler... About a week time, more than 50 hours programming and observing, not counting the time I read news in Palemoon to test it - just for this port... (The original version I reused was few weeks of work and debugging etc...) The DLL has 15kb when compiled, the source is over 300kb in assembler...
Potentially I could rewrite it to an opaque function MemMgr* InitMmAlloc(HANDLE hKernel32), that would get GetModuleHandleA('kernel32.dll'), and return struct with functions malloc,realloc,free, ... etc... You would properly #ifdef that to 32-bit windows version...
Or you can use this DLL as it is... Any user can use that even without your action...
The assembler source is included with instructions, how to compile it, and where to download nasm version needed, from SourceForge... (my code is not on SourceForge or git or elsewhere...)
The small blocks are thread-safe even without locking, the middle (above 256 bytes) and large blocks use EnterCriticalSection locking... Small blocks are aligned on their size, at least 16 byte alignment, the middle blocks are aligned on 8-byte boundaries, just _aligned_malloc aligns them on 32 bytes, and large blocks are page-aligned (4k)... The middle blocks use same in-page format as Kernel memory in Windows XP... Since the memory format is shared with my program, my tool TaskView auto-detects memory block boundaries when watching memory - TaskView while it runs also on 64-bit windows, but it can view only memory and annotated stacks of 32-bit processes...
Potentially I could add tagging middle and large blocks (i.e. over 256 bytes) with the address calling malloc() to aid hunting the leaks, since there is spare dword (uint32) available... It is not possible with smaller blocks at the heap level, as they are tightly packed...
I assume you would probably not use it, when you find another malloc replacement (because I won't provide unix or win64 versions), since the old version of jemalloc you use is unsustainable...
(but if you decide so, you may redistribute it or modify it as you like... and you may request some modification from me...)
πα½
-
- Moon Magic practitioner
- Posts: 2411
- Joined: 2012-08-19, 20:32
Re: Pale Moon User Interface Freezes
Safe Mode / clean profile info: Help/Restart in Safe Mode
Information to include when asking for support - How to apply user agent overrides
How to auto-fill passwords
Windows 10 Pro x64
Information to include when asking for support - How to apply user agent overrides
How to auto-fill passwords
Windows 10 Pro x64
-
- Pale Moon guru
- Posts: 35651
- Joined: 2011-08-28, 17:27
- Location: Motala, SE
Re: Pale Moon User Interface Freezes
That isn't a solution. That dll is part of the Microsoft CRT run-time libraries which we cannot change (and which we semi-regularly update to new versions of the run-time). If there is a bug in that library then you should take that up with Microsoft.
If you want to provide something to help, please provide a solution that lies within our scope of code.
In the meantime I have to strongly advise against anyone using an unverified binary DLL replacement. I have no knowledge of what all was changed (especially with it all being ASM it's not easy to audit) and have no knowledge whether the introduced code can be trusted to not be a security risk/malicious at this time.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
-
- Newbie
- Posts: 6
- Joined: 2021-12-13, 03:52
- Location: Jerusalem
Re: Pale Moon User Interface Freezes
In the first version of xul.dll compiled for sse and non sse2 cpu's were require just two api's that never were included in the firefox releases: api-ms-win-core-winrt-l1-1-0.dll, api-ms-win-core-winrt-string-l1-1-0.dll, both of 3,584 kb. It affects how fast the pages are loading with these missing, but does not applies to any version of xul, even so may apply to Palemoon because is based on older versions of mozilla.
-
- Pale Moon guru
- Posts: 35651
- Joined: 2011-08-28, 17:27
- Location: Motala, SE
Re: Pale Moon User Interface Freezes
Note: i removed the archive with unsigned dlls from the post above.
Please don't just toss dlls in to "see what sticks". Winrt libs will be N/A for pale Moon to begin with since it's not a WinRT application. On top, mixing and matching run-time libs from different run-time versions will often cause issues.
Please don't just toss dlls in to "see what sticks". Winrt libs will be N/A for pale Moon to begin with since it's not a WinRT application. On top, mixing and matching run-time libs from different run-time versions will often cause issues.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
-
- Hobby Astronomer
- Posts: 22
- Joined: 2023-07-12, 00:14
Re: Pale Moon User Interface Freezes
This was just a hot-fix solution, so that a middle-level user, who is able to unpack, copy and rename a file, can apply it... (So that they should be able to undo these changes in case of any problem...)
The DLL can also be renamed to whatever else (possibly using the original MozMem_v12.dll), and placed in dependentlibs.list file just after the line of mozglue.dll , it's sufficient to load it thus and it takes care of all else...
Sometimes it is more interesting, what is not there... There is no compression to hide anything... If you inspect the source or the binary, there is none of the words "file","net","connect","load", or similar... It will not infect files, it will not connect anywhere... There is GetModuleHandle and GetProcAddress, so that it operates only on what you already loaded...
(I cannot guarantee it is bug-free... I'm already working on MozMem_v13.dll , that could be more optimized... It is a question, whether it would be sufficiently stable and not crash the program ? Meanwhile it seems so, but you cannot tell without testing it for some time... Bugs like "reallocating large block to small odd size not copying the last byte" I already catched and corrected before releasing it...)
(There is word "write" in "LogWrite", but removed by #ifdef from released version, and it writes only to in-memory cyclic buffer... I used that during debugging and may need it again in case of catching some possible problem...)
Either in source or in the binary DLL, you see the names of 9 functions related to memory management, that are being replaced...
(There is also code, that if you loaded AltD5.bpl from my Delphi programs, which you don't, it would use SetMemoryManager to replace memory manager there, which I use for testing that... In the source there is behind unused %ifdef a code to place stop-point in xul.dll NS_DebugBreak, but it is not used in released version...)
I understand that you are a group of C and Javascript programmers and have no expertise in low-level assembler, I wouldn't expect that, since then you could replace the memory-manager yourself...
I also understand there are high standards of commenting the source and making it manageable by other team members, which standards my code does not meet... Sorry...
But please don't be jealous... On the other hand, I have problems orienting in C language with all its defines... A good memory manager is a low-level component, that does very precisely what it should, and that you don't need to care about, when writing higher-level code... Very typically written by different team members, or brought as a ready-made solution from elsewhere... (But you on higher levels should occasionally free everything you malloc, beside things that are living forever until program terminates, and that do not accumulate over time... And foremost you must not write into memory you free... Jemalloc could possibly tolerate such bug better...)
Would you expect of race driver champion (you) to also service and replace his tyres ? There are team members for that (I now try to volunteer in that role... And yes, you are competing with Chrome and newer Firefox for users, while I'm competing with jemalloc, Microsoft or other memory-manager suppliers as a tyre serviceman, telling you, that in your competition you have disadvantage of an outdated "tyre", a low-level tool...)...
I've written I offered it as a "temporary solution", and an incentive to find some newer "malloc" implementation, than the outdated jemalloc, which is there since very long ago... And showing, that and how it may help, and what it will not solve...
Sorry, there is no other gray color in the list, and it is used for almost-off-topic remarks that you can safely skip, reading only the non-colored text...back2themoon wrote: ↑2024-02-21, 11:58Please avoid using this colour because text becomes nearly invisible in the dark themes available here - thanks.
(Would you prefer darker or lighter than 808080 ?)
Just an advise, that if there is somewhere a poorly readable text color (not just my writing but anywhere on web-pages), you can Ctrl+A select all the text and read it inversed and mono-color...
(Which I use if someone has web-page in dark mode, black background and white letters, which I dislike... So I invert it thus simply using Ctrl+A...) (I know I write too much, so by using gray color on less important things I'm trying to help others to skip it and ignore the gray parts...)
πα½
-
- Contributing developer
- Posts: 1537
- Joined: 2018-10-28, 19:56
- Location: Georgia
Re: Pale Moon User Interface Freezes
So, wait... are you saying that your version of the dll effectively overrides jemalloc and uses the system malloc for Windows? Or are you saying that the heap allocation dll from the CRT that we're loading in normally is already partially overriding jemalloc in ways we wouldn't expect, and you're just compensating for that?P_A_Semi wrote: ↑2024-02-21, 01:16I know you don't use the heap functions from that DLL much, but when the DLL is loaded, it revectors all relevant functions in mozglue.dll :
malloc,realloc,free,calloc,recalloc,_aligned_malloc,malloc_usable_size,posix_memalign (and by that also moz_xmalloc,strdup etc depending on the lower-level ones...)
Anyway, I would recommend for anyone interested in testing this out to try it in a VM or something, don't replace this dll in your main Pale Moon installation because we can't really audit this or support it. That said, I will be somewhat impressed from a programming/engineering perspective if someone managed to partially resolve memory issues in a stable manner by just messing around with an assembler on an old laptop! That doesn't seem like it should be possible, memory management is a pretty complex thing... seems like it would require at least knowledge of C/C++ for our codebase? It would be very hard to deal with memory at that low a level. The closest I got to doing something like that in the past was admittedly a bit shady... when I was a kid I sometimes wanted to cheat in games that didn't have official cheats, and would just mess around with ways of modifying the application's memory directly on a lot of games that didn't have any anti-cheat protection, sort of like a very advanced game genie. To think you're actually trying to rework the way memory is used in a serious way, from the type of tools I've only ever used to mod or cheat at games and hack support for 4:3 resolutions into "16:9 only games" so they look good on my old CRTs...
So the leaks are a separate problem, what you've solved here is an issue with the heap allocation itself, which is dealt with in jemalloc, right? If I'm following along with you correctly.Palemoon is noticeably (a little, I think like 10% but I don't know how to measure that) faster with this, especially the memory intensive tasks, it endures a lot more (about half more), before it crashes from out-of-memory, while it uses more memory (while the original crashed with 800Mb of memory reported by Windows TaskManager, this one started to have repaint problems with 1300Mb memory reported, and a lot later... this solves problems only at the heap level, but not the leaks themselves... but memory is reused around the leaks well...)
I can imagine that being on 32-bit had a lot to do with your motivation to solve this issue... for context, I develop Pale Moon on a system 32GB of RAM, and tend not to encounter problems nearly as quickly...Tested with Palemoon 32.2.1 on Windows 7 32-bit, and on somehow outdated Windows 10 64-bit with that same 32-bit Palemoon...
"The Athenians, however, represent the unity of these opposites; in them, mind or spirit has emerged from the Theban subjectivity without losing itself in the Spartan objectivity of ethical life. With the Athenians, the rights of the State and of the individual found as perfect a union as was possible at all at the level of the Greek spirit." -- Hegel's philosophy of Mind
-
- Pale Moon guru
- Posts: 35651
- Joined: 2011-08-28, 17:27
- Location: Motala, SE
Re: Pale Moon User Interface Freezes
Don't get me wrong, I find it very interesting and I appreciate your attempt at trying to solve inefficiencies in the Windows CRT. I'm not "jealous" or anything as you seem to think, but I'm approaching this from a security-first perspective. Bugs in any memory allocation implementation (intentional or not) will almost certainly cause critical security vulnerabilities in a program that deals with foreign and untrusted content so I advise against using it for the average user. You can if you want to, but be aware we can't really guarantee proper operation of you exchange run-time DLLs like this and you may be putting yourself at considerable risk, even if there is no overt calling to obvious risky malware routines.
Being on 32-bit, your implementation will likely have significant impact for your use case as I expect Microsoft to take a safer-but-slower approach in their system run-times. There's room for improvement, but it'd be something we'd have to let the authors of the API tackle. Once again if you're interested in solving this in more than a hack-y/temporary way, then you're welcome to look at our malloc implementation and work with that to contribute.
Being on 32-bit, your implementation will likely have significant impact for your use case as I expect Microsoft to take a safer-but-slower approach in their system run-times. There's room for improvement, but it'd be something we'd have to let the authors of the API tackle. Once again if you're interested in solving this in more than a hack-y/temporary way, then you're welcome to look at our malloc implementation and work with that to contribute.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
-
- Moon Magic practitioner
- Posts: 2411
- Joined: 2012-08-19, 20:32
Re: Pale Moon User Interface Freezes
Off-topic:
I avoid using any colour, and only extremely rarely use red to highlight a single word (or even a single letter). Using colours on big blocks of text can never look good, and can also lead to unexpected results such as this. Use italics, parentheses or provided tags such as 'offtopic' and 'hide'. Ctrl-A is a bad workaround. These are my personal preferences of course, feel free to use whatever you want.
Safe Mode / clean profile info: Help/Restart in Safe Mode
Information to include when asking for support - How to apply user agent overrides
How to auto-fill passwords
Windows 10 Pro x64
Information to include when asking for support - How to apply user agent overrides
How to auto-fill passwords
Windows 10 Pro x64
-
- Hobby Astronomer
- Posts: 22
- Joined: 2023-07-12, 00:14
Re: Pale Moon User Interface Freezes
Sorry for a delay, I had a medical issue and was working and testing another version:
I published another version, now MozMem_v15, at same location MozMem.zip or MozMem_15.zip ... (The original version is there also as MozMem_12.zip)
Now there is beside normal version another
This MozMem version compared to previous should be little faster, little more efficient (i.e. less problems with free-space fragmentation), and it corrects a very rare problem, that the background thread in v12 could dead-lock repairing a non-probable damage - now it tries at most twice...
It primarily overrides jemalloc, but because it replaces api*crt-heap*.dll, it also hooks on DLLs, that don't use malloc in mozglue.dll, but use directly malloc() in that api heap, so that now they use my heap both... (And there are still dlls, probably system ones, that use msvcrt.dll malloc() or HeapAlloc(), which I don't handle yet...)
The original api-ms-win-crt-heap-l1-1-0.dll has no code inside, it just forwards functions to ucrtbase.dll ... Microsoft makes these small API dlls, so that they can be hotfixed separately without need to apply a one huge fix to base runtime just to fix some small part...
(So it is more thorough and efficient, than what I suggested as an alternative placing it in dependentlibs.list with different name...)
I don't analyze or touch, what you allocate, just where are the blocks arranged in memory... (and once allocated the heap cannot move it elsewhere, since the heap does not know, where the higher levels may have a pointer to it... when analyzing leaks as a programmer I can search for it, but it takes long, often multiple minutes, to sift through 2Gb of address space, whether there is some pointer...)
The heap can be described as "malloc() me some block of memory as quick as possible and don't cause fragmentation or handle it graciously..." and "free() this block, I don't care about it any more", but the heap must care about the free blocks to reuse them on next malloc, but time spent to free the block is also counted in the "cost" of malloc, i.e. as quick as possible...
But there are stability issues...
At this moment I found one page that crashes Palemoon with this memory manager, due to rambler.ru video embed, that aggressively preloads 9Mb video in 70 XHR requests shortly after page load even without media.autoplay enabled, and Palemoon crashes in lgpllibs.dll somewhere in video decoder in background thread, most probably because it references some memory that the foreground thread already released?? (i.e. not properly add-referencing the memory block and hoping the lazy memory manager did not release the large pages too soon?? Not sure...)
About using assembler - there already are few parts in Palemoon written in assembler, for example AES encryption, or code from libavcodec, libjpeg\simd, etc... But there is a C-language alternative, although slower...
About security - on higher application levels, you don't care where malloc() gets the memory block and how are they arranged, and the security should not depend on that... But there is issue of stability, that the previous memory-manager was lazier releasing free() memory, and could tolerate accessing released blocks from higher-levels... But that could cause Palemoon crash, not a security vulnerability...
But these issues are usually solved by thorough testing... (as I think this is an issue of higher level not passing parameters to lgpllibs.dll correctly, just with jemalloc it could be better tolerated...?)
As a security-related issue - now the memory blocks are arranged in well-documented format easier to access by third-party tools on user's computer (like the supplied dump scripts), but analyzing jemalloc or heap blocks was also possible, it is just a matter of little more complexity, and there shouldn't be a program on user computer accessing other program memory, it would be a security issue always, even without MozMem having simple well-documented memory format... (in case you doubt where is it "documented"? In the dump scripts... Just the small pages have so simple format they are "documented" at start of MozMem.asm...)
The TaskView (that should be called "Task Scalpel Pro", but I didn't want to boast) is my tool, the disassembler I use is my tool...
The addon scripts for dumping memory statistics for this memory manager are written in my own programming language
(There was a proverb, that every non-trivial program has at least 1 bug and at least 1 line of code in vain, and applying recursion, one can get to a single-line program that has an error... Which I do quite often and the "error" there is at least that it does not properly check for error conditions like missing arguments...)
But now I rather work as a web-admin for multiple news web-sites, and I use Palemoon for news reading and commenting... (I don't have my own web-browser yet...)
I wanted to beat FastMM4 for Delphi in speed and did not, they are 20% faster, while my ones are more stable and efficient... About twice as fast as XP heap or old Delphi memory manager...
But FastMM4 was incorporated and built-in into newer Delphi, so I finished second in the race, and there is no demand for my attempts, beside that commercial Medical application and my tools...
(and as expected, most parts of FastMM4 are also written in assembler... you don't get such speed in higher-level language...)
As a developper I didn't want too fast or huge computer and then not noticing program inefficiency the ordinary users may experience...
If you wanted different format of the dump, the dumping script is open-source and can be simply modified... I don't know which MAP files the MSVC produces to translate function address to a name?
Now it dumps lines like this:
$0814F738 272 (0x110) $64F3C03C = mozjs.dll + 0x31C03C
address, size (size hex), malloc caller = DLL + offset
(Or if there were the MAP files available for some released version of Palemoon 32-bit, i.e. some form of listing of function_name=address, I could analyse that in more detail, than dll+offset... While I don't have disk space to install MSVC, and I don't install any Microsoft software beside original windows...)
Notes about auditing/understanding my assembler code:
πα½
I published another version, now MozMem_v15, at same location MozMem.zip or MozMem_15.zip ... (The original version is there also as MozMem_12.zip)
Now there is beside normal version another
Not quite so...athenian200 wrote: ↑2024-02-22, 04:51So, wait... are you saying that your version of the dll effectively overrides jemalloc and uses the system malloc for Windows? ...
It primarily overrides jemalloc, but because it replaces api*crt-heap*.dll, it also hooks on DLLs, that don't use malloc in mozglue.dll, but use directly malloc() in that api heap, so that now they use my heap both... (And there are still dlls, probably system ones, that use msvcrt.dll malloc() or HeapAlloc(), which I don't handle yet...)
The original api-ms-win-crt-heap-l1-1-0.dll has no code inside, it just forwards functions to ucrtbase.dll ... Microsoft makes these small API dlls, so that they can be hotfixed separately without need to apply a one huge fix to base runtime just to fix some small part...
(So it is more thorough and efficient, than what I suggested as an alternative placing it in dependentlibs.list with different name...)
When there is some piece of memory not freed, it holds whole 4kb page around. The original jemalloc did not reuse that page... With my implementation, in that case it holds the page prepared for allocating around the leak, in case of small and middle blocks... Just when there are too many leaks, especially in 8kb or larger blocks, it clogs anyway... But that cannot be solved at heap level...athenian200 wrote: ↑2024-02-22, 04:51So the leaks are a separate problem, what you've solved here is an issue with the heap allocation itself
I don't analyze or touch, what you allocate, just where are the blocks arranged in memory... (and once allocated the heap cannot move it elsewhere, since the heap does not know, where the higher levels may have a pointer to it... when analyzing leaks as a programmer I can search for it, but it takes long, often multiple minutes, to sift through 2Gb of address space, whether there is some pointer...)
The heap can be described as "malloc() me some block of memory as quick as possible and don't cause fragmentation or handle it graciously..." and "free() this block, I don't care about it any more", but the heap must care about the free blocks to reuse them on next malloc, but time spent to free the block is also counted in the "cost" of malloc, i.e. as quick as possible...
athenian200 wrote: ↑2024-02-22, 04:51Anyway, I would recommend for anyone interested in testing this out to try it in a VM or something, don't replace this dll in your main Pale Moon installation because we can't really audit this or support it.
I doubt there are any security issues beside the mess that is already there...
But there are stability issues...
At this moment I found one page that crashes Palemoon with this memory manager, due to rambler.ru video embed, that aggressively preloads 9Mb video in 70 XHR requests shortly after page load even without media.autoplay enabled, and Palemoon crashes in lgpllibs.dll somewhere in video decoder in background thread, most probably because it references some memory that the foreground thread already released?? (i.e. not properly add-referencing the memory block and hoping the lazy memory manager did not release the large pages too soon?? Not sure...)
About using assembler - there already are few parts in Palemoon written in assembler, for example AES encryption, or code from libavcodec, libjpeg\simd, etc... But there is a C-language alternative, although slower...
About security - on higher application levels, you don't care where malloc() gets the memory block and how are they arranged, and the security should not depend on that... But there is issue of stability, that the previous memory-manager was lazier releasing free() memory, and could tolerate accessing released blocks from higher-levels... But that could cause Palemoon crash, not a security vulnerability...
But these issues are usually solved by thorough testing... (as I think this is an issue of higher level not passing parameters to lgpllibs.dll correctly, just with jemalloc it could be better tolerated...?)
As a security-related issue - now the memory blocks are arranged in well-documented format easier to access by third-party tools on user's computer (like the supplied dump scripts), but analyzing jemalloc or heap blocks was also possible, it is just a matter of little more complexity, and there shouldn't be a program on user computer accessing other program memory, it would be a security issue always, even without MozMem having simple well-documented memory format... (in case you doubt where is it "documented"? In the dump scripts... Just the small pages have so simple format they are "documented" at start of MozMem.asm...)
I'm 51 years old and have over 30 years programming practice, of which is 20 years a large Medical application, and over 100 windows programs and tools...athenian200 wrote: ↑2024-02-22, 04:51That said, I will be somewhat impressed from a programming/engineering perspective if someone managed to partially resolve memory issues in a stable manner by just messing around with an assembler on an old laptop!
The TaskView (that should be called "Task Scalpel Pro", but I didn't want to boast) is my tool, the disassembler I use is my tool...
The addon scripts for dumping memory statistics for this memory manager are written in my own programming language
There was a bad memory manager in Delphi, so I've written my own in cca 2003... I made about 20 different memory manager attempts, of which I use 5 to this date in my programs...athenian200 wrote: ↑2024-02-22, 04:51That doesn't seem like it should be possible, memory management is a pretty complex thing...
I wanted to beat FastMM4 for Delphi in speed and did not, they are 20% faster, while my ones are more stable and efficient... About twice as fast as XP heap or old Delphi memory manager...
But FastMM4 was incorporated and built-in into newer Delphi, so I finished second in the race, and there is no demand for my attempts, beside that commercial Medical application and my tools...
(and as expected, most parts of FastMM4 are also written in assembler... you don't get such speed in higher-level language...)
On one computer I have 64-bit windows 10 with 8Gb memory, and I tried Firefox 45 64-bit... Within one week it swelled to over 3Gb memory usage... I deleted it and use 32-bit version instead, because better crash with 1.5Gb, than use all my ram memory, which I need for other tasks... (now on that computer I use Palemoon and Opera, both 32-bit... Because neither Palemoon or Firefox are not supported by some web-hosting admin panels, which is unfair but I can't do anything about that...)athenian200 wrote: ↑2024-02-22, 04:51I can imagine that being on 32-bit had a lot to do with your motivation to solve this issue... for context, I develop Pale Moon on a system 32GB of RAM, and tend not to encounter problems nearly as quickly...
As a developper I didn't want too fast or huge computer and then not noticing program inefficiency the ordinary users may experience...
You programmers could try it in some sandbox, a VM or something, the AllocTag version, to debug the memory-leaks, i.e. blocks of memory you allocate and don't free later...
If you wanted different format of the dump, the dumping script is open-source and can be simply modified... I don't know which MAP files the MSVC produces to translate function address to a name?
Now it dumps lines like this:
$0814F738 272 (0x110) $64F3C03C = mozjs.dll + 0x31C03C
address, size (size hex), malloc caller = DLL + offset
(Or if there were the MAP files available for some released version of Palemoon 32-bit, i.e. some form of listing of function_name=address, I could analyse that in more detail, than dll+offset... While I don't have disk space to install MSVC, and I don't install any Microsoft software beside original windows...)
Notes about auditing/understanding my assembler code:
-
- Pale Moon guru
- Posts: 35651
- Joined: 2011-08-28, 17:27
- Location: Motala, SE
Re: Pale Moon User Interface Freezes
Honestly if this is a better replacement for the system memory allocator handling, then you may actually find it working better (more stable) by disabling jemalloc in mozconfig, and therefore not mixing 2 allocators, making all malloc calls use the system memory allocator. That does require a rebuild, though.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
-
- Hobby Astronomer
- Posts: 27
- Joined: 2022-05-16, 08:00
- Location: Chernihiv, Ukraine
Re: Pale Moon User Interface Freezes
I made some measurements and I want to share the results. I build a PM with the option "--disable-jemalloc" and, according to my sensations, it uses less memory and works faster.
I also found a site that leads to memory leaks. Here is my expression. This is a completely new clean profile. Open page https://www.olx.ua/uk/elektronika/noutbuki-i-aksesuary/chernigov/q-%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%82%D0%B8%D0%B2%D0%BD%D0%B0%D1%8F-%D0%BF%D0%B0%D0%BC%D1%8F%D1%82%D1%8C-%D0%B4%D0%B4%D1%803/, open several links in new tabs, wait for the load and then close the tabs. As can be seen in the memory report, some windows remained and that occupy a lot of memory. After that, PM begins to freeze.
I also found a site that leads to memory leaks. Here is my expression. This is a completely new clean profile. Open page https://www.olx.ua/uk/elektronika/noutbuki-i-aksesuary/chernigov/q-%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%82%D0%B8%D0%B2%D0%BD%D0%B0%D1%8F-%D0%BF%D0%B0%D0%BC%D1%8F%D1%82%D1%8C-%D0%B4%D0%B4%D1%803/, open several links in new tabs, wait for the load and then close the tabs. As can be seen in the memory report, some windows remained and that occupy a lot of memory. After that, PM begins to freeze.
You do not have the required permissions to view the files attached to this post.
-
- Contributing developer
- Posts: 1537
- Joined: 2018-10-28, 19:56
- Location: Georgia
Re: Pale Moon User Interface Freezes
That actually doesn't surprise me. I remember hearing rumors that the original reason Mozilla went with jemalloc is because glibc's malloc on Linux is awful, and someone found that jemalloc was better... and at some point it was pulled into the tree to be used elsewhere. They decided they wanted memory handling to be consistent across operating systems as they optimized everything around jemalloc's quirks, essentially giving up on supporting Firefox with the system allocator on any OS. Even though it arguably wasn't really necessary on any OS but Linux.__Sandra__ wrote: ↑2024-03-07, 08:32I made some measurements and I want to share the results. I build a PM with the option "--disable-jemalloc" and, according to my sensations, it uses less memory and works faster.
So your test, was that just from disabling jemalloc, or was it from disabling jemalloc and using PA Semi's dll? I feel like we're kinda stuck with jemalloc potentially now due to the JS engine being written around its quirks, but it is interesting to see that at least on Windows, there is a potential performance improvement that comes from not using it, even if it might not be as stable due to everything assuming we're on jemalloc.
"The Athenians, however, represent the unity of these opposites; in them, mind or spirit has emerged from the Theban subjectivity without losing itself in the Spartan objectivity of ethical life. With the Athenians, the rights of the State and of the individual found as perfect a union as was possible at all at the level of the Greek spirit." -- Hegel's philosophy of Mind
-
- Hobby Astronomer
- Posts: 27
- Joined: 2022-05-16, 08:00
- Location: Chernihiv, Ukraine
Re: Pale Moon User Interface Freezes
I just turned off jemalloc in .mozconfig (ac_add_options --disable-jemalloc). I don't want to use PA Semi's dll. He seems to me a suspicious guy. He writes incomprehensible programs on assembler instead of fixing the source. And on that site where his are posted the programs, he distributes Russian propaganda and fakes about the Russian war in Ukraine.athenian200 wrote: ↑2024-03-07, 09:08So your test, was that just from disabling jemalloc, or was it from disabling jemalloc and using PA Semi's dll?