I now published
MozMem_18.zip, which corrects the bug I wrote earlier, that some sites were crashing...
Now it seems stable...
It is probably the last release, as I see no interest in it, and instead
I identified the actual problem elsewhere...
Moonchild wrote: ↑2024-03-02, 21:45
Honestly if this is a better replacement for the system memory allocator handling, then you may actually find it working better (more stable) by disabling jemalloc in mozconfig, and therefore not mixing 2 allocators, making all malloc calls use the system memory allocator. That does require a rebuild, though.
There are GC chunks, jemalloc heap, then some dlls are using api-*crt-heap*.dll forwarding to ucrtbase.dll heap
(which I intercept also), then there are dlls, including system ones, that use msvcrt.dll with another heap
(that I started to intercept and unify into my single heap also since v16), and then there are about 20 different heaps acquired by HeapCreate() or GetProcessHeap(), which I do not plan to intercept and unify into the common heap...
Then I made a graphic representation of memory usage, see attachment...
Pmx_PM_Mem_240309a2.png
On the attachment, there is 2Gb address space, every pixel is 4kb page, 1Mb row width, 2048 px height... There are four examples...
The colors are: red executable code (dlls), gray reserved blocks, brown is unidentified, green is heap, either jemalloc or MmHeap (slightly different color), purple are system HeapAlloc, and blue are GC chunks... In those, light blue are pages completely zero...
Of the four columns: first is Palemoon soon after restart, the one in which I write this... The second column is using jemalloc... Third is using MmHeap... Notice a lot of light-blue (zero-filled) GC chunks with just first 16 pages (arenas) non-zero... Fourth is Palemoon with that problem fixed, now it endured 4x more browsing before thus completely filling space...
Using that I identified there are a lot of GC chunks, which have only cca 50kb of data, all else 00 null bytes, completely empty, with RelocationOverlay 0xbad0bad1 cells pointing "nowhere"
(in released memory, in heap blocks in middle of strings, etc...), with JSRuntime at the end pointing to deallocated memory or released memory... I call them "
abandoned" GC chunks, they are really signicifant leaks, and it's safe to release them...
Originally it seemed jemalloc is causing the "swelling", but instead it was releasing earlier pages, which were occupied by abandoned GC chunks, and the jemalloc was "swelling" on newer and newer addresses because of that...
----------------------------------------------------------------------
In
MozMem_18.zip there is now
MozGcGc.dll - "garbage collector of garbage collector", that if it is loaded into Palemoon, it periodically releases the leaked (abandoned) GC chunks...
I checked it also with Palemoon 33 now on download and there is still this same problem, and it works also there...
MozGcGc.dll can be used independently of my MozMem allocator, it also works with original jemalloc heap, it is
only 32-bit... It cycles, waits in 15 second sleep(), then sifts address space using VirtualQuery, identifies "abandoned" GC chunks, which are completely deallocated, their JSRuntime does not point to a valid JSRuntime (it was already deallocated). If it finds some, it marks it, waits another 15 seconds, and if it is still there and still empty, it is released...
Example empty abandoned chunk trailer, with the freé mark by MozGcGc.dll in place of padding...
(If you use my TaskView to view process memory, if it is black&white, press Ctrl+H or use Options/Highlight to switch on coloring, it looks much better...)
Clpx267902_crop.png
On the following example, there is end of one "abandoned" Chunk, its JSRuntime points to all zero bytes, and there is visible start of another abandoned (relocated and leaked) Chunk:
Clpx267474_crop.png
The MozGcgc.dll checks all bytes from 0xfffa0 thus, beside the free-mark, and the JSRuntime is checked that it has pointer to itself at offset +0x10... If it doesn't identify any valid JSRuntime, then the detection is incompatible and it doesn't release anything... Only if it identifies some valid, then the invalid ones with completely deallocated bitmap are thus released...
That Palemoon is stable afterwards is a proof they were abandoned leaks and program makes no reference to them ever...
(When I catch it with DumpBlocks.script, such Chunk is on a place where was not a Chunk 10 minutes earlier, and also its JSRuntime is already released, it has to be something temporary used during webpage load, on which relocateArenas runs and soon the JSRuntime is destroyed...)
Over last 24 hours, it
released 187 Mb memory, and it consumed 4 and half minutes CPU time total, making it second most "expensive" background thread...
(4 minutes per day is not a problem...)
On another host I tested it, there were over 360 Mb of abandoned GC chunks...
(It depends, how much web-pages you open, each leaks this 1 Mb GC chunk, beside some minor leaks...)
If you want to just see the blocks, there is also Tools\DropAbandon.script (for Eval.exe), that is only for Palemoon 32, 32-bit, not compatible with Palemoon 33, that can list the blocks, or also drop them...
(Or with the released MozGcGc.dll, if you edit bytes at file offset 0x645 reading 89 F8 E8 7D 01, and replace the first one 89 with C3, then it will only mark the blocks with "freé" and list them in its data segment, without actually releasing them, so that they can be inspected using some memory editor... (In MozGcGc.dll in runtime memory at offset 0x8080 is list of the released chunks...))
It can be "installed" by copying it to Bin\Palemoon folder and listing MozGcGc.dll in dependentlibs.list before the last line... Last line must stay xul.dll
Or you can
test it without installing: Have Palemoon running, and where you unpacked MozMem_18.zip in folder MozGcGc, there is rldmod.exe tool, which can be used thus from command line:
rldmod.exe Palemoon.exe C:\Path\To\MozGcGc\MozGcGc.dll
(if you open cmd.exe command line, write: rldmod.exe Pallemoon.exe , and after space drag-drop the dll onto command-line, so that it is filled with full path...)
My rldmod.exe tool can load DLL into another running process. The process is found by name, the DLL must be "visible" from the process (in its folder), or it can be specified with full path...
(If you have multiple Palemoon processes running, instead of Palemoon.exe specify integer PID (process id) to load only into some instance... It allocates memory using VirtualAllocEx, writes there DLL name, and uses CreateRemoteThread on LoadLibraryA function to load it, waits for thread finished, and frees the memory for dll name by VirtualFreeEx...)
It could write a log-file of released pages, but then I couldn't claim it is pristine clear not writing to any files... So it instead logs the released pages in cyclic memory buffer...
(Again it is published with source and with byte listing, so it can be "verified" it does not contain any other "harmful" code... It is just 3 KiB file, of which 1 KiB is dll header, 2 KiB code...)
In case it encounters Exception
(there is a try-catch), it sleeps a while and restarts the thread... It can be manually stopped and unloaded by placing byte 1 at offset 0x800C from where it is loaded in memory... If it is loaded on startup, the stop-flag is on 0EFE800C and statistics on 0EFE8010, but if loaded later in running Palemoon, it will be on different address...
(it takes 15 or 30 seconds before that stop-flag byte is noticed, and not if it restarts due to Exception... Normal user does not need this, it is for debugging and playing with it without restarting Palemoon...)
To get statistic, what it does
Again - it is just an
Emergency Solution before you correct the Leak, for users not patient to wait for corrected Palemoon 34...
--------------
Then on my Windows 7, there is another significant leak, that system dlls
--------------------------
Please instead of whining you dislike my DLLs, can you do something to
fix that bug, that every web-page load forgets to release and abandons 1 or more GC chunks, 1 Mb memory lost on each page load...?!
(not sure but possibly each iframe looses it's own GC chunk? I'm not sure...)
They are full of RelocationOverlay, and their JSRuntime was already destroyed and freed... So they passed through jsgc.cpp GCRuntime::relocateArenas, but were not released later... There are some #ifdef differing between DEBUG and normal version... I suspected js\src\vm\HelperThreads.cpp struct parseTask, but when reading the source, it seems it releases its explicit context properly... May it be, that the chunks are queued to release in some bg thread, while their JSRuntime is released sooner??? Not sure...
-------------------------------------------------------------------------------------------------------------------
Off-topic:__Sandra__ wrote: ↑2024-03-07, 09:45
he distributes Russian propaganda and fakes about the Russian war in Ukraine
Maybe you were instead fed with Western Propaganda full of Lies and Hypocrisy, that you do not recognize the Truth any more ?
I really do not want to discuss politics here, and I am professional enough to
distinguish between my personal preferences, journalism, and my professional IT work...
--------------------------
Nb - while reading this forum from my last post and previewing (now 7x) this post edits and saving each in mht, the MozGcGc.dll already released cca 80Mb of abandoned GC chunks... Before finishing it (with 16 previews and saved mhts), it released 130 Mb of abandoned GC chunks and there appeared one stray mapped imageres.dll from save-as file-dialog, on the following map it is the brown-field on 240E0000, and unmapping it manually I see no problems with SaveAs file dialog...
(this map is progress from the first column in the 4-column collage above, during writing this text in forum, with MmHeap, and MozGcGc.dll doing periodic cleanup...)
Mem_I_240309_1914.png
πα½
You do not have the required permissions to view the files attached to this post.