A question regarding the internal functioning of the Pale Moon browser

[tellu-white]

"To make Google Search play nice, it's not enough to remove JavaScript from the page with search results."

https://stackoverflow.com/questions/29970467/removed-javascript-is-still-executable

https://stackoverflow.com/questions/34997399/unload-a-javascript-from-memory

That's why I created the add-on "Google Search" with Brakes :

https://forum.palemoon.org/viewtopic.php?f=71&t=32743

***

After the tests that I made before creating this add-on, I ended up with a question regarding the internal functioning of the Pale Moon browser.

The question is: How is it possible that an HREF on a page with Google Search results can change after right-clicking on a sponsored link (?), even though I had previously performed the following actions:

1. After loading the page with Google Search results, I disconnected from the Internet.

2. I removed all "script", "noscript", "style", "iframe", "link" (as == "script") tags from the page, as well as those with the "jsaction" attribute.

Note: I removed some tags "just in case", after removing the "script" tags proved to be insufficient.

3. I disabled "javascript" in "about:config" ("javascript.enabled" = false).

4. I disabled "dom.moduleScripts" in "about:config" ("dom.moduleScripts.enabled" = false).

5. I ran "Free memory" in the "about:memory" page.

I thought that after all these measures, page with Google Search results should be inert, i.e. one that can no longer respond to user actions (HREF change after right-click on sponsored link).

I would be grateful if someone knowledgeable about the internal workings of Pale Moon could help me resolve this puzzle: what exactly keeps a page active in the scenario described above ?!

***

Note: With the help of the add-on "Google Search" with Brakes, the (identical) copy of the page with Google Search results becomes truly inert (the HREF no longer changes after right-clicking on the sponsored link).

***

Screenshots:

1. Test performed under normal conditions:

01.png

02.png

2. Test performed in the scenario with the 5 actions described above (and after reloading the page):

03.png

04.png

05.png

06.png

07.png

08.png

09.png

[tellu-white]

It's been a while since I made my previous post, and it seems that no one is willing to answer the question I asked there. I don't know if this lack of response has anything to do with the fact that my example was related to Google Search. However, the problem I raised also occurs at DeepL Translate, when the page becomes blank after a random interval of time (without any interaction with the page).

https://forum.palemoon.org/viewtopic.php?f=70&t=32874#p267168

In this case, too, this behavior (the page going blank) cannot be stopped by performing the five actions described in my previous post. In fact, in this case, the first step (disconnecting from the Internet) should be omitted because this action instantly clears the DeepL Translate page.

After performing the four actions, the page becomes blank when you try to enter new text to be translated. I didn't expect the translation to be possible, but I also didn't think it was possible for the page to still be active (to become empty by itself) rather than being idle.

I think DeepL Translate is of interest to many users, so perhaps an answer to my question in the previous post would be useful as a starting point for further investigation.

[RealityRipple]

Sorry, are you expecting a page that's already been loaded to... unload parts of itself? None of the things your doing are having any effect on active script space, nor would anyone with any computer science experience expect them to. Let's run through them one by one:

1) Disconnecting from the internet - all the code's already been loaded, this only stops xhr and fetch of content, not code (unless the code is designed to load parts of itself only when necessary, see #4).
2) Removing tags - the tags are out of DOM. good job. this changes nothing except the DOM structure.
3) Disabling javascript - any sites that try to start execution of javascript will not. too bad you already loaded this site and started that execution before step 1.
4) Disabling moduleScripts - so those parts I mentioned as a possibility in step 1? You just stopped the zero of them that could be loaded from the internet from being executed, effectively doing nothing. Any that were already loaded and executed are still active in memory, bringing us to...
5) Freeing memory - anything that wasn't in use is now free. too bad you've still got the site in question open and everything you're talking about is still very much active, so this again does nothing relevant.

If you want to manually stop a script's execution after a page has already loaded, you're going about it all wrong. It's simple: open the Developer Tools, go to the Debugger tab and hit the pause button. Any time a script tries to execute anything, hit the Step Out button until the script's stopped again.

Edit: Oh, and look up how to cancel all intervals and timeouts in JS; it can be done as a little one-liner - generates a new timer, gets the id, iterates from 0 to that id and clears all of them.

[tellu-white]

None of the things your doing are having any effect on active script space, nor would anyone with any computer science experience expect them to.
...
Any that were already loaded and executed are still active in memory

I already knew the explanations you gave for the first four points:

10.png

https://forum.palemoon.org/viewtopic.php?f=71&t=32743

However, if I didn't stop the connection between the browser and the server, the page could continue to load and execute additional scripts. That is why I performed the steps described in the first post: to prevent any new actions (new "executions") that could be related to the problem being analyzed.

The question I did not receive an answer to is the following: shouldn't the setting "javascript.enabled" = false in "about:config" stop any JavaScript execution on the page? If execution can be stopped from "Developer Tools / Debugger," how does Pale Moon proceed internally in this case? How can I replicate this behavior in an add-on?

[RealityRipple]

I don't understand why you think changing a preference would make an active change. A preference is checked before an action is undertaken, not afterward. If you set javascript.enabled to false, when do you expect it to check that preference? Before it runs every line? Every function? If you want to slow a JS interpreter down, that'd be a great way to do it. Preferences for how to behave for pages are read before the page is loaded, not during, not after.

Regarding how the debugger works: it never stops either, that's a halt. It has to be continued or the whole page is frozen.

[tellu-white]

OK, I understand that the settings in "prefs.js" cannot stop a further execution of scripts that have already been executed.

In your first post, you said (I quote): "Any that were already loaded and executed are still active in memory." However, the script that clears the page has not yet been executed, otherwise the page would already be empty (and it is not). How can this script (which clears the page) still be "active in memory" if it has not been "already executed"?

Furthermore, is there no way to block a further execution of a script that is "already loaded and executed" (with another script - from an add-on)?
Does Pale Moon control scripts, or is it controlled by them?

[Kris_88]

In fact, you can effectively disable and enable script execution.
Try Scratchpad in Environment -> Browser mode:

Code: Select all

gBrowser.selectedBrowser.docShell.allowJavascript = ! gBrowser.selectedBrowser.docShell.allowJavascript;

[tellu-white]

In fact, you can effectively disable and enable script execution.
Try Scratchpad in Environment -> Browser mode:

Code: Select all

gBrowser.selectedBrowser.docShell.allowJavascript = ! gBrowser.selectedBrowser.docShell.allowJavascript;

Thank you Kris_88. I'll do some tests, tomorrow.

[RealityRipple]

OK, I understand that the settings in "prefs.js" cannot stop a further execution of scripts that have already been executed.

In your first post, you said (I quote): "Any that were already loaded and executed are still active in memory." However, the script that clears the page has not yet been executed, otherwise the page would already be empty (and it is not). How can this script (which clears the page) still be "active in memory" if it has not been "already executed"?

Furthermore, is there no way to block a further execution of a script that is "already loaded and executed" (with another script - from an add-on)?
Does Pale Moon control scripts, or is it controlled by them?

The page event being attached to the unload event is the execution of importance in this instance. Or whatever triggers that attachment. Ad infinitum back to the initial execution point. When script files are loaded,they're executed. Functions and classes appear to break this paradigm on the surface, but only because they haven't been called or initialized yet, not because the code they reside in isn't executed already.

[tellu-white]

When script files are loaded,they're executed. Functions and classes appear to break this paradigm on the surface, but only because they haven't been called or initialized yet, not because the code they reside in isn't executed already.

It seems that the notion of "execution" you describe eludes logical thinking.

Execution in computer and software engineering is the process by which a computer or virtual machine interprets and acts on the instructions of a computer program. Each instruction of a program is a description of a particular action which must be carried out, in order for a specific problem to be solved. Execution involves repeatedly following a "fetch–decode–execute" cycle for each instruction done by the control unit. As the executing machine follows the instructions, specific effects are produced in accordance with the semantics of those instructions.

https://en.wikipedia.org/wiki/Execution_(computing)

I had the same idea: if and only if the code of a function is executed, the effects for which that code was designed occur. And the reverse must also be true: if the expected effects do not occur, then the code has not yet been executed.

Execute stage
This is the only stage of the instruction cycle that is useful from the perspective of the end-user. Everything else is overhead required to make the execute step happen.

https://en.wikipedia.org/wiki/Instruction_cycle?useskin=vector#Execute_stage

From this, I logically deduce that if a function has not been "called or initialized yet" (RealityRipple), then it could not be executed because it is not yet "useful from an end user's perspective".

***

I based myself on this logic when I performed the steps described in my first post. I wanted to prevent any execution of code that aims to change the HREF of a link that is right-clicked. Since the HREF had not yet been changed when those steps were performed, I logically deduced that the code (function) that is activated after right-clicking on a link had not yet been executed, so it could not already be "active in memory".

You wrote:

Any that were already loaded and executed are still active in memory.

https://forum.palemoon.org/viewtopic.php?f=3&t=32949#p267641

Script code is executed, creates things in the JavaScript environment, and is completely disconnected from the script element that loaded it.

https://stackoverflow.com/questions/29970467/removed-javascript-is-still-executable

Once the JavaScript is loaded and executed, there is no link between it and the script element that loaded it at all.

https://stackoverflow.com/questions/34997399/unload-a-javascript-from-memory

[RealityRipple]

JavaScript is a little funny about this - a lot of people assume that it interprets/executes code line-by-line because it parses and spits out errors that way, but the execution process is a little messier. After a couple conversions and efficiency optimizations, the resulting bytecode is fed into a JavaScript Virtual Machine. This is the important bit - when the VM gets the code in question, this is what I call "execution" because that's what the step is referred to as when you look it up. Within the virtual machine, the non-function, non-eval-string code is put into a "global execution context" and that context is put onto the JVM's "stack" for evaluation. Before a function is called, its bytecode is loaded into a "function execution context" and put onto the same stack, then removed when it's done. However, the interpreter is not sending new things to the VM; that's all already been done and "executed". All the "execution" inside the virtual machine tends to, instead, be referred to under the term "evaluated" rather than "executed", at least in my experience, referring to the fact that from the script's side, it's all already been fed into its version of a compiler.

[tellu-white]

JavaScript is a little funny about this - a lot of people assume that it interprets/executes code line-by-line because it parses and spits out errors that way, but the execution process is a little messier. After a couple conversions and efficiency optimizations, the resulting bytecode is fed into a JavaScript Virtual Machine. This is the important bit - when the VM gets the code in question, this is what I call "execution" because that's what the step is referred to as when you look it up. Within the virtual machine, the non-function, non-eval-string code is put into a "global execution context" and that context is put onto the JVM's "stack" for evaluation. Before a function is called, its bytecode is loaded into a "function execution context" and put onto the same stack, then removed when it's done. However, the interpreter is not sending new things to the VM; that's all already been done and "executed". All the "execution" inside the virtual machine tends to, instead, be referred to under the term "evaluated" rather than "executed", at least in my experience, referring to the fact that from the script's side, it's all already been fed into its version of a compiler.

Thank you RealityRipple for this in-depth explanation.

[tellu-white]

In fact, you can effectively disable and enable script execution.
Try Scratchpad in Environment -> Browser mode:

Code: Select all

gBrowser.selectedBrowser.docShell.allowJavascript = ! gBrowser.selectedBrowser.docShell.allowJavascript;

It works! This code executed in "Scratchpad / Environment / Browser" completely blocks JavaScript, even the code "already active in memory."

Screenshots:

01.png

02.png

03.png

[RealityRipple]

JavaScript is a little funny about this - a lot of people assume that it interprets/executes code line-by-line because it parses and spits out errors that way, but the execution process is a little messier. After a couple conversions and efficiency optimizations, the resulting bytecode is fed into a JavaScript Virtual Machine. This is the important bit - when the VM gets the code in question, this is what I call "execution" because that's what the step is referred to as when you look it up. Within the virtual machine, the non-function, non-eval-string code is put into a "global execution context" and that context is put onto the JVM's "stack" for evaluation. Before a function is called, its bytecode is loaded into a "function execution context" and put onto the same stack, then removed when it's done. However, the interpreter is not sending new things to the VM; that's all already been done and "executed". All the "execution" inside the virtual machine tends to, instead, be referred to under the term "evaluated" rather than "executed", at least in my experience, referring to the fact that from the script's side, it's all already been fed into its version of a compiler.

Thank you RealityRipple for this in-depth explanation.

This also highlights an issue with task-related lexicon: according to another user, Chrome's efficiency utility, Lighthouse, uses the term evaluation time to refer to the parsing, interpreting, executing, and initial global execution stack "evaluation", all together, without breaking it down much at all; which is, of course, fine for determining what they're trying to determine in that case, and uses established lexicon to describe the entire process with an understandable term to choose for the complete load and run of a script. However, within documentation, you'll run into terms like line evaluation and the eval() function, which don't quite match this definition. And I can't think of a better term, within the lexicon of JavaScript, to use there. In the larger sense, I'd call that "initial process time" rather than evaluation time, but that would start other debates about the terms initial and process within the lexicon of JavaScript..

Either way, glad Kris found you a much more specific and reliable method of interruption.

[Kris_88]

It works!

Excellent. I'm glad it worked for you.
The only thing is, I haven't tested how it works for child iframes. The iframe has its own docshell, and I don't know if the "allowJavascript" flag is inherited from the parent.

[tellu-white]

Excellent. I'm glad it worked for you.

Unfortunately, after reactivating JavaScript, only the code on the Google Search results page worked, and so the HREF was changed again after right-clicking on the link.

However, DeepL Translate no longer responds to a new attempt to translate a new text. I have to find another workaround to prevent the DeepL Translate page to become blank after a while (when I browse other pages).

[Kris_88]

I have to find another workaround to prevent the DeepL Translate page to become blank after a while (when I browse other pages).

It's likely that the solution you've chosen (stopping scripts) simply isn't suitable for this situation. As far as I understand, the page is being cleared because the page's internal scripts detect some problem and consider "nothing" to be better than "wrong." It's best to investigate the exact cause, which is most likely a partial browser incompatibility that needs to be fixed.

[tellu-white]

I have to find another workaround to prevent the DeepL Translate page to become blank after a while (when I browse other pages).

It's likely that the solution you've chosen (stopping scripts) simply isn't suitable for this situation.

Indeed, I found the reason why the "DeepL Translator" page became blank after a while: I also removed tags from page that some scripts try to access later.

https://forum.palemoon.org/viewtopic.php?f=70&t=32874&start=20#p267916

Important!
If I don't remove the unnecessary tags from "DeepL Translator" page, Pale Moon ends up using up to 90% of the CPU ( I have an older PC ). After removing these tags, the CPU level drops to 0%.

Screenshots:

1. Before removing the tags that are not necessary for "DeepL Translator" to function, Pale Moon uses up to 90% of the CPU:

01.png

2. After removing the tags from page "DeepL Translator" that cause Pale Moon to use up to 90% of the CPU, the page becomes practically inert, with the CPU usage dropping to 0% ( making fun of the JavaScript code that is already "active in memory" ):

02.png

[tellu-white]

It seems that not only Javascript code can go undercover (be "active in memory" - a euphemism for "lurking in memory") waiting for the right moment to go uncovered. HTML code does not hesitate and goes undercover too (lurking in memory), with a little help from its friend, Javascript

All joking aside, it seems that some website developers use "sessionStorage" to prevent saving a page to hard-disk with all available information, in a single step. If they used tags with the "hidden" attribute to display the searched information sequentially (through a series of clicks), then the task of an advanced user to make the hidden content visible would be quite simple. Using "sessionStorage", part of the searched information is only found in memory at a given moment (and not in the current source code of the page), so the original source code of the page must be retrieved.

I ran into this problem when I searched for "Netflix hidden genre codes" ("secret codes") that make it easier to navigate the Netflix jungle. The page in question is called "Netflix Codes 2025: All The Secret Movie & Series Category Codes on Netflix":

https://www.whats-on-netflix.com/news/the-netflix-id-bible-every-category-on-netflix/

I wanted to save that page on my hard-disk, but the "secret codes" are displayed on the page using a "<table>" tag, with each code located in a "<tr>" tag. The problem is that the "<table>" tag displayed at a given moment on page contains only a small subset of the "<tr>" tags with "secret codes", with the rest of these tags being found only in memory. There are 165 subsets with "secret codes," so to have them all on your hard-disk, you would need to save that page 165 times. For example, the last category in "<table>" (in alphabetical order) is called "Zombies" and can be found in subset number 165. Fortunately, all the "secret codes" can be found in the original source code of the HTML page.

I used two "Custom Buttons" to retrieve the "Current Source Code of Current Page" and "Original Source Code of Current Page" (see codes below).

Screenshots:

Table with the first subset of "secret codes". I searched for the string "Zombies" in "Developer Tools" and the result was negative (0 matches):

01.png

I copied the "Current Source Code of Current Page" (the page with the first subset) to the Clipboard with a "Custom Button" using the following Javascript code:

Code: Select all

/*CODE*/

var prompts = Components.classes["@mozilla.org/embedcomp/prompt-service;1"].getService(Components.interfaces.nsIPromptService);
var gClipboardHelper = Components.classes["@mozilla.org/widget/clipboardhelper;1"].getService(Components.interfaces.nsIClipboardHelper);

var current_Source_Code_of_Current_Page = content.document.querySelector('html').outerHTML;

// Copy to Clipboard Current Source Code of Current Page

gClipboardHelper.copyString(current_Source_Code_of_Current_Page);

var message = ' \nThe Current Source Code of the Current Page has been Copied to Clipboard. \n ';
prompts.alert(null, 'Current Source Code to Clipboard', message);

I searched for the string "Zombies" in "Current Source Code of Current Page" and the result was negative (0 hits):

02.png

I copied the "Original Source Code of Current Page" to the Clipboard with a "Custom Button" using the following Javascript code:

Code: Select all

/*CODE*/

var prompts_for_get_page_source = Components.classes["@mozilla.org/embedcomp/prompt-service;1"].getService(Components.interfaces.nsIPromptService);
var gClipboardHelper_for_get_page_source = Components.classes["@mozilla.org/widget/clipboardhelper;1"].getService(Components.interfaces.nsIClipboardHelper);

get_Original_Source_Code_of_Current_Page_in_Temp_File();

function get_Original_Source_Code_of_Current_Page_in_Temp_File(){
	var requestor = window.content.document.defaultView.QueryInterface(Components.interfaces.nsIInterfaceRequestor);
	var browser = requestor.getInterface(Components.interfaces.nsIWebNavigation).QueryInterface(Components.interfaces.nsIDocShell).chromeEventHandler;
	var URL = browser.currentURI.spec;

	var doc = {
		characterSet: browser.characterSet,
		contentType: browser.documentContentType,
		title: browser.contentTitle
	};

	try{
		var ios = Components.classes["@mozilla.org/network/io-service;1"].getService(Components.interfaces.nsIIOService);
		var charset = doc ? doc.characterSet : null;
		var uri = ios.newURI(URL, charset, null);
		var contentType = doc ? doc.contentType : null;
		
		var file = get_File_for_Original_Source_Code_from_Temp_Folder(uri, doc, contentType);
		
		var mnsIWebBrowserPersist = Components.interfaces.nsIWebBrowserPersist;
		var webBrowserPersist = Components.classes["@mozilla.org/embedding/browser/nsWebBrowserPersist;1"].createInstance(mnsIWebBrowserPersist);
		webBrowserPersist.persistFlags = mnsIWebBrowserPersist.PERSIST_FLAGS_REPLACE_EXISTING_FILES;
		
		webBrowserPersist.progressListener = {
			onStateChange: function(aWebProgress, aRequest, aStateFlags, aStatus) {
				// The next "if" is executed AFTER the Source Code of the Current Page has been Saved to Hard-Disk
					
				if (webBrowserPersist.currentState == webBrowserPersist.PERSIST_STATE_FINISHED) {
					var source_code_path = file.path;
					
					// var message = ' \nThe Original Source Code of the Current Page has been Saved to Hard-Disk with Path : \n\n' + source_code_path + '\n ';					
					// prompts_for_get_page_source.alert(null, 'Original Source Code', message);
					
					copy_Original_Source_Code_of_Current_Page_to_Clipboard(source_code_path);
					
					file = null;
					doc = null;
				}
			}
		}
		
		let referrerPolicy = Components.interfaces.nsIHttpChannel.REFERRER_POLICY_NO_REFERRER;
		webBrowserPersist.savePrivacyAwareURI(uri, null, null, referrerPolicy, null, null, file, false);
	  
	} catch(err){
		alert(err.message);
	}
}
  
function get_File_for_Original_Source_Code_from_Temp_Folder(aURI, aDocument, aContentType){
	var scriptLoader = Components.classes["@mozilla.org/moz/jssubscript-loader;1"].getService(Components.interfaces.mozIJSSubScriptLoader);
	var caUtils = {};
	scriptLoader.loadSubScript("chrome://global/content/contentAreaUtils.js", caUtils);

	var fileLocator = Components.classes["@mozilla.org/file/directory_service;1"].getService(Components.interfaces.nsIProperties);
	var tempFile = fileLocator.get("TmpD", Components.interfaces.nsIFile);
	var fileName = caUtils.getDefaultFileName(null, aURI, aDocument, aContentType);
	var extension = caUtils.getDefaultExtension(fileName, aURI, aContentType);
	var leafName = caUtils.getNormalizedLeafName(fileName, extension);
	tempFile.append(leafName);
	
	return tempFile;
}

function get_Original_Source_Code_from_Temp_File(source_code_path){
	var source_code = "";

	var source_code_file = Components.classes["@mozilla.org/file/local;1"].createInstance(Components.interfaces.nsILocalFile);
	source_code_file.initWithPath(source_code_path);
	
	if(!source_code_file.exists()){
		var message = ' \nThe file :\n\n' + source_code_path + '\n\ndoes not exist. \n ';		
		prompts_for_get_page_source.alert(null, 'ERROR Source Code', message);
		
		return source_code;
		
	} else{
		// open an input stream from file
		var fileInputStream = Components.classes["@mozilla.org/network/file-input-stream;1"].createInstance(Components.interfaces.nsIFileInputStream);
		var converterInputStream = Components.classes["@mozilla.org/intl/converter-input-stream;1"].createInstance(Components.interfaces.nsIConverterInputStream);
		fileInputStream.init(source_code_file, -1, 0, 0);
		converterInputStream.init(fileInputStream, "UTF-8", 0, 0);
		
		{
			let str = {};
			var read = 0;
			
			do {
				read = converterInputStream.readString(0xffffffff, str);
				source_code += str.value;
				
			} while (read != 0);
		}
		
		converterInputStream.close();
		
		return source_code;
	}
}

function copy_Original_Source_Code_of_Current_Page_to_Clipboard(source_code_path){
	var source_code = get_Original_Source_Code_from_Temp_File(source_code_path);
	
	// Delete File with Original Source Code
	
	try{
		var file = Components.classes["@mozilla.org/file/local;1"].createInstance(Components.interfaces.nsIFile);
		file.initWithPath(source_code_path);
		
		if(file.exists()){
			file.remove(true);
			
			// var message = ' \nThe File with Original Source Code :\n\n' + source_code_path + '\n\nhas been Deleted. \n ';		
			// prompts_for_get_page_source.alert(null, 'Delete File with Original Source Code', message);
		}
		
	} catch(err){
		alert(err.message);
	}

	if(source_code != ""){
		// Copy "source_code" to Clipboard

		gClipboardHelper_for_get_page_source.copyString(source_code);
		
		var message = ' \nThe Original Source Code of the Current Page has been Copied to Clipboard. \n ';
		prompts_for_get_page_source.alert(null, 'Original Source Code to Clipboard', message);
	}
}

I searched for the string "Zombies" in "Original Source Code of Current Page" and the result was positive (1 hit):

03.png

I searched for the string "Zombies" in all files (in all folders) created by Pale Moon and the result was negative (0 hits), so I used "Process Hacker Portable" to search for the string "Zombies" in memory. Here is the result:

04.png

05.png

06.png

As you can see, the HTML code that displays the "Zombies" category on page is found in memory:

Code: Select all

<a href="https://www.netflix.com/browse/genre/3719" rel="noopener" target="_blank"> Zombies</a>
Zombies 3719

The page where I imported the last "subset" (number 165) into the table, a "subset" that also displays the last category (alphabetically) called "Zombies":

07.png

I used the "Intercept & Modify HTTP Response" add-on with the "Requests Info" option to intercept all source codes of all files used by the Netflix "secret codes" page and found the string "sessionStorage" 46 times. This led me to use the "Intercept & Modify HTTP Response" add-on to remove the string "sessionStorage" from all the intercepted source codes (mentioned above). Here is the used filter:

Code: Select all

[`/whats-on-netflix.com/`, `/.*/`, [`/sessionStorage/g`, ``]]

The result was convincing: the page loaded with all available "secret codes", without creating the 165 "subsets":

08.png

09.png