Possible X-Frame-Options bug Topic is solved

Talk about code development, features, specific bugs, enhancements, patches, and similar things.
Forum rules
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific bugs, git, the repositories, etc.

This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Please post issues with specific websites, extensions, etc. in the relevant boards for those topics.

Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
User avatar
Tomaso
Board Warrior
Board Warrior
Posts: 1622
Joined: 2015-07-23, 16:09
Location: Norway

Possible X-Frame-Options bug

Unread post by Tomaso » 2020-01-11, 15:29

Issue:

Unable to shop @ solarisjapan.com.

How to reproduce:

1) Go to https://solarisjapan.com/collections/mi ... t-adapter/

2) Click on "Buy Unit" > "Check Out".

3) Check "I am aware of potential import taxes. Keep the invoice value at 100%", then click on "Check Out".

Result: Nothing happens!

Console error:

Code: Select all

Content Security Policy: Ignoring ‘x-frame-options’ because of ‘frame-ancestors’ directive.
Mozregression result:

Last bad: Firefox v60.0a1 (2018-02-15)
First good: Firefox v60.0a1 (2018-02-16)

Pushlog:
https://hg.mozilla.org/mozilla-central/ ... 9ef0943b35

--

(Pale Moon v28.8.0,x64 / Win8.1Pro)

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35602
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Possible X-Frame-Options bug

Unread post by Moonchild » 2020-01-11, 16:55

Tomaso wrote:
2020-01-11, 15:29
Content Security Policy: Ignoring ‘x-frame-options’ because of ‘frame-ancestors’ directive.
That's not a bug. The site specifies both, and CSP's frame-ancestors takes precedence over the x-frame-options header in that case.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
Tomaso
Board Warrior
Board Warrior
Posts: 1622
Joined: 2015-07-23, 16:09
Location: Norway

Re: Possible X-Frame-Options bug

Unread post by Tomaso » 2020-01-11, 17:01

Any idea why Mozilla suddenly decided to allow it to work in Firefox v60.0a1 (2018-02-16)?

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35602
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Possible X-Frame-Options bug

Unread post by Moonchild » 2020-01-11, 17:19

Why do you think it's even related?
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
Sajadi
Board Warrior
Board Warrior
Posts: 1226
Joined: 2013-04-19, 00:46

Re: Possible X-Frame-Options bug

Unread post by Sajadi » 2020-01-11, 17:28

Web console spits out the following things:

XML Parsing Error: no root element found
Location: https://bt.signifyd.com:11103/ingest
Line Number 1, Column 1:

New Tobin Paradigm

Re: Possible X-Frame-Options bug

Unread post by New Tobin Paradigm » 2020-01-11, 17:36

And the point here?

User avatar
Tomaso
Board Warrior
Board Warrior
Posts: 1622
Joined: 2015-07-23, 16:09
Location: Norway

Re: Possible X-Frame-Options bug

Unread post by Tomaso » 2020-01-11, 17:43

Sajadi wrote:
2020-01-11, 17:28
Web console spits out the following things:

XML Parsing Error: no root element found
Location: https://bt.signifyd.com:11103/ingest
Line Number 1, Column 1:
The pushlog (which I linked to in the first post) is filled with "XMLHttpRequest" fixes, so there might be something there worth looking into.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35602
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Possible X-Frame-Options bug

Unread post by Moonchild » 2020-01-11, 19:49

not fixes. XHR rearchitecturing.

But if you -really- want to help, do some proper bisecting and find out exactly which one of the way-too-long list is actually improving that particular site.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35602
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Possible X-Frame-Options bug

Unread post by Moonchild » 2020-01-11, 19:50

Sajadi wrote:
2020-01-11, 17:28
Web console spits out the following things:

XML Parsing Error: no root element found
Location: https://bt.signifyd.com:11103/ingest
Line Number 1, Column 1:
This is because the server in question spits back html error pages for objects that aren't found (which isn't XML)
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
athenian200
Contributing developer
Contributing developer
Posts: 1535
Joined: 2018-10-28, 19:56
Location: Georgia

Re: Possible X-Frame-Options bug

Unread post by athenian200 » 2020-01-14, 22:16

Tomaso wrote:
2020-01-11, 17:43
The pushlog (which I linked to in the first post) is filled with "XMLHttpRequest" fixes, so there might be something there worth looking into.
That long list is unfortunately all one bug, and that one bug could change the way XML is handled in a way that would negatively impact XUL plugins. That's likely why it wasn't implemented until Firefox 60, because that's when they completely ditched XUL plugins. If you'll notice, it was proposed 7 years ago and implemented much later.

This is likely one of those situations where we really don't want to use their fix because we're still supporting XUL plugins. Basically what they did was gut XPCOM to have everything handled by WebIDL rather than fixing their own existing implementation. If you look through this bug list, we see they're ripping out XPCOM interfaces right and left and adding stuff to Servo.

Highlighting the checkbox in red when you fail to check it works. Subscribing to the newsletter works. But you can't dismiss the subscription dialog box using the X, or click checkout after ticking the box. Also, if you try reloading the page, the checkout button becomes greyed out and the box stays ticked. You have to go back to the item select screen and change your cart to get it to be clickable again.

I saw a message about a mutation observer show up at one point, and I'm noticing that jQuery is getting used here. If this does turn out to involve mutation observers, the relevant change they made was probably to Servo and not to what we use.

Doing a bisection with these builds isn't going to be easy either, because Mozilla only keeps the builds in-between nightlies for about a year, and I can't build them from source because getting my hands on an appropriate version of Rust is next to impossible. It needs version 1.23, and the Rust website will only give me version 1.40. It would be non-trivial to get this Firefox 60 code compiling with newer Rust. I almost had it after applying a couple of patches, but then it failed right near the end in a way that's really difficult to fix without knowing Rust well. I've been struggling with it for two days already and I'm starting to think the answer won't us help much even if I do get it to compile.
"The Athenians, however, represent the unity of these opposites; in them, mind or spirit has emerged from the Theban subjectivity without losing itself in the Spartan objectivity of ethical life. With the Athenians, the rights of the State and of the individual found as perfect a union as was possible at all at the level of the Greek spirit." -- Hegel's philosophy of Mind

User avatar
Tomaso
Board Warrior
Board Warrior
Posts: 1622
Joined: 2015-07-23, 16:09
Location: Norway

Re: Possible X-Frame-Options bug

Unread post by Tomaso » 2020-01-14, 22:45

Thanks for looking into this issue, and for your detailed elaboration, athenian200!

New Tobin Paradigm

Re: Possible X-Frame-Options bug

Unread post by New Tobin Paradigm » 2020-01-15, 01:18

What is a "XUL plugin"?

Also, xml is very strict so the parser reporting no root (xml) element for invalid data shoved into it (such as the output for a server error) is correct. The fact the site scripting does not handle this condition shouldn't be our problem.

Andrew Herbert

Re: Possible X-Frame-Options bug

Unread post by Andrew Herbert » 2020-01-15, 12:42

XHR isn't necessarily related to XML, despite the name.

New Tobin Paradigm

Re: Possible X-Frame-Options bug

Unread post by New Tobin Paradigm » 2020-01-15, 13:04

Absolutely that is what it was designed for albeit it is abused these days for everything from html to json to god knows what.

But it's in the damned name for crying outloud and processing the response by shoving it through the xml parser seems correct to me.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35602
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Possible X-Frame-Options bug

Unread post by Moonchild » 2020-01-15, 14:03

In this case it most definitely seems to want an XML response and the server sends back an HTTP error page instead which (for obvious reasons) doesn't parse as XML.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
athenian200
Contributing developer
Contributing developer
Posts: 1535
Joined: 2018-10-28, 19:56
Location: Georgia

Re: Possible X-Frame-Options bug

Unread post by athenian200 » 2020-01-16, 02:24

Hmm... what I'm thinking is that the XML parsing of the error page could be a symptom of the problem rather than the actual root cause. It's pretty typical in older versions of Firefox for a blank page or any page with a blank Content-Type header to be parsed as XML, and it usually just floods the console with error messages rather than causing things to fail.

The working build of Firefox 60 still displays these XML parsing errors all over the place in the console, so they may very well be irrelevant.

Another thing that's worth noting is that I actually have found a way to get to the checkout using UXP based browsers, but I have to edit the HTML after clicking the button the first time. Here's a video I made using Loom (may need an alternate browser to see it):

https://www.loom.com/share/b73a18f88845 ... 66f6a523e1

Basically, I click on the button once so the "disabled" attribute appears, and then go in and remove that attribute so I can click a second time. On the second press, it seems to work.

UPDATE: I've just managed to confirm that the main issue isn't XHR at all, and doesn't require touching Servo. I was going on a wild goose chase because of the error console. Turns out it's another bug in this list and I just randomly clicked on bugs until I found something that mentioned JavaScript and the "disable" keyword. This is what I found:

https://bugzilla.mozilla.org/show_bug.cgi?id=1405087

I was able to use this as a guide to create this (I'll need to wait for someone to open an issue so I can give it a number):

https://github.com/athenian200/UXP/comm ... 41a94d8139

And the result of compiling that... would look like this:

https://www.loom.com/share/f5f6281cc5ca ... 5e5f9f1cdc

The main problem is, I don't actually know what the pitfalls of removing the disabled and enabled properties from moz-user-input might actually be, or if there's a better way to achieve this outcome.

If we don't want this change after all, then maybe some kind of GreaseMonkey user script can be used to get around this? I feel like I've managed to define the problem well enough that it can probably be dealt with one way or another now.
"The Athenians, however, represent the unity of these opposites; in them, mind or spirit has emerged from the Theban subjectivity without losing itself in the Spartan objectivity of ethical life. With the Athenians, the rights of the State and of the individual found as perfect a union as was possible at all at the level of the Greek spirit." -- Hegel's philosophy of Mind

User avatar
Tomaso
Board Warrior
Board Warrior
Posts: 1622
Joined: 2015-07-23, 16:09
Location: Norway

Re: Possible X-Frame-Options bug

Unread post by Tomaso » 2020-01-16, 06:08

Thanks again, for troubleshooting this athenian200.
..and for backporting promise-based media playback support, which now allows those Loom videos to work in the latest unstable build (amongst other things)! :thumbup:

User avatar
athenian200
Contributing developer
Contributing developer
Posts: 1535
Joined: 2018-10-28, 19:56
Location: Georgia

Re: Possible X-Frame-Options bug

Unread post by athenian200 » 2020-01-16, 07:09

Thanks, Tomaso! I may not be able to create much that's original yet, but I try to make the most of what's available.

Unlike the other fix where it was pretty much a done deal because it was FF53 code, though, this one may very well need a lot of testing because it comes from a much newer FF.

For instance, one thing that slipped into my original patch was a direct use of NS_EVENT_STATE_DISABLED, which technically did exist already, but usually wasn't called directly in that part of the code because it wasn't initialized early enough or something. That changed around Firefox 56. We would technically still use nsGkAtom for that, and I almost missed that at first. My original patch worked in this instance because of dumb luck, and there might be issues with it I haven't thought to test for.

I'm also kind of wondering if we have other consumers of moz-user-input that require us to retain the disabled attribute. Firefox 60 only had one and it was a very minor issue, but that's not guaranteed to be the case with us.

So we may not be out of the woods yet, and I don't want to give anyone false hope here, but there is at least a decent chance we could fix this someday. :)
"The Athenians, however, represent the unity of these opposites; in them, mind or spirit has emerged from the Theban subjectivity without losing itself in the Spartan objectivity of ethical life. With the Athenians, the rights of the State and of the individual found as perfect a union as was possible at all at the level of the Greek spirit." -- Hegel's philosophy of Mind

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35602
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Possible X-Frame-Options bug

Unread post by Moonchild » 2020-01-16, 12:27

Thanks for researching this and finding the problem!
Issue created: Issue #1356 (UXP)

Of note: this regression bug resulting from it should also be looked at: https://bugzilla.mozilla.org/show_bug.cgi?id=1461706
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

Locked