Details about UXP in 2018

Talk about code development, features, specific bugs, enhancements, patches, and similar things.
Forum rules
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific bugs, git, the repositories, etc.

This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Please post issues with specific websites, extensions, etc. in the relevant boards for those topics.

Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35473
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Details about UXP in 2018

Unread post by Moonchild » 2018-02-01, 10:48

As people are aware, we've been developing on UXP for a while now which carries the Basilisk browser as its current "showcase application" and development vessel.

The problem

While platform progress has been steady, investigating the compatibility as a platform with Mozilla-style applications other than a matching Firefox browser application has turned up some very disturbing things, that can be summarized as follows:

Our fork point chosen when starting our Möbius platform development was optimal from a feature set and web technology point of view, but has turned out to be increasingly problematic when trying to use it for anything other than the single product Mozilla has focused on working towards in 2017: Firefox Quantum. Its immediate predecessors are already crippled in several ways and build on an inflexible in-progress code tree that is neither catering to other applications than the one supplied with it, nor will they or what they build on stand up to the test of time.

The severity and reach of these compatibility issues have only come to light very recently as we took steps to try and move Pale Moon code forward to Möbius, and finding some deep-rooted and very well hidden fundamental changes that were applied to the very core of the Mozilla code that could not be undone without completely unbalancing the tree.

The choice

This means a choice was forced upon us, after seeing that this damage (rapid rearchitecturing for Quantum, Rust, Servo and forcing e10s technologies in all components) could not be undone with simple patching or "working back" through the changes made. The choice being that we either had to choose a Mozilla path to continue as we were and focus on a single product with no regard to other applications (à la Waterfox), or to take a step back, re-fork, and focus on many applications being able to use the code tree to build upon.

Since we are steadfast in our resolve to provide a usable platform, the choice fell solidly on the latter of the two.

Doing this has pros and cons, of course. The pros being, as outlined, being able to build a solid foundation that has not yet been tainted with Firefox Quantum precepts and rearchitecturing, with high likelihood of other XUL applications working with minimal changes, GTK2 support for Linux being less broken, clearer choices of optional components being built or not for applications (WebRTC, EME, e10s). The cons being that we will have to re-do a lot of work to bring the new fork forward to our current UXP state-of-development and will need to add new implementations for what will be missing in the new fork that we carried over from our previously forked state (back-porting Mozilla developments like full WASM support and some missing ES6 features).

I think if we make a concerted effort, as a community, we can make this happen, and have the platform ready for beta before the end of summer 2018. We will need to show commitment, focus, and tenacity. We've seen our community can do this, when Tycho was in its early stages, so I'm confident we can make this happen if we work together, put in the effort, and above all keep in mind that we will be developing a platform for wide use, not for one or two applications or workflows.

What does this mean in practice?

In practice, it means we will be re-forking our base code to work from, from the Mozilla tree, at the ESR-52 level and go from that base forward. We're choosing ESR-52 to get as much official Mozilla support for security as possible allowing us to focus more directly on the amount of work needed to re-establish a solid UXP base, as well as that branch being the very last that can be used for our intended purposes (53 and later all have the architectural issues to varying degrees that make them unsuitable). Of course we will still be porting additional security patches that ESR releases normally don't get, to keep an as high security standard as possible regardless of developments.

Since we already know our targets in the tree, and can re-use our patches to a reasonable extent that have been used on Möbius, with research already done, we should be able to use very rapid development here. If all goes as planned, we may have both Pale Moon and our current Basilisk (both will be put in maintenance mode while we focus completely on UXP) on the new platform before the end of the year with a bright future ahead of us for both our products and others alike!
Last edited by Moonchild on 2018-02-01, 11:05, edited 1 time in total.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

JustOff

Re: Details about UXP in 2018

Unread post by JustOff » 2018-02-01, 12:28

Tough times, tough decisions. Thank you for sharing these details.

Are you confident that ESR52 code base is not subject to the same problems that made it almost impossible to further develop UXP as a truly universal platform based on the current Moebius code? Also, since these compatibility issues were discovered when moving Pale Moon code to the new platform, could it be useful shift this step to the earliest possible stage of the UXP development?

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35473
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Details about UXP in 2018

Unread post by Moonchild » 2018-02-01, 12:39

JustOff wrote:Are you confident that ESR52 code base is not subject to the same problems that made it almost impossible to further develop UXP as a truly universal platform based on the current Moebius code?
... do you think this decision would have been made if that wasn't the case?
JustOff wrote:could it be useful shift this step to the earliest possible stage of the UXP development?
No, it couldn't because we need a compatible platform first -- unless you want to do triple or quadruple the work. This step needs to happen when the code is ready for it.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
adesh
Board Warrior
Board Warrior
Posts: 1277
Joined: 2017-06-06, 07:38

Re: Details about UXP in 2018

Unread post by adesh » 2018-02-01, 14:37

So, this was the reason why there has been no progress in the repository lately.
Don't worry, with that resolve, we will definitely get there.
I am not a big help when it comes to C++, but I'll see if I can pick something that fits my level. Maybe I can help you buy an ice cream!

User avatar
Sajadi
Board Warrior
Board Warrior
Posts: 1226
Joined: 2013-04-19, 00:46

Re: Details about UXP in 2018

Unread post by Sajadi » 2018-02-01, 17:55

ESR 52 - sounds good. As long as it helps to push Pale Moon forward when the time has come. Do you think that choice will also help reduce the impact of potential could-be showstoppers like new ecmascript features which are existing right now in the current Pale Moon code? That is the big open and worrying issue in my opinion.

New Tobin Paradigm

Re: Details about UXP in 2018

Unread post by New Tobin Paradigm » 2018-02-01, 21:22

adesh wrote:So, this was the reason why there has been no progress in the repository lately.
Don't worry, with that resolve, we will definitely get there.
I am not a big help when it comes to C++, but I'll see if I can pick something that fits my level. Maybe I can help you buy an ice cream!
I work in my own branches on my own repos testing and researching before it ever hits the main repos. So a lot of what happens never sees the light of day except the positive end result in the cleanest mannor possible. A lot of us work that way to avoid having 4000 commits where only a few hundred actually count for anything like Mozilla does every cycle.

Give us say a month or two to get the codebase back to where we were as far as stuff we have done/changed because that is just reproduction of what has already been researched and done, few more months to get anything desirable we may be missing as far as features. Then we can move forward.

It will be a lot of work no doubt but we HAVE to be getting efficient at it by now.

We do what we must because we can. For the good of all of us, except the ones who are dead.. But there's no sense crying over every mistake. You just keep on trying till you run out of cake. And the science gets done and you make a neat.. Wait, right codebase... Work to be done!
Last edited by New Tobin Paradigm on 2018-02-01, 21:45, edited 6 times in total.

User avatar
ron_1
Moon Magic practitioner
Moon Magic practitioner
Posts: 2852
Joined: 2012-06-28, 01:20

Re: Details about UXP in 2018

Unread post by ron_1 » 2018-02-01, 23:15

Moonchild wrote:
I think if we make a concerted effort, as a community, we can make this happen, and have the platform ready for beta before the end of summer 2018.
I take this to mean no new Basilisk update until then, right?

New Tobin Paradigm

Re: Details about UXP in 2018

Unread post by New Tobin Paradigm » 2018-02-01, 23:19

helloimustbegoing wrote:
Moonchild wrote:
I think if we make a concerted effort, as a community, we can make this happen, and have the platform ready for beta before the end of summer 2018.
I take this to mean no new Basilisk update until then, right?
Sec updates of course.
Last edited by New Tobin Paradigm on 2018-02-01, 23:20, edited 1 time in total.

KingsMan

Re: Details about UXP in 2018

Unread post by KingsMan » 2018-02-02, 07:50

Dear Moonchild,
Won't you think FF52ESR will be more work?
What about comparing FF55.0a code with FF52 code to get some missing changes? Or the changes are too drastic that it can't be traced out easily.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35473
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Details about UXP in 2018

Unread post by Moonchild » 2018-02-02, 08:16

Dear KingsMan,

Please re-read my initial post.

Also, for a laugh, go ahead and do a diff between Firefox 55 and 52-ESR yourself. Grasp the scale of changes you're talking about. And then understand that refactoring means that most of those changes are tightly interconnected.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

KingsMan

Re: Details about UXP in 2018

Unread post by KingsMan » 2018-02-02, 13:33

Dear Moonchild,
thanks for reply. I diffed it and I found changes are drastic and interlinked. It seems Mozilla is devilish in changing code. I don't know that rust is good for what.
Thou troubles brought by ff
Sorry for trouble
Last edited by KingsMan on 2018-02-02, 13:34, edited 1 time in total.

joe04

Re: Details about UXP in 2018

Unread post by joe04 » 2018-02-03, 17:56

Wow, this is quite a development. Here's hoping take 2 goes well!

Regardless of what happens, one of the things I've always appreciated about Pale Moon is the clear communication by its leadership, even when the news isn't so cheery. MC and Tobin filling us in here continues this tradition of excellence. Lots of respect for what you guys do.

Fedor2

Re: Details about UXP in 2018

Unread post by Fedor2 » 2018-02-03, 20:23

What about that mobious name, will you place new code there or create new one?

Also i must add, that mobious build scripts are better optimized, on my old machine it builds four times faster then regular Palemoon. Will you new system has that build scripts?
Last edited by Fedor2 on 2018-02-03, 20:39, edited 1 time in total.

User avatar
SpockFan02
Astronaut
Astronaut
Posts: 535
Joined: 2017-09-24, 16:35
Location: Mak pupulusšum, California

Re: Details about UXP in 2018

Unread post by SpockFan02 » 2018-02-03, 21:50

Out of curiosity, is this related to the build issues that have prevented a lot of recent SeaMonkey versions from being released?

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35473
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Details about UXP in 2018

Unread post by Moonchild » 2018-02-03, 22:07

SpockMan02 wrote:Out of curiosity, is this related to the build issues that have prevented a lot of recent SeaMonkey versions from being released?
I wouldn't rule it out, but since they declined working with us when reaching out to them I'm not involved with their code or their issues, so I can't say.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

gpower2

Re: Details about UXP in 2018

Unread post by gpower2 » 2018-02-04, 08:33

There are no words that can extress my admiration and gratitude for your team and its determination!
After a rebase for Tycho, then a fork for UXP, now you decide to rebase yet again, in order to implement your dream of a fully working XUL platform, you have my respect gentlemen!

Since now you are working on porting previous commits to the new fork, will you post an update when this is finished, in order for people who want to be involved to start creating pull requests to help you in this big "quest"?

All the best to you and keep up the good job! :)

Thehandyman1957

Re: Details about UXP in 2018

Unread post by Thehandyman1957 » 2018-02-05, 03:33

Yes, my hat is off to you guys. We never really see how much work you guys put in to make such a great browser
and it's humbling to know you are now going to have to start again, in a way. I do hope you get all the help you can.

KingsMan

Re: Details about UXP in 2018

Unread post by KingsMan » 2018-02-05, 05:15

Will you people add Ffvpx? 52ESR lacks ffvpx

New Tobin Paradigm

Re: Details about UXP in 2018

Unread post by New Tobin Paradigm » 2018-02-05, 05:28

Maybe.. I dunno.. What is wrong with just libvpx? BTW this is NOT a feature request thread. Take those elsewhere.
Last edited by New Tobin Paradigm on 2018-02-05, 05:28, edited 1 time in total.

User avatar
fatboy
Astronaut
Astronaut
Posts: 556
Joined: 2017-12-19, 08:03
Location: Canada

Re: Details about UXP in 2018

Unread post by fatboy » 2018-02-05, 13:29

Thank You so much for communicating this to the community. I'm no coder and don't have a lot of experience on this forum, but this post is truly refreshing and inspiring.
Thank You yet again. I will be making some donations to thank the team for their efforts. This code rebase/development sounds like quite the task.
Looking forward to the future of Pale Moon and Basilisk :)
Systemd Free - MX Linux, Antix Linux & Artix Linux

Locked