I created a Solaris/illumos port of UXP out of boredom.

Talk about code development, features, specific bugzilla bugs, enhancements, patches, and other highly technical things.

Moderator: satrow

Forum rules
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific referenced Bugzilla bugs, mercurial, etc.

This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Most "bug reports" do not belong in this board and should initially be posted in Community Support or other relevant support boards.

Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
User avatar
athenian200
Fanatic
Fanatic
Posts: 153
Joined: 2018-10-28, 19:56
Contact:

I created a Solaris/illumos port of UXP out of boredom.

Unread post by athenian200 » 2019-09-08, 09:08

I wanted to post this story somewhere in case anyone finds it interesting, it's Pale Moon-related, and I don't really have a blog.

So, about this time last month, I was looking for something to distract myself from a stressful situation in real life and keep my mind occupied. I was looking at the Pale Moon source code and noticed they'd removed Solaris support. So I was thinking to myself, "How hard would it be to add it back in and then make the program actually compile and run?" So I simply installed OpenIndiana (basically the official successor to OpenSolaris after Oracle closed that project down) in a virtual machine and got to work despite having no real experience with Solaris, Firefox, or Pale Moon. The only thing I knew about Solaris going in is that it's the "other" Unix they offered on x86 systems at my college besides Linux so that they could teach about POSIX compliance, avoiding "Linuxisms," and say that they teach Unix and not just Linux. I wasn't able to stick with my degree because of Calculus, but I always wondered what working with it would have been like.

There were five things I learned that were encouraging to me early on.

1. Oracle Solaris and the illumos distributions build Firefox with GCC now, and haven't used Sun Studio to do so in ages, so all the code that makes those assumptions is outdated. In fact, most of OpenIndiana is built with GCC 7 specifically. They do use their own linker, but I knew going in I wouldn't have to deal with any clang weirdness.

2. Most of the GNU toolchain is available, but you have to prefix commands with "g" to get the GNU version instead of the Solaris version.

3. Mozilla regards Solaris as a Tier 2 or 3 platform, and a ton of high-quality patches for it were created during or just after the Firefox 52ESR lifecycle by Mozilla at the request of an incredibly overworked Oracle employee trying to get the biggest Solaris issues fixed upstream.

4. All of the UXP project's major dependencies, like SQLite, NSS, NSPR, libevent, libffi, and other libraries are available and more or less up-to-date on Solaris. NSS and NSPR have been on it since the beginning, with Netscape getting involved with Sun/Java offerings early on to power their server products back in the day.

5. Solaris and Linux are both based on System V in some form or other, unlike the BSDs. Solaris/illumos seriously has system headers with a 1989 AT&T copyright notice attached, because it is actually System V Unix code from Bell Labs that has barely changed in 30 years. So there's a lot of overlap in the design, and a lot of POSIX functionality to fall back on where the differences lie.

So after I got the system up and running, I tried to load a mozconfig file... and hit my first error before ever starting the build. Turns out that Solaris uses Ksh, and while Bash is available, it's hard to convince it to execute a script as a Bash script with all Bash features rather than a version limited to Ksh features. Anyway, it turned out Mozilla actually made a patch to remove the "Bash localism," and the mozconfig loader is now POSIX compliant (which it should have been in the first place). That was the first patch I applied.

From there, it was mostly a matter of applying build system patches so the build system would recognize Solaris. 90% of the time, it would take the same code as Linux, and it was like FreeBSD the other 10% of the time, basically. One theme that kept coming up was that I had to replace several memory-related functions like malign and madvise with posix_malign and posix_madvise, because Solaris has versions of those functions that take different arguments like caddr_t. This had to be ifdefed only because apparently a few versions of Linux don't actually have posix_malign and only have the regular version with the POSIX syntax. I would say that this was the most common unexpected compile error I kept getting caught by, some "malign" or "madvise" function somewhere in the code I forgot to change.

The build issue that consumed most of my time was figuring out why I was getting text relocations and .eh_frame issues in libxul.so. I learned everything I could about linkers and the ELF file format, and about libxul.so. Even to the point of reading Mike Hommey's blog and learning more about him, his interests, and the reasons behind his weird linker hacks and frustration with manual component registration than I really should have. I even found out that apparently on OI's official Firefox 52 build, the guy who got everything else working gave up and tried in desperation to build libxul.so with GNU LD (which usually doesn't work out) and use the Sun linker for the rest of it, and they were lucky that it worked. Turns out the reason that made it work is because Mozilla packaged a mapfile for GNU LD, and that linker actually fails even worse than the Sun linker without it.

However, it turned out that I had been trying to solve a problem I hadn't yet run into. My actual build issue was because of libffi, and it took me a while to figure out that it was relying on an external script to configure libffi that was making incorrect assumptions about several things. First issue is it assumed I wanted my .eh_frames to be read only just because I'm on x86. Well, that's not a safe assumption on Solaris, you want writable .eh_frames. Then I saw tons of text relocations, so I started researching how to avoid text relocations in PIC code (which Solaris seems to require). Then I found out you actually can't avoid them completely, because assembler code needs to access the global offset table at some point, and usually needs a PC relative relocation at some point to do so. Then, I remembered a comment I saw in a libffi source code file. "Solaris uses datarel encoding for PIC on x86." So I figured out that I had to enable that hack by changing Mozilla's libffi configuration not to use PC relative relocations on Solaris x86. So it does have a mechanism for allowing relative relocations of some kind, just not PC relative ones. That got rid of most of the text relocations, but I was still getting them in a file called win32.S, which was always included whether I wanted/needed it or not. I eventually looked at that code and found that the Solaris hack was not available there, and instead it hardcoded PC relative encoding. I was somehow able to look at that hack from sysv.S and copy it into win32.S, perform the same tests and make it apply datarel encoding where necessary (easier than it sounds if you see the file). After this, I'd already fixed an issue that made the libxul.so modules appear out of order on Solaris with a patch from Mozilla, so everything worked.

After this, I was finally able to build the browser, but it crashed almost immediately with an assertion failure to NS_IsMainThread() in NSS, that only one person had ever gotten before, and in their case it was an SSL policy issue. I found a way to avoid crashing right away by sheer accident. I specify the word "file" on the command line, and it takes me to a very simple HTTP page called file.com, with nothing but a single image on it advertising some kind of file storage service or something. None of the stacktraces really helped or made much sense, it appeared that the attempt to initialize NSS was itself the cause of the failure.

I compiled a debug version, took a crash course in how to read stacktraces, and tried in desperation applying several patches I didn't think were necessary and didn't really even like. I found this set of patches from Mozilla upstream that stabilized the browser and stopped the assertion failure, but only got it to work offline. It was able to load up XUL plugins and offline saved web pages in this state, as well as show about:config and such. It generated error pages saying the PSM component appeared to be broken or disabled. I could see threads in gdb spinning up and then crashing immediately every time I'd try to go online. I thought that NSS was completely busted for some reason. I even tried running the NSS test suite, but it passed and nothing seemed to be wrong.

I applied this one patch that changed the way the browser looked and completely busted the interface, kept it from saving any history, but only because I typed it in wrong. It went like this:

palemoon.js:

Code: Select all

pref("storage.nfs_filesystem, true);
Yes, notice that the ending quote after filesystem is missing. For whatever reason, this made Pale Moon behave a lot like a really old version of Firefox used to act when it had a corrupted database. Same symptoms, history not not being saved, navigation being busted except the URL bar, etc.

I had a weird feeling this might have changed or fixed something else, so I removed the temporary NSS patches and tried loading the browser again... and although the interface was still broken, I could now type in any URL I wanted, and nothing crashed. For some reason, even YouTube was working in this state. Though it took a full minute for a video to start playing, it was smooth once it started playing back. It's a feat I haven't been able to replicate since, the videos just refuse to play entirely due to a software raster feature failure or something. The only change I'd made recently that seemed like it could have fixed things was a change to compile NSS and NSPR with pthreads after seeing that the repositories for the official OS versions had added them in.

Thinking that adding pthreads had solved the problem (a suggestion my my mind was vulnerable to because i remembered inexplicable segfaults on Linux 20 years ago due to things being compiled without them by default), I fixed that typo... and the browser started crashing again.

So I assumed that maybe something was wrong with SQLite, if busting the database access by accident had somehow made the browser work after resolving the NSS issue. I ended up making absolutely sure that SQLite built with -D_POSIX_PTHREAD_SEMANTICS and set it up to include a linker mapfile provided from the OI repositories to make absolutely sure it built correctly. And then everything started working again. I assumed I'd finally done it... but the the next day, while trying to get YouTube to work again and making very small changes, I was getting the same problem again with every build of the browser, even with the exact same configuration that had worked before.

When I figured out why, I felt like like a huge idiot. You want to know what the difference was between the browser successfully running, and crashing this whole time, since getting it to build? It was which terminal window I ran ./mach run from. Why? Because I'd used one of those terminal windows to run the NSS test suite. Why would that make a difference? While I was running the test suite... I'd added the files in dist/bin in the object directory to LD_LIBRARY_PATH because it didn't know where to look for its own object files. So whenever I tried to run the browser from the terminal window where I'd added the NSS I'd just built to the LD_LIBRARY_PATH, everything worked fine, and when I ran it from the other one, it crashed. And so the last several patches I'd been applying and things I'd thought I'd been doing to fix or break the browser were actually completely irrelevant. I'd probably had it working since the first time I got it built and didn't realize it had no idea where to find its own libraries in the build directory.

So yeah, apparently now it builds and runs on Solaris perfectly fine. VP9 videos work, YouTube videos try to work for a few frames and then stop, but I have a feeling it might work better on actual hardware rather than using a software renderer in a VM. I have to disable Libevent's use of Solaris event ports for some weird reason to stop websites from sending PHP files to me rather than trying to parse them on the server. But yeah, I somehow got this to work in just under a month, I think. It helped a lot that the browser hasn't had extensive changes to memory handling or assembler code, that there were a ton of existing patches to a code base very similar to this one for Solaris support (though there were a lot of low-quality ones and I did have to try and sift through them), and that most of the potential trouble points were in external libraries anyway.

User avatar
therube
Board Warrior
Board Warrior
Posts: 1101
Joined: 2018-06-08, 17:02

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by therube » 2019-09-08, 13:01

Sounds like a lot of (good detective) work :-).

person45
Hobby Astronomer
Hobby Astronomer
Posts: 29
Joined: 2017-10-20, 07:00

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by person45 » 2019-09-09, 00:05

That's impressive that you would spend that much time on it without getting paid.

Your skills would come in handy when a developer wants to port a game to different platforms.

User avatar
athenian200
Fanatic
Fanatic
Posts: 153
Joined: 2018-10-28, 19:56
Contact:

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by athenian200 » 2019-09-09, 01:47

Thanks, person45. But yeah, what I did this month was probably nothing compared to what the Pale Moon developers have to do every month, trying to implement new features with little guidance, fix bugs and issues that no one's ever encountered before, and try to sift through modern Firefox security updates and figure out if we're vulnerable to the same issues or not. In my case, most of the work was done for me by Oracle and Mozilla, I just had to reach out and take advantage of it. Even the answer to my libffi issues was already sitting in the source code comments, I just had to reach a point where I had a vague idea of what that comment about datarel encoding meant. The way I see it, I got free lessons about POSIX compliance issues and working around linker/assembler bugs that I might have otherwise had to pay a lot of money to go to school for. On top of that, when I pay for a video game, I'm basically paying to challenge myself and get a bit frustrated before figuring something out. This month, I got to do that for free and achieve something in the process.

I actually have very little experience with programming. Most of what I know is writing shell scripts (MS-DOS/Windows 9x BAT files as a kid, then a few Bash scripts as a teenager, and then mostly PowerShell as an adult), though I did look at a C programming book as a teenager and take a C++ intro class at a community college a few years ago. Also played around with hex editing at one point and had a vague idea of what pointer tables were, etc.

But yeah, I'm honestly just glad I was able to do it. I kind of thought I might reach a point where I couldn't continue or had to ask for help, but I never did. I just somehow stayed focused until all the major problems were resolved. Honestly, I'm not sure if any of the places I could submit a package to would even be interested in taking this. I'm finding the package submission process and figuring out how to justify my reasoning for what I did and why it should be included a lot harder than just getting it working.

User avatar
New Tobin Paradigm
Off-Topic Sheriff
Off-Topic Sheriff
Posts: 6189
Joined: 2012-10-09, 19:37
Location: Sector 001

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by New Tobin Paradigm » 2019-09-09, 03:03

Can your changes be quantified and conditional so they don't interfere with the primary os targets?
Image
- Old and insecure for legitimate and reasonable purposes. -
http://binaryoutcast.com/ | http://thereisonlyxul.org/ | Freenode #binaryoutcast

User avatar
athenian200
Fanatic
Fanatic
Posts: 153
Joined: 2018-10-28, 19:56
Contact:

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by athenian200 » 2019-09-09, 09:18

New Tobin Paradigm wrote:
2019-09-09, 03:03
Can your changes be quantified and conditional so they don't interfere with the primary os targets?
Actually, most of them already are conditional, I was pretty careful to use #ifdef XP_SOLARIS statements or the equivalent around anything that might affect the way it builds on other platforms, though I should probably try compiling it on Windows and Linux to make sure I didn't break anything if there's any chance of this actually being accepted upstream. I'm going through and cleaning up a few things that looked a bit messy.

There are a few changes that I pulled in from Oracle and OI patchsets that I'm not really sure are necessary and weren't ifdefed, but I know what most of them do basically. One of them adds a preference to declare whether your user profile is on an NFS filesystem and sets it to "true" by default, possibly that should be "false" by default. The other one I recall is a patch that apparently improves the performance of the software renderer on displays that don't have RGB subpixel rendering (BGR and GBR displays), which I didn't ifdef because I can't imagine it being Solaris specific if it actually works or does anything useful. There's also a few changes like changing queue to std::queue because of namespace conflicts, and changing strcasestr to the PL_strcasestr from the NSPR libraries because Solaris 10 doesn't have a strcasestr function. Also, there is one place where they cast getpid to int in nsLayoutUtils.cpp for some reason, possibly because pid_t can be long on some Solaris implementations. It seems like it shouldn't cause problems because the code that's already there appears to assume pid_t is int, and I believe pid_t is usually a 32-bit signed int on Windows and Linux anyway.

I wasn't sure what direction I would end up going with it, possibly asking if there's any interest at all in contributing it upstream since it's actually done, maintaining my own personal fork to test random things out (which I would also want to be able to use on Windows if I did), or just trying to convince the OI package maintainers that they should add my New Moon build to their repositories without official branding.

User avatar
New Tobin Paradigm
Off-Topic Sheriff
Off-Topic Sheriff
Posts: 6189
Joined: 2012-10-09, 19:37
Location: Sector 001

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by New Tobin Paradigm » 2019-09-09, 10:28

I dunno if we would want it back. It is a maintainace burdon after all.. the changes you describe do consern me though.

It would depend. How big is this community? Are they gonna accept a Pale Moon with our libs and configuration or require system libs. Etc.

Are they willing to work directly with us or just piss off into their own world? Are you commited to this for the long haul?
Image
- Old and insecure for legitimate and reasonable purposes. -
http://binaryoutcast.com/ | http://thereisonlyxul.org/ | Freenode #binaryoutcast

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 24836
Joined: 2011-08-28, 17:27
Location: 58°2'16"N 14°58'31"E
Contact:

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by Moonchild » 2019-09-09, 13:33

Moved the topic to a better board.
"If you want to build a better world for yourself, you have to be willing to build one for everybody." -- Coyote Osborne
Image

User avatar
athenian200
Fanatic
Fanatic
Posts: 153
Joined: 2018-10-28, 19:56
Contact:

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by athenian200 » 2019-09-09, 19:28

New Tobin Paradigm wrote:
2019-09-09, 10:28
I dunno if we would want it back. It is a maintainace burdon after all.. the changes you describe do consern me though.
Yeah, that's definitely understandable. I would be willing to remove the changes I know aren't needed and ifdef all the remaining ones, but I do know it adds things to the code and that might be an issue. I knew Solaris was unsupported when I started, and I wasn't expecting the work to be accepted upstream.
It would depend. How big is this community? Are they gonna accept a Pale Moon with our libs and configuration or require system libs. Etc.
The build I created uses internal Pale Moon libs and configuration, because I'm not sure it would build properly with system libs. After all, there might be a need to make Pale Moon specific changes to those libraries at some point and being dependent on system libs would preclude that.

Oracle Solaris still probably has the most users and is the best supported, and OpenIndiana is essentially the "core" distribution for all the other illumos kernel builds and seems to have the closest ties to Sun/Oracle employees current and former, the original OpenSolaris project that was cancelled, and the kernel developers. I was thinking of trying to add SPARC support next after I got it building on Solaris if I managed to get the interest of that community with what I was doing, and I'm pretty sure doing that would maximize the potential number of Solaris users.

One thing I do know about them is that they're a bit desperate for a modern browser and may be willing to compromise. Their system libraries aren't in the best condition and they know it, and they're currently dependent on Firefox 52ESR as their stable/working browser and a half-busted version of Firefox 60ESR that they've had to rely on Oracle begging Mozilla to keep alive. Some of them still use Firefox 52ESR because it works better for them than Firefox 60ESR. It seems like as fast as they fix all the issues on their platform, Mozilla makes these huge changes that break everything again, and they're just barely able to keep moving from one ESR to the next by the end of the security update lifecycle.
Are they willing to work directly with us or just piss off into their own world? Are you commited to this for the long haul?
Well, I'm definitely willing to commit to supporting what I've done so far, and trying to get better at what I'm doing as I go along. I would certainly hope they wanted to work with the Pale Moon team, because I think they'd be foolish not to do so.

User avatar
New Tobin Paradigm
Off-Topic Sheriff
Off-Topic Sheriff
Posts: 6189
Joined: 2012-10-09, 19:37
Location: Sector 001

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by New Tobin Paradigm » 2019-09-09, 19:43

Well not to sound like a god damned corporate bastard but it might be a good strategic move for us provided their interests don't seriously conflict with ours. Deff sounds a million times better than the bsd people ever could be.

The reason it was removed is because we have well frankly no interest in Solaris and thought no one else did either and without a reason it didn't make sense to retain it. But shit.. if they have a need and we have the support dev wise.. I personally don't see a problem with reintroducing (properly of course) support for it. Sparc tho.. That i dunno..

How about this.. for the time being you keep your repo separate and let's work towards package acceptance and official branding status.. If it proves popular with the Solaris people we can look towards reunification.

Now for the bit that only applies to me.. You willing to do some work to get mailnews core building so Interlink can be offered as well?
Image
- Old and insecure for legitimate and reasonable purposes. -
http://binaryoutcast.com/ | http://thereisonlyxul.org/ | Freenode #binaryoutcast

User avatar
athenian200
Fanatic
Fanatic
Posts: 153
Joined: 2018-10-28, 19:56
Contact:

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by athenian200 » 2019-09-10, 00:20

New Tobin Paradigm wrote:
2019-09-09, 19:43
Well not to sound like a god damned corporate bastard but it might be a good strategic move for us provided their interests don't seriously conflict with ours. Deff sounds a million times better than the bsd people ever could be.

The reason it was removed is because we have well frankly no interest in Solaris and thought no one else did either and without a reason it didn't make sense to retain it. But shit.. if they have a need and we have the support dev wise.. I personally don't see a problem with reintroducing (properly of course) support for it. Sparc tho.. That i dunno..

How about this.. for the time being you keep your repo separate and let's work towards package acceptance and official branding status.. If it proves popular with the Solaris people we can look towards reunification.

Now for the bit that only applies to me.. You willing to do some work to get mailnews core building so Interlink can be offered as well?
Yeah, that sounds good to me. I currently have it in a forked repo called Solaris-UXP (for lack of a better name), and I'm referring to it as a Solaris compatibility fork of PM28. I've also explicitly removed the official branding from the repo and left a note asking people to use the New Moon branding for now until I obtain permission or come up with another alternative, making it clear this isn't officially endorsed (yet?) and that I just wanted to get this working on my own initiative. I was careful from the start to make sure that my little personal project didn't end up being an inconvenience for the PM team in the future if they weren't interested in Solaris support. I also didn't really know how to use Git when I started, so the way I did the commits might be a little messy and confusing.

I would be happy to try and port over Interlink as well, actually one of my main motivations was to get several UXP applications working in general, I never intended to stop with just Pale Moon. I was thinking to myself that if I could get Pale Moon and Interlink ported over to Solaris, I might switch over to using that on my PC, so that way I'd have a strong motivation to keep the port maintained as I would be using it on a regular basis. I've kinda been thinking of trying to find a Linux alternative and I was looking for something more System V-ish, and this seemed to fit the bill.

One thing I liked about OpenIndiana/OpenSolaris is that the guy behind it was Ian Murdock, the guy who is the "Ian" in Debian. He left the Linux Foundation and ended up working for Sun, and he wanted OpenSolaris to be his next big thing, the successor to Linux. He ended up resigning after Oracle acquired Sun and shut down the OpenSolaris project officially (though unofficially you still see Oracle employees showing up on the mailing list). The name change was for legal reasons, Sun had it codenamed as "Project Indiana" internally, and Indiana was also the state where he was born.

bgstack15
Moonbather
Moonbather
Posts: 57
Joined: 2018-01-22, 23:04

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by bgstack15 » 2019-09-12, 15:40

Thank you for sharing that! That post was a fantastic read. I barely understood the first parts, because I'm just a Linux noob and I've never worked with any real Unixes, and then I got lost when you started talking about .eh_frame in the compiled libs.

I don't use OpenSolaris or OpenIndiana or anything, but I think that a monoculture of Linux is a bad idea, even if the idea is a great one.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 24836
Joined: 2011-08-28, 17:27
Location: 58°2'16"N 14°58'31"E
Contact:

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by Moonchild » 2019-09-12, 18:41

Just to state here that I am happy to have Illumos/Solaris support (re)introduced into UXP, provided it is maintained by someone with the capacity, hardware and time to do so. As long as the main premise behind the browser and standard configuration is maintained, there should not be any issue making it an official distribution (with official branding) either.
Just like Tobin I was under the impression that Sun Solaris was pretty much archaic, unmaintained and not in any sort of serious use any longer.
"If you want to build a better world for yourself, you have to be willing to build one for everybody." -- Coyote Osborne
Image

User avatar
athenian200
Fanatic
Fanatic
Posts: 153
Joined: 2018-10-28, 19:56
Contact:

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by athenian200 » 2019-09-21, 20:12

I can definitely understand how you got that impression. Some of that Solaris code in Mozilla's tree hadn't been touched since 1998. Anyway, I fixed the whole casting getpid to int issue that had people concerned in a fairly... permanent way. No one will have to wonder whether they should cast to int when calling getpid() ever again. https://github.com/athenian200/Solaris- ... 6621211237

But yeah, I'm definitely interested in getting this project up to a standard where it could have official branding and everything. The main thing I'm concerned about now is how to test properly to make sure it supports everything it should support. I mean, yeah, it builds and runs without crashing, but I don't really know how to "put it through its paces," as it were. Should I try fixing up the internal testsuite to try and get that working on Solaris, or do you guys test things in a different way?

I know that right now it supports basic HTML5 VP8 and VP9 videos, but most other kinds refuse to play and just load forever, including YouTube videos. So fixing that issue the main thing still blocking me from putting this out there at this point, because from what I can tell so far, everything else works.

User avatar
adesh
Astronaut
Astronaut
Posts: 747
Joined: 2017-06-06, 07:38

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by adesh » 2019-09-22, 05:50

Internal test suite is not used for testing as it is incomplete (and broken?). Test the browser like you'd in normal daily use. Load some popular sites, load some random sites, watch some streams, read news, browse cat images, read some spam in your mail... and then give it to the sharks (if you have any). And whatever you do don't forget Facebook and YouTube otherwise no one will use it. ;)

User avatar
athenian200
Fanatic
Fanatic
Posts: 153
Joined: 2018-10-28, 19:56
Contact:

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by athenian200 » 2019-09-24, 09:51

adesh wrote:
2019-09-22, 05:50
Internal test suite is not used for testing as it is incomplete (and broken?). Test the browser like you'd in normal daily use. Load some popular sites, load some random sites, watch some streams, read news, browse cat images, read some spam in your mail... and then give it to the sharks (if you have any). And whatever you do don't forget Facebook and YouTube otherwise no one will use it. ;)
Thanks. Yeah, that's pretty much where my intuition was leading me... test JavaScript-heavy sites like YouTube and Facebook, and try to watch various kinds of videos. I used to use some Firefox nightly builds a long time ago, and what I noticed was that I didn't find a lot of middle ground during that time. Maybe it's just me, but it seemed like either I would hit really big problems within the first couple hours of using it, or else it would act pretty much like a "normal" release build, with very little in-between those two extremes.

Anyway, after some further testing, I found the video playback issue is actually an audio playback issue. My first thought was that maybe there was a problem with handling one codec or container format and I just needed to fix libvpx or ffvpx, but it seems like a lot of videos on test sites work, like ALL the videos here play back that will play just like they do on Linux or Windows.

https://test-videos.co.uk/bigbuckbunny/mkv

Eventually I noticed that every video that has an audio track (whether muted/silent or not) fails to play, and most video decoder test videos don't have an audio track. That's something I didn't notice before because the VM I'm using for Solaris doesn't actually have working audio to begin with. But now that I am testing audio files, I notice that when I try to play any audio file, the progress bar doesn't move at all, just like with any video that has an audio track. So my lack of audio support in my test environment made troubleshooting this a little harder.

I ended up installing Solaris on an actual computer that has a compatible audio chipset, and now that audio is working, my hypothesis is confirmed. YouTube (and every other video I've tested) works perfectly fine now that audio is functional, but unfortunately anyone using this on a machine with no working sound (which is a higher likelihood than on Linux or Windows) will run into video playback issues. I compared with Firefox and while it has the same freezing progress bar issue with audio, on videos it seems to fail gracefully after displaying a status message mentioning PulseAudio and just plays the video content. Also, on actual hardware, it seems that the program builds in 30 minutes instead of two hours. I had no idea how slow building inside my VM actually was, because I was used to building Firefox on ancient Linux machines (in a desperate attempt to use extra compiler optimizations and cutting out features to get it to run at a decent speed) and having it take 4 or 5 hours. Disabling audio causes YouTube to stop working again.

So it looks like I've actually got everything working. All the extensions I've tried work, videos are working, sound works fine, and I haven't run into any issues.

User avatar
athenian200
Fanatic
Fanatic
Posts: 153
Joined: 2018-10-28, 19:56
Contact:

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by athenian200 » 2019-10-07, 21:59

Well, I've confirmed that sound, YouTube, the Solaris Flash player, all the extensions I use on Windows, the theming engine, and basically everything that worked before still work after cleaning up my repository and trying to make the code as nice as possible. I still need to go back and do any kind of tagging or changing the titles to reflect issue numbers if it actually does wind up meeting Pale Moon's standards for inclusion. I know the code I have compiles and runs on Linux as well (because I used that for comparison purposes), so I really only need to test Windows (which probably shouldn't have changed because all of the Windows stuff is very separated from the Unix platforms anyway).
Image
https://github.com/athenian200/UXP-Clean/commits/master

This is about as good as I can get it for now. One odd thing to note is that what I have working right now is technically a 32-bit build produced for (and running on) a 64-bit only OS because Firefox was configured to produce 32-bit binaries on Solaris (on both x86 and SPARC) up until much later on, and it doesn't help that the compilers on here tend to generate 32-bit code by default unless you specifically try to use -m64 everywhere. However, this isn't as big a liability as it would be on Linux, because Solaris ships with both 64-bit and 32-bit libraries by default, and has a high degree of compatibility with 32-bit applications for historical reasons, even though the OS itself has been 64-bit only for a few years now. There is a possibility of the 32-bit version having higher compatibility with NPAPI plugins, so it may even be desirable for some people to run this version.

I am now in the process of trying to get it to build a 64-bit version as well in order to future-proof things, because they are slowly recompiling various packages to be 64-bit instead of 32-bit, but it's a slow process since the urgency isn't there due to the ease of running 32-bit binaries on a 64-bit system, and they'll probably retain that 32-bit backwards compatibility for a long time. It's not like the situation I remember from Linux where people would pretty much have to cross-compile 32-bit binaries for a 64-bit system, so I was actually surprised when I found out it was building a 32-bit version at first. Solaris tends to just install both versions of all the main development libraries, all the tools you need for both, and uses the 32-bit legacy mode by default to avoid breaking source code meant to build on older versions (if they changed the default to 64-bit, it would break decades worth of code that assumes it's not).

User avatar
New Tobin Paradigm
Off-Topic Sheriff
Off-Topic Sheriff
Posts: 6189
Joined: 2012-10-09, 19:37
Location: Sector 001

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by New Tobin Paradigm » 2019-10-07, 23:04

So.. I notice your UXP repo did not come as a result from GitHub Forking our UXP repo... Thus, compare and pull requests are impossible.. You might want to recreate your repo as a GitHub Fork of ours and re-apply your patches.

Failing that, none of us can properly evaluate the scope of your changes making a potential upstream merge exceedingly difficult.
Image
- Old and insecure for legitimate and reasonable purposes. -
http://binaryoutcast.com/ | http://thereisonlyxul.org/ | Freenode #binaryoutcast

User avatar
athenian200
Fanatic
Fanatic
Posts: 153
Joined: 2018-10-28, 19:56
Contact:

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by athenian200 » 2019-10-10, 21:01

Thanks for letting me know. I'm a bit new to GitHub, and I was trying to clean up my repository, but I think it did it in the wrong way. I think I can fix it now that you've told me what the problem is, I never tried to do any compare or pull requests so I never noticed.

But in any case, I got the AMD64 build working. So I am basically completely done with the original task and can focus on clean-up and learning about how to do pull requests, how to use GitHub properly, etc. Probably should have started with that, but I kind of got in that mindframe where I was laser-focused on getting this specific task done.

https://github.com/athenian200/UXP/commits/master

EDIT: This should look better, though it has one dummy commit near the end where I tried to merge the branches, which shouldn't have changed anything.

User avatar
New Tobin Paradigm
Off-Topic Sheriff
Off-Topic Sheriff
Posts: 6189
Joined: 2012-10-09, 19:37
Location: Sector 001

Re: I created a Solaris/illumos port of UXP out of boredom.

Unread post by New Tobin Paradigm » 2019-10-10, 21:43

What stands out to me is this.. https://github.com/MoonchildProductions/UXP/compare/master...athenian200:master#diff-f1275a779e0e27ba3e8cd72530af6d4e

This is not acceptable. We need to figure out why or how the defines are not being passed down the line. Temporarily exterminate this change and then issue a mach configure and attach config.status from the obj-dir.

It might help to give me a primer on constructing a build env. Where to get stuff and do the things. Feel free to stop by #binaryoutcast on freenode if you wanna chat directly.
Image
- Old and insecure for legitimate and reasonable purposes. -
http://binaryoutcast.com/ | http://thereisonlyxul.org/ | Freenode #binaryoutcast

Post Reply