A linux build framework for Pale Moon

Users and developers helping users with generic and technical Pale Moon issues on all operating systems.

Moderator: trava90

Forum rules
This board is for technical/general usage questions and troubleshooting for the Pale Moon browser only.
Technical issues and questions not related to the Pale Moon browser should be posted in other boards!
Please keep off-topic and general discussion out of this board, thank you!
JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-02-27, 21:16

Hi!
'ac_add_options --enable-x32' does not seem to make any difference, same error, same place.

I also follow the sugestion by stevepusser to try the debian way, again, same error, same place:

Code: Select all

In file included from /usr/include/x86_64-linux-gnux32/sys/sysctl.h:63:0,
                 from ../../dist/system_wrappers/sys/sysctl.h:3,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/ipc/chromium/src/third_party/libevent/./arc4random.c:62,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/ipc/chromium/src/third_party/libevent/evutil_rand.c:104:
/usr/include/x86_64-linux-gnux32/bits/sysctl.h:19:3: error: #error "sysctl system call is unsupported in x32 kernel"
 # error "sysctl system call is unsupported in x32 kernel"
   ^
/opt/BUILD/debian/palemoon-27.1.1~repack/config/rules.mk:904: recipe for target 'evutil_rand.o' failed
make[7]: *** [evutil_rand.o] Error 1
make[7]: Leaving directory '/opt/BUILD/debian/palemoon-27.1.1~repack/obj-x86_64-pc-linux-gnu/ipc/chromium'
So, the offending lines seem to be these, in arc4random.c:
#ifdef _EVENT_HAVE_SYS_SYSCTL_H
#include <sys/sysctl.h>
#endif
but, if there is no <sys/sysctl.h>, the build will also error, of course.

Now a question, why the Palemoon build needs to build a libevent, instead of using the system libevent, that is installed and available??

ii libevent-2.0-5:x32 2.0.21-stable-3 x32 Asynchronous event notification library
ii libevent-core-2.0-5:x32 2.0.21-stable-3 x32 Asynchronous event notification library (core)
ii libevent-dev 2.0.21-stable-3 x32 Asynchronous event notification library (development files)
ii libevent-extra-2.0-5:x32 2.0.21-stable-3 x32 Asynchronous event notification library (extra)
ii libevent-openssl-2.0-5:x32 2.0.21-stable-3 x32 Asynchronous event notification library (openssl)
ii libevent-pthreads-2.0-5:x32 2.0.21-stable-3 x32 Asynchronous event notification library (pthreads)

( and the same goes for other libs, like libssl3, libnss3, etc. )

Walter Dnes
Astronaut
Astronaut
Posts: 650
Joined: 2015-07-30, 20:29
Location: Vaughan, ON, Canada

Re: A linux build framework for Pale Moon

Unread post by Walter Dnes » 2017-02-27, 23:26

JotaMG wrote:Now a question, why the Palemoon build needs to build a libevent, instead of using the system libevent, that is installed and available??
[...snip...]
( and the same goes for other libs, like libssl3, libnss3, etc. )
Some API/ABI changes occur every so often in external libs, and cause trouble for Pale Moon. It's generally recommended not to use "--with-system-libfoo", even when it is available.

When I tried building Pale Moon unstable under 32-bit uclibc-ng (instead of glibc), The build died on the exact same file...

Code: Select all

 4:46.34 /home/waltdnes/pmmain/pmsrc/ipc/chromium/src/third_party/libevent/./arc4random.c:449:1: error: conflicting types for 'arc4random_addrandom'
 4:46.34  arc4random_addrandom(const unsigned char *dat, int datlen)
 4:46.35  ^
 4:46.35 In file included from ../../dist/system_wrappers/stdlib.h:3:0,
 4:46.35                  from /home/waltdnes/pmmain/pmsrc/ipc/chromium/src/third_party/libevent/util-internal.h:35,
 4:46.35                  from /home/waltdnes/pmmain/pmsrc/ipc/chromium/src/third_party/libevent/evutil_rand.c:39:
 4:46.35 /usr/include/stdlib.h:927:13: note: previous declaration of 'arc4random_addrandom' was here
 4:46.35  extern void arc4random_addrandom(unsigned char *, int);
 4:46.35              ^
I asked on github https://github.com/MoonchildProductions ... issues/910 and was told that uclibc-ng is not supported. It seems there's something hard-coded in that file, expecting very specific stuff.
There's a right way
There's a wrong way
And then there's my way

User avatar
stevenpusser
Project Contributor
Project Contributor
Posts: 903
Joined: 2015-08-01, 18:33

Re: A linux build framework for Pale Moon

Unread post by stevenpusser » 2017-02-28, 20:59

You could try adding that system libevent option (ac_add_options --with-system-libevent) to the debian/mozconfig file and add libevent-dev to the build-depends in the debian/control file as an experiment.

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-03-01, 10:15

stevepusser wrote:You could try adding that system libevent option (ac_add_options --with-system-libevent) to the debian/mozconfig file and add libevent-dev to the build-depends in the debian/control file as an experiment.
Yes, I've done that already, also using system's libicu, and the build goes on, until hitting assembler problems like this:

Code: Select all

Unified_cpp_gfx_ycbcr0.o
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp: Assembler messages:
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:150: Error: `(%r11,%r9d)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:155: Error: `(%r10,%r9d)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:157: Error: `(%edi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:172: Error: `(%esi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:174: Error: `(%edx,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:266: Error: `(%esi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:267: Error: `1(%esi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:276: Error: `(%edx,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:277: Error: `1(%edx,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:287: Error: `(%r11,%r9d)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:290: Error: `(%edi,%r11,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:291: Error: `1(%edi,%r11,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:300: Error: `(%r10,%r9d)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:302: Error: `(%edi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:303: Error: `1(%edi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:324: Error: `(%esi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:326: Error: `(%edx,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/config/rules.mk:936: recipe for target 'yuv_row_posix.o' failed
make[7]: *** [yuv_row_posix.o] Error 1
make[7]: Leaving directory '/opt/BUILD/debian/palemoon-27.1.1~repack/obj-x86_64-pc-linux-gnu/gfx/ycbcr'
/opt/BUILD/debian/palemoon-27.1.1~repack/config/recurse.mk:74: recipe for target 'gfx/ycbcr/target' failed
make[6]: *** [gfx/ycbcr/target] Error 2
make[6]: *** Waiting for unfinished jobs....
These are most operand size mismatches, I'm fixing them, but this is very time consuming, so going slow, step by step.
BTW, at this stage, how far is the build from the end?

New Tobin Paradigm

Re: A linux build framework for Pale Moon

Unread post by New Tobin Paradigm » 2017-03-01, 10:24

System Libs are NOT supported when in-tree versions exist. Deal with that fact.

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-03-01, 11:26

Matt A Tobin wrote:System Libs are NOT supported when in-tree versions exist. Deal with that fact.

If you have a better solution, please help!

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-03-01, 13:47

The sysctl syscall has been deprecated on Linux for ages, but still it pops up in Palemoon code....

/memory/mozjemalloc/jemalloc.c:
#if !defined(MOZ_MEMORY_SOLARIS) && !defined(MOZ_MEMORY_ANDROID)
#include <sys/sysctl.h>
#endif

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-03-01, 21:34

Progressing, but this one don't know how to deal:

Code: Select all

In file included from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jit/shared/MacroAssembler-x86-shared.h:16:0,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jit/x64/MacroAssembler-x64.h:12,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jit/MacroAssembler.h:17,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jit/BaselineJIT.h:18,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/vm/TypeInference-inl.h:17,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jsobjinlines.h:26,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/builtin/RegExp.cpp:17:
/opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jit/x64/Assembler-x64.h:213:81: error: no matching function for call to ‘js::jit::ValueOperand::ValueOperand(const js::jit::Register&)’
 static MOZ_CONSTEXPR_VAR ValueOperand JSReturnOperand = ValueOperand(JSReturnReg);
                                                                                 ^
Assembler-x64.h:
...
namespace js {
namespace jit {

// Return operand from a JS -> JS call.
static MOZ_CONSTEXPR_VAR ValueOperand JSReturnOperand = ValueOperand(JSReturnReg);
...


Any ideas??

Walter Dnes
Astronaut
Astronaut
Posts: 650
Joined: 2015-07-30, 20:29
Location: Vaughan, ON, Canada

Re: A linux build framework for Pale Moon

Unread post by Walter Dnes » 2017-03-02, 02:30

JotaMG wrote:The sysctl syscall has been deprecated on Linux for ages, but still it pops up in Palemoon code....

/memory/mozjemalloc/jemalloc.c:
#if !defined(MOZ_MEMORY_SOLARIS) && !defined(MOZ_MEMORY_ANDROID)
#include <sys/sysctl.h>
#endif
I'm having a similar "excellent adventure" with uclibc-ng. It appears that both x32 and uclibc-ng building get tripped up by some hard-coded assumptions in the Pale Moon / Firefox code.

The 1st build attempt died because it couldn't find "/lib/libc.so.6". Not surprising, because there is no such file. There is, however, "/lib/libc.so.0" which is actually a symlink to "libuClibc-0.1.1.22.so". I created a symlink "/lib/libc.so.6" pointing at "libuClibc-0.1.1.22.so", and the build got further before dying.

Now it died during the building of the internal libevent code. Steve Pusser had suggested using the system libevent library for someone building under x32. I also did that, and the build died almost immediately. Problem was that with this being a fresh install, nothing had yet pulled in libevent, so of course the build script couldn't find it.

I installed libevent manually and tried the build again. It got a lot further before dying. Now it died during the building of the internal jemalloc code. If it worked once, let's try it again. I disabled building the jemalloc library. The result is that Pale Moon falls back to the system malloc. I tried the build script, and it actually finished building this time.

So I scp'd the tarball from the uclibc-ng chroot on my desktop to the Atom netbook, and decompressed it in directory /home/waltdnes/pm/, cd'd to directory palemoon and launched it like so...
==================================================
[aa1][waltdnes][~/pm/palemoon] ./palemoon
XPCOMGlueLoad error for file /home/waltdnes/pm/palemoon/libplc4.so:
File not found
Couldn't load XPCOM.
==================================================

But, but, but /home/waltdnes/pm/palemoon/libplc4.so does exist. A bit of web-searching suggested using LD_LIBRARY_PATH, so I set up a script like so...

=================================================================
#!/bin/bash
LD_LIBRARY_PATH=/home/waltdnes/pm/palemoon /home/waltdnes/pm/palemoon/palemoon
=================================================================

...and ran it. Pale Moon actually came up, but froze almost immediately. There were a bunch of error messages in the xterm console I launched from. One of them complained about not being able to do a text relocation in libavcodec.so.54, and suggesting that I rebuild with "-fPIC". I rebuilt Pale Moon and ffmpeg, and tried again. It did not freeze, but still had the same error messages. I could click on icons in the "start.me" page or on bookmarks and Pale Moon would take me there. But although I could type in URLs in the address bar, it did nothing.

I guess it wants everything rebuilt with "-fPIC". There are 343 installed packages in the chroot on my desktop and 352 on the Atom netbook. The desktop chroot is chugging along. The poor netbook is now at 51 of 352. Once both are rebuilt, I'll rebuild Pale Moon and see what happens. Stay tuned for the next episode of "As The Hard Drive Turns".
There's a right way
There's a wrong way
And then there's my way

User avatar
stevenpusser
Project Contributor
Project Contributor
Posts: 903
Joined: 2015-08-01, 18:33

Re: A linux build framework for Pale Moon

Unread post by stevenpusser » 2017-03-02, 03:28

I was going to suggest looking at the Debian source files for something like libevent to see how they got it to build on x32 for the unofficial repo they have, and then see if any patches could be applied to the internal libevent in PM, but the repository seems to only have the finished deb packages, and no source. :wtf:

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-03-02, 09:31

I think they just disable the use of sys/sysctl.h with something like

#if defined(BSD) || defined(DARWIN)
#include <sys/sysctl.h>
#endif


This Gentoo firefox https://gentoobrowse.randomdan.homeip.n ... nt/firefox is built with system libs:
system-cairo
system-harfbuzz
system-icu
system-jpeg
system-libevent
system-libvpx
system-sqlite


Well, for my x32 build, is there a way to disable the JIT building, that is now the blocker??

New Tobin Paradigm

Re: A linux build framework for Pale Moon

Unread post by New Tobin Paradigm » 2017-03-02, 12:17

Pale Moon is not Firefox and never will be again.

Walter Dnes
Astronaut
Astronaut
Posts: 650
Joined: 2015-07-30, 20:29
Location: Vaughan, ON, Canada

Re: A linux build framework for Pale Moon

Unread post by Walter Dnes » 2017-03-02, 20:33

JotaMG wrote:Well, for my x32 build, is there a way to disable the JIT building, that is now the blocker??
In the source code in file js/src/configure.in at approximately line 2120, look for...

Code: Select all

MOZ_ARG_DISABLE_BOOL(ion,
[  --disable-ion      Disable use of the IonMonkey JIT],
  ENABLE_ION= )
I'm not certain if you need to pass "0" or "1" to "ENABLE_ION=". I don't know if either option would help, but it's worth a try.
There's a right way
There's a wrong way
And then there's my way

User avatar
stevenpusser
Project Contributor
Project Contributor
Posts: 903
Joined: 2015-08-01, 18:33

Re: A linux build framework for Pale Moon

Unread post by stevenpusser » 2017-03-02, 21:12

That sounds to me like an option that could be set in mozconfig instead of having to change the source code.

I also wonder if Debian's libevent sources just build on x32 without any modifications, and what differences there are between the version included in PM and the Debian versions. It's Debian policy to build programs to use system libraries as much as possible instead of the included versions--damn what the upstream developers say--I can see this leading to friction if someone ever does try to get it into Debian.

The fight over the xscreensaver packages and the included time bomb nag screen was quite interesting.

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-03-02, 22:55

Yes,
ac_add_options --disable-ion
in mozconfig.txt disable JIT

So, my build has progressed, but has stopped with a "collect2: error: ld return 1 exit status"
arising, it seems, from some "undefined reference to 'ffi_call'"

I'm doing this build on a old laptop with only 2GB Ram, and a 2GB swap, so, is it possible that this is from lack of memory, or not?

Regarding libevent, I'm using the debian system libevent, so it is build for x32.

* by the way, I'm typing this now on a x32 desktop, using Midori browser, that used to crash a lot, but today, there was a massive update of packages, and it seems more stable. Very nice experience!

Walter Dnes
Astronaut
Astronaut
Posts: 650
Joined: 2015-07-30, 20:29
Location: Vaughan, ON, Canada

Re: A linux build framework for Pale Moon

Unread post by Walter Dnes » 2017-03-02, 22:58

stevepusser wrote:That sounds to me like an option that could be set in mozconfig instead of having to change the source code.

Code: Select all

ac_add_options --disable-ion
may not work. My standard "spelunker" script found "--enable-x32", but not this. I had to "grep -ir jit" through the source tree to find it.
stevepusser wrote:I also wonder if Debian's libevent sources just build on x32 without any modifications, and what differences there are between the version included in PM and the Debian versions. It's Debian policy to build programs to use system libraries as much as possible instead of the included versions--damn what the upstream developers say--I can see this leading to friction if someone ever does try to get it into Debian.
No problem. Disable official branding, like you temporarily did. If they want to go one step further, they could repeat what they once did with Firefox

Code: Select all

ac_add_options --with-app-name=iceweasel
or maybe even

Code: Select all

ac_add_options --with-app-name=brightsun
There's a right way
There's a wrong way
And then there's my way

User avatar
stevenpusser
Project Contributor
Project Contributor
Posts: 903
Joined: 2015-08-01, 18:33

Re: A linux build framework for Pale Moon

Unread post by stevenpusser » 2017-03-03, 03:09

JotaMG wrote:Yes,
ac_add_options --disable-ion
in mozconfig.txt disable JIT

So, my build has progressed, but has stopped with a "collect2: error: ld return 1 exit status"
arising, it seems, from some "undefined reference to 'ffi_call'"

I'm doing this build on a old laptop with only 2GB Ram, and a 2GB swap, so, is it possible that this is from lack of memory, or not?

Regarding libevent, I'm using the debian system libevent, so it is build for x32.

* by the way, I'm typing this now on a x32 desktop, using Midori browser, that used to crash a lot, but today, there was a massive update of packages, and it seems more stable. Very nice experience!
.
Yes, PM does use the most memory during its linking step after the compiling is done. Do you see it using the swap file during this?

But I've built it in a 3 GB amd64 virtual machine without having it touch swap, so I don't think you're running out of memory. When I've had that happen in the past, the whole machine, virtual or real, just locks up.

Once in a while when I get a build failure that mystifies me, I can find a fix by doing a websearch for the error message. Maybe there's some parameter that can be passed to the linker, but I'm just a hobby packager and don't know much of that deep stuff at all.

JotaMG

Re: A linux build framework for Pale Moon

Unread post by JotaMG » 2017-03-06, 18:42

I've give up of building Palemoon on x32 ABI.
The blocker seems to be JS, with JIT the assembler code is not ready for x32 and not compile, without JIT, lots of linker errors.
I thank stevepusser and Walter for the help.

New Tobin Paradigm

Re: A linux build framework for Pale Moon

Unread post by New Tobin Paradigm » 2017-03-06, 20:15

You guys do realize you are going WAY the hell out of spec for Pale Moon builds right?

User avatar
stevenpusser
Project Contributor
Project Contributor
Posts: 903
Joined: 2015-08-01, 18:33

Re: A linux build framework for Pale Moon

Unread post by stevenpusser » 2017-03-07, 01:19

Yes...but for SCIENCE!!!!! :crazy:

I like to try experiments with some packages--right now I'm trying to get the Debian ffmpeg 3.2.4 to support Intel Quick Sync hardware GPU-accelerated video transcoding with my Skylake GPU. I successfully ported over the Intel libmfx-dev library and rebuilt ffmpeg so it shows that it supports the encoder, but I get an error when I actually try it to use it, so I need to research that.

Locked