Page 4 of 5

Re: A linux build framework for Pale Moon

Posted: Mon, 27 Feb 2017, 21:16
by JotaMG
Hi!
'ac_add_options --enable-x32' does not seem to make any difference, same error, same place.

I also follow the sugestion by stevepusser to try the debian way, again, same error, same place:

Code: Select all

In file included from /usr/include/x86_64-linux-gnux32/sys/sysctl.h:63:0,
                 from ../../dist/system_wrappers/sys/sysctl.h:3,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/ipc/chromium/src/third_party/libevent/./arc4random.c:62,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/ipc/chromium/src/third_party/libevent/evutil_rand.c:104:
/usr/include/x86_64-linux-gnux32/bits/sysctl.h:19:3: error: #error "sysctl system call is unsupported in x32 kernel"
 # error "sysctl system call is unsupported in x32 kernel"
   ^
/opt/BUILD/debian/palemoon-27.1.1~repack/config/rules.mk:904: recipe for target 'evutil_rand.o' failed
make[7]: *** [evutil_rand.o] Error 1
make[7]: Leaving directory '/opt/BUILD/debian/palemoon-27.1.1~repack/obj-x86_64-pc-linux-gnu/ipc/chromium'


So, the offending lines seem to be these, in arc4random.c:
#ifdef _EVENT_HAVE_SYS_SYSCTL_H
#include <sys/sysctl.h>
#endif
but, if there is no <sys/sysctl.h>, the build will also error, of course.

Now a question, why the Palemoon build needs to build a libevent, instead of using the system libevent, that is installed and available??

ii libevent-2.0-5:x32 2.0.21-stable-3 x32 Asynchronous event notification library
ii libevent-core-2.0-5:x32 2.0.21-stable-3 x32 Asynchronous event notification library (core)
ii libevent-dev 2.0.21-stable-3 x32 Asynchronous event notification library (development files)
ii libevent-extra-2.0-5:x32 2.0.21-stable-3 x32 Asynchronous event notification library (extra)
ii libevent-openssl-2.0-5:x32 2.0.21-stable-3 x32 Asynchronous event notification library (openssl)
ii libevent-pthreads-2.0-5:x32 2.0.21-stable-3 x32 Asynchronous event notification library (pthreads)

( and the same goes for other libs, like libssl3, libnss3, etc. )

Re: A linux build framework for Pale Moon

Posted: Mon, 27 Feb 2017, 23:26
by Walter Dnes
JotaMG wrote:Now a question, why the Palemoon build needs to build a libevent, instead of using the system libevent, that is installed and available??
[...snip...]
( and the same goes for other libs, like libssl3, libnss3, etc. )

Some API/ABI changes occur every so often in external libs, and cause trouble for Pale Moon. It's generally recommended not to use "--with-system-libfoo", even when it is available.

When I tried building Pale Moon unstable under 32-bit uclibc-ng (instead of glibc), The build died on the exact same file...

Code: Select all

 4:46.34 /home/waltdnes/pmmain/pmsrc/ipc/chromium/src/third_party/libevent/./arc4random.c:449:1: error: conflicting types for 'arc4random_addrandom'
 4:46.34  arc4random_addrandom(const unsigned char *dat, int datlen)
 4:46.35  ^
 4:46.35 In file included from ../../dist/system_wrappers/stdlib.h:3:0,
 4:46.35                  from /home/waltdnes/pmmain/pmsrc/ipc/chromium/src/third_party/libevent/util-internal.h:35,
 4:46.35                  from /home/waltdnes/pmmain/pmsrc/ipc/chromium/src/third_party/libevent/evutil_rand.c:39:
 4:46.35 /usr/include/stdlib.h:927:13: note: previous declaration of 'arc4random_addrandom' was here
 4:46.35  extern void arc4random_addrandom(unsigned char *, int);
 4:46.35              ^

I asked on github https://github.com/MoonchildProductions ... issues/910 and was told that uclibc-ng is not supported. It seems there's something hard-coded in that file, expecting very specific stuff.

Re: A linux build framework for Pale Moon

Posted: Tue, 28 Feb 2017, 20:59
by stevepusser
You could try adding that system libevent option (ac_add_options --with-system-libevent) to the debian/mozconfig file and add libevent-dev to the build-depends in the debian/control file as an experiment.

Re: A linux build framework for Pale Moon

Posted: Wed, 01 Mar 2017, 10:15
by JotaMG
stevepusser wrote:You could try adding that system libevent option (ac_add_options --with-system-libevent) to the debian/mozconfig file and add libevent-dev to the build-depends in the debian/control file as an experiment.


Yes, I've done that already, also using system's libicu, and the build goes on, until hitting assembler problems like this:

Code: Select all

Unified_cpp_gfx_ycbcr0.o
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp: Assembler messages:
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:150: Error: `(%r11,%r9d)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:155: Error: `(%r10,%r9d)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:157: Error: `(%edi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:172: Error: `(%esi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:174: Error: `(%edx,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:266: Error: `(%esi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:267: Error: `1(%esi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:276: Error: `(%edx,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:277: Error: `1(%edx,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:287: Error: `(%r11,%r9d)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:290: Error: `(%edi,%r11,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:291: Error: `1(%edi,%r11,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:300: Error: `(%r10,%r9d)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:302: Error: `(%edi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:303: Error: `1(%edi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:324: Error: `(%esi,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/gfx/ycbcr/yuv_row_posix.cpp:326: Error: `(%edx,%r10,1)' is not a valid base/index expression
/opt/BUILD/debian/palemoon-27.1.1~repack/config/rules.mk:936: recipe for target 'yuv_row_posix.o' failed
make[7]: *** [yuv_row_posix.o] Error 1
make[7]: Leaving directory '/opt/BUILD/debian/palemoon-27.1.1~repack/obj-x86_64-pc-linux-gnu/gfx/ycbcr'
/opt/BUILD/debian/palemoon-27.1.1~repack/config/recurse.mk:74: recipe for target 'gfx/ycbcr/target' failed
make[6]: *** [gfx/ycbcr/target] Error 2
make[6]: *** Waiting for unfinished jobs....


These are most operand size mismatches, I'm fixing them, but this is very time consuming, so going slow, step by step.
BTW, at this stage, how far is the build from the end?

Re: A linux build framework for Pale Moon

Posted: Wed, 01 Mar 2017, 10:24
by New Tobin Paradigm
System Libs are NOT supported when in-tree versions exist. Deal with that fact.

Re: A linux build framework for Pale Moon

Posted: Wed, 01 Mar 2017, 11:26
by JotaMG
Matt A Tobin wrote:System Libs are NOT supported when in-tree versions exist. Deal with that fact.


If you have a better solution, please help!

Re: A linux build framework for Pale Moon

Posted: Wed, 01 Mar 2017, 13:47
by JotaMG
The sysctl syscall has been deprecated on Linux for ages, but still it pops up in Palemoon code....

/memory/mozjemalloc/jemalloc.c:
#if !defined(MOZ_MEMORY_SOLARIS) && !defined(MOZ_MEMORY_ANDROID)
#include <sys/sysctl.h>
#endif

Re: A linux build framework for Pale Moon

Posted: Wed, 01 Mar 2017, 21:34
by JotaMG
Progressing, but this one don't know how to deal:

Code: Select all

In file included from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jit/shared/MacroAssembler-x86-shared.h:16:0,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jit/x64/MacroAssembler-x64.h:12,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jit/MacroAssembler.h:17,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jit/BaselineJIT.h:18,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/vm/TypeInference-inl.h:17,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jsobjinlines.h:26,
                 from /opt/BUILD/debian/palemoon-27.1.1~repack/js/src/builtin/RegExp.cpp:17:
/opt/BUILD/debian/palemoon-27.1.1~repack/js/src/jit/x64/Assembler-x64.h:213:81: error: no matching function for call to ‘js::jit::ValueOperand::ValueOperand(const js::jit::Register&)’
 static MOZ_CONSTEXPR_VAR ValueOperand JSReturnOperand = ValueOperand(JSReturnReg);
                                                                                 ^


Assembler-x64.h:
...
namespace js {
namespace jit {

// Return operand from a JS -> JS call.
static MOZ_CONSTEXPR_VAR ValueOperand JSReturnOperand = ValueOperand(JSReturnReg);
...


Any ideas??

Re: A linux build framework for Pale Moon

Posted: Thu, 02 Mar 2017, 02:30
by Walter Dnes
JotaMG wrote:The sysctl syscall has been deprecated on Linux for ages, but still it pops up in Palemoon code....

/memory/mozjemalloc/jemalloc.c:
#if !defined(MOZ_MEMORY_SOLARIS) && !defined(MOZ_MEMORY_ANDROID)
#include <sys/sysctl.h>
#endif

I'm having a similar "excellent adventure" with uclibc-ng. It appears that both x32 and uclibc-ng building get tripped up by some hard-coded assumptions in the Pale Moon / Firefox code.

The 1st build attempt died because it couldn't find "/lib/libc.so.6". Not surprising, because there is no such file. There is, however, "/lib/libc.so.0" which is actually a symlink to "libuClibc-0.1.1.22.so". I created a symlink "/lib/libc.so.6" pointing at "libuClibc-0.1.1.22.so", and the build got further before dying.

Now it died during the building of the internal libevent code. Steve Pusser had suggested using the system libevent library for someone building under x32. I also did that, and the build died almost immediately. Problem was that with this being a fresh install, nothing had yet pulled in libevent, so of course the build script couldn't find it.

I installed libevent manually and tried the build again. It got a lot further before dying. Now it died during the building of the internal jemalloc code. If it worked once, let's try it again. I disabled building the jemalloc library. The result is that Pale Moon falls back to the system malloc. I tried the build script, and it actually finished building this time.

So I scp'd the tarball from the uclibc-ng chroot on my desktop to the Atom netbook, and decompressed it in directory /home/waltdnes/pm/, cd'd to directory palemoon and launched it like so...
==================================================
[aa1][waltdnes][~/pm/palemoon] ./palemoon
XPCOMGlueLoad error for file /home/waltdnes/pm/palemoon/libplc4.so:
File not found
Couldn't load XPCOM.
==================================================

But, but, but /home/waltdnes/pm/palemoon/libplc4.so does exist. A bit of web-searching suggested using LD_LIBRARY_PATH, so I set up a script like so...

=================================================================
#!/bin/bash
LD_LIBRARY_PATH=/home/waltdnes/pm/palemoon /home/waltdnes/pm/palemoon/palemoon
=================================================================

...and ran it. Pale Moon actually came up, but froze almost immediately. There were a bunch of error messages in the xterm console I launched from. One of them complained about not being able to do a text relocation in libavcodec.so.54, and suggesting that I rebuild with "-fPIC". I rebuilt Pale Moon and ffmpeg, and tried again. It did not freeze, but still had the same error messages. I could click on icons in the "start.me" page or on bookmarks and Pale Moon would take me there. But although I could type in URLs in the address bar, it did nothing.

I guess it wants everything rebuilt with "-fPIC". There are 343 installed packages in the chroot on my desktop and 352 on the Atom netbook. The desktop chroot is chugging along. The poor netbook is now at 51 of 352. Once both are rebuilt, I'll rebuild Pale Moon and see what happens. Stay tuned for the next episode of "As The Hard Drive Turns".

Re: A linux build framework for Pale Moon

Posted: Thu, 02 Mar 2017, 03:28
by stevepusser
I was going to suggest looking at the Debian source files for something like libevent to see how they got it to build on x32 for the unofficial repo they have, and then see if any patches could be applied to the internal libevent in PM, but the repository seems to only have the finished deb packages, and no source. :wtf:

Re: A linux build framework for Pale Moon

Posted: Thu, 02 Mar 2017, 09:31
by JotaMG
I think they just disable the use of sys/sysctl.h with something like

#if defined(BSD) || defined(DARWIN)
#include <sys/sysctl.h>
#endif


This Gentoo firefox https://gentoobrowse.randomdan.homeip.n ... nt/firefox is built with system libs:
system-cairo
system-harfbuzz
system-icu
system-jpeg
system-libevent
system-libvpx
system-sqlite


Well, for my x32 build, is there a way to disable the JIT building, that is now the blocker??

Re: A linux build framework for Pale Moon

Posted: Thu, 02 Mar 2017, 12:17
by New Tobin Paradigm
Pale Moon is not Firefox and never will be again.

Re: A linux build framework for Pale Moon

Posted: Thu, 02 Mar 2017, 20:33
by Walter Dnes
JotaMG wrote:Well, for my x32 build, is there a way to disable the JIT building, that is now the blocker??

In the source code in file js/src/configure.in at approximately line 2120, look for...

Code: Select all

MOZ_ARG_DISABLE_BOOL(ion,
[  --disable-ion      Disable use of the IonMonkey JIT],
  ENABLE_ION= )

I'm not certain if you need to pass "0" or "1" to "ENABLE_ION=". I don't know if either option would help, but it's worth a try.

Re: A linux build framework for Pale Moon

Posted: Thu, 02 Mar 2017, 21:12
by stevepusser
That sounds to me like an option that could be set in mozconfig instead of having to change the source code.

I also wonder if Debian's libevent sources just build on x32 without any modifications, and what differences there are between the version included in PM and the Debian versions. It's Debian policy to build programs to use system libraries as much as possible instead of the included versions--damn what the upstream developers say--I can see this leading to friction if someone ever does try to get it into Debian.

The fight over the xscreensaver packages and the included time bomb nag screen was quite interesting.

Re: A linux build framework for Pale Moon

Posted: Thu, 02 Mar 2017, 22:55
by JotaMG
Yes,
ac_add_options --disable-ion
in mozconfig.txt disable JIT

So, my build has progressed, but has stopped with a "collect2: error: ld return 1 exit status"
arising, it seems, from some "undefined reference to 'ffi_call'"

I'm doing this build on a old laptop with only 2GB Ram, and a 2GB swap, so, is it possible that this is from lack of memory, or not?

Regarding libevent, I'm using the debian system libevent, so it is build for x32.

* by the way, I'm typing this now on a x32 desktop, using Midori browser, that used to crash a lot, but today, there was a massive update of packages, and it seems more stable. Very nice experience!

Re: A linux build framework for Pale Moon

Posted: Thu, 02 Mar 2017, 22:58
by Walter Dnes
stevepusser wrote:That sounds to me like an option that could be set in mozconfig instead of having to change the source code.

Code: Select all

ac_add_options --disable-ion

may not work. My standard "spelunker" script found "--enable-x32", but not this. I had to "grep -ir jit" through the source tree to find it.

stevepusser wrote:I also wonder if Debian's libevent sources just build on x32 without any modifications, and what differences there are between the version included in PM and the Debian versions. It's Debian policy to build programs to use system libraries as much as possible instead of the included versions--damn what the upstream developers say--I can see this leading to friction if someone ever does try to get it into Debian.

No problem. Disable official branding, like you temporarily did. If they want to go one step further, they could repeat what they once did with Firefox

Code: Select all

ac_add_options --with-app-name=iceweasel

or maybe even

Code: Select all

ac_add_options --with-app-name=brightsun

Re: A linux build framework for Pale Moon

Posted: Fri, 03 Mar 2017, 03:09
by stevepusser
JotaMG wrote:Yes,
ac_add_options --disable-ion
in mozconfig.txt disable JIT

So, my build has progressed, but has stopped with a "collect2: error: ld return 1 exit status"
arising, it seems, from some "undefined reference to 'ffi_call'"

I'm doing this build on a old laptop with only 2GB Ram, and a 2GB swap, so, is it possible that this is from lack of memory, or not?

Regarding libevent, I'm using the debian system libevent, so it is build for x32.

* by the way, I'm typing this now on a x32 desktop, using Midori browser, that used to crash a lot, but today, there was a massive update of packages, and it seems more stable. Very nice experience!

.
Yes, PM does use the most memory during its linking step after the compiling is done. Do you see it using the swap file during this?

But I've built it in a 3 GB amd64 virtual machine without having it touch swap, so I don't think you're running out of memory. When I've had that happen in the past, the whole machine, virtual or real, just locks up.

Once in a while when I get a build failure that mystifies me, I can find a fix by doing a websearch for the error message. Maybe there's some parameter that can be passed to the linker, but I'm just a hobby packager and don't know much of that deep stuff at all.

Re: A linux build framework for Pale Moon

Posted: Mon, 06 Mar 2017, 18:42
by JotaMG
I've give up of building Palemoon on x32 ABI.
The blocker seems to be JS, with JIT the assembler code is not ready for x32 and not compile, without JIT, lots of linker errors.
I thank stevepusser and Walter for the help.

Re: A linux build framework for Pale Moon

Posted: Mon, 06 Mar 2017, 20:15
by New Tobin Paradigm
You guys do realize you are going WAY the hell out of spec for Pale Moon builds right?

Re: A linux build framework for Pale Moon

Posted: Tue, 07 Mar 2017, 01:19
by stevepusser
Yes...but for SCIENCE!!!!! :crazy:

I like to try experiments with some packages--right now I'm trying to get the Debian ffmpeg 3.2.4 to support Intel Quick Sync hardware GPU-accelerated video transcoding with my Skylake GPU. I successfully ported over the Intel libmfx-dev library and rebuilt ffmpeg so it shows that it supports the encoder, but I get an error when I actually try it to use it, so I need to research that.