PPC64le linux build failure: error: undefined reference to 'chacha20vsx'

[o _ o]

Trying to build on ppc64le, but it seems the VSX speedups are causing a build failure.

CPU: Power8nv (supports altivec, vsx, and vsx2, but not vsx3)
OS: Gentoo Linux
CC: GCC 13.2.1

Will attach full build log, but the relevant lines seem to be:

Code: Select all

91021: 4:27.67 /var/tmp/portage/www-client/palemoon-33.1.0/work/palemoon-33.1.0/platform/security/nss/lib/freebl/chacha20poly1305.c:224: error: undefined reference to 'chacha20vsx'
91022: 4:27.67 /var/tmp/portage/www-client/palemoon-33.1.0/work/palemoon-33.1.0/platform/security/nss/lib/freebl/chacha20poly1305.c:297: error: undefined reference to 'Chacha20Poly1305_vsx_aead_encrypt'
91023: 4:27.67 /var/tmp/portage/www-client/palemoon-33.1.0/work/palemoon-33.1.0/platform/security/nss/lib/freebl/chacha20poly1305.c:366: error: undefined reference to 'Chacha20Poly1305_vsx_aead_decrypt'
91024: 4:27.67 /var/tmp/portage/www-client/palemoon-33.1.0/work/palemoon-33.1.0/platform/security/nss/lib/freebl/chacha20poly1305.c:422: error: undefined reference to 'Chacha20Poly1305_vsx_aead_encrypt'
91025: 4:27.67 /var/tmp/portage/www-client/palemoon-33.1.0/work/palemoon-33.1.0/platform/security/nss/lib/freebl/chacha20poly1305.c:476: error: undefined reference to 'Chacha20Poly1305_vsx_aead_decrypt'

Will also attach an `emerge --info`, it gives basic information about my system, available libs, etc.

Any palemoon or basilisk devs are also free to send me an SSH pubkey if they want to use this box for development or testing. Host is little endian glibc but planning to make kvm hosts for every combination of endianness, glibc/musl, and 32/64bit.

[Gaming4JC]

This is a known issue in the NSS library from upstream on older PPC64 that don't have the full VSX ABI.

Try disabling it in /platform/security/nss/coreconf/config.gypi:

Code: Select all

'disable_crypto_vsx%': 1,

See here for more info: https://bugzilla.mozilla.org/show_bug.cgi?id=1687164

[o _ o]

Hello @Gaming4JC-- thanks for the reply!!

Sadly, this issue I'm pretty sure is unrelated. While it's true that these flags (-mvsx and -mcrypto) would produce binaries that would error out on POWER6 and POWER7 respectively, they are both fully supported on my POWER8 ppc64le box, and I use both of these flags globally on my gentoo system without issue. Plus, looking at this bug doesn't show any build failures, just the bin quitting at runtime.

These VSX optimizations seemed to be introduced in palemoon 32.1.0, which I tested and encountered the same build failure.

While disabling the vsx and crypto would likely result in a successful build (and will do so after I submit this comment), that doesn't solve the issue of this vsx/crypto related code not being functional and causing a build failure. Either the offending code should be removed, or the `undefined references` should be fixed(which I'd personally much prefer)

This is a known issue in the NSS library from upstream on older PPC64 that don't have the full VSX ABI.

Try disabling it in /platform/security/nss/coreconf/config.gypi:

Code: Select all

'disable_crypto_vsx%': 1,

See here for more info: https://bugzilla.mozilla.org/show_bug.cgi?id=1687164

[Moonchild]

While disabling the vsx and crypto would likely result in a successful build (and will do so after I submit this comment), that doesn't solve the issue of this vsx/crypto related code not being functional and causing a build failure. Either the offending code should be removed, or the `undefined references` should be fixed(which I'd personally much prefer)

I'm sorry but that is not how building for multiple target platforms works. We cannot start removing code that works for other target platforms just because it doesn't work on yours. On top, this is in a third party library and we don't have control over what code goes in there (and we really don't want to fork NSS)

[o _ o]

@Gaming4J -- Doing the change you described in `platform/security/nss/coreconf/config.gypi` didn't seem to disable palemoon building with VSX, and neither did `ac_add_options --disable-optimize` in `.mozconfig`. The only thing that disabled these VSX optimizations was adding `-DNSS_DISABLE_CRYPTO_VSX` to my CFLAGS and CXXFLAGS.

Code: Select all

 0:05.90 /var/tmp/tmp/Pale-Moon/platform/security/nss/lib/freebl/chacha20poly1305.c:225: error: undefined reference to 'chacha20vsx'
 0:05.90 /var/tmp/tmp/Pale-Moon/platform/security/nss/lib/freebl/chacha20poly1305.c:305: error: undefined reference to 'Chacha20Poly1305_vsx_aead_encrypt'
 0:05.90 /var/tmp/tmp/Pale-Moon/platform/security/nss/lib/freebl/chacha20poly1305.c:378: error: undefined reference to 'Chacha20Poly1305_vsx_aead_decrypt'
 0:05.90 /var/tmp/tmp/Pale-Moon/platform/security/nss/lib/freebl/chacha20poly1305.c:454: error: undefined reference to 'Chacha20Poly1305_vsx_aead_encrypt'
 0:05.90 /var/tmp/tmp/Pale-Moon/platform/security/nss/lib/freebl/chacha20poly1305.c:527: error: undefined reference to 'Chacha20Poly1305_vsx_aead_decrypt'

Also.... considering your desire to *not* fork nss, why are you guys vendoring it by default instead of using the system installed version? Because I have nss installed on my system and it functions perfectly fine with firefox..

Also, another note, I needed to add `-Wno-implicit-function-declaration` to my *FLAGS.

Also, I encountered your altivec optimizations also causing build failures.

Code: Select all

 1:28.57 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:32:22: error: expected ',' or ';' before 'float'
 1:28.57    32 | always_inline vector float load_aligned_float(float *dataPtr)
 1:28.57       |                      ^~~~~
 1:28.57 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c: In function 'qcms_transform_data_rgb_out_lut_altivec':
 1:28.57 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:87:17: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.57    87 |         vec_r = load_aligned_float((float*)&igtbl_r[src[0]]);
 1:28.57       |                 ^~~~~~~~~~~~~~~~~~
 1:28.57 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:88:17: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.57    88 |         vec_g = load_aligned_float((float*)&igtbl_r[src[1]]);
 1:28.57       |                 ^~~~~~~~~~~~~~~~~~
 1:28.57 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:89:17: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.57    89 |         vec_b = load_aligned_float((float*)&igtbl_r[src[2]]);
 1:28.57       |                 ^~~~~~~~~~~~~~~~~~
 1:28.58 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:116:25: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.58   116 |                 vec_r = load_aligned_float((float*)&igtbl_r[src[0]]);
 1:28.58       |                         ^~~~~~~~~~~~~~~~~~
 1:28.58 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:117:25: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.58   117 |                 vec_g = load_aligned_float((float*)&igtbl_r[src[1]]);
 1:28.58       |                         ^~~~~~~~~~~~~~~~~~
 1:28.58 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:118:25: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.58   118 |                 vec_b = load_aligned_float((float*)&igtbl_r[src[2]]);
 1:28.58       |                         ^~~~~~~~~~~~~~~~~~
 1:28.58 Warning: -Wunused-variable in /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c: unused variable 'igtbl_b'
 1:28.58 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:59:22: warning: unused variable 'igtbl_b' [-Wunused-variable]
 1:28.58    59 |         const float *igtbl_b = transform->input_gamma_table_b;
 1:28.58       |                      ^~~~~~~
 1:28.58 Warning: -Wunused-variable in /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c: unused variable 'igtbl_g'
 1:28.58 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:58:22: warning: unused variable 'igtbl_g' [-Wunused-variable]
 1:28.58    58 |         const float *igtbl_g = transform->input_gamma_table_g;
 1:28.58       |                      ^~~~~~~
 1:28.59 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c: In function 'qcms_transform_data_rgba_out_lut_altivec':
 1:28.59 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:199:17: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.59   199 |         vec_r = load_aligned_float((float*)&igtbl_r[src[0]]);
 1:28.59       |                 ^~~~~~~~~~~~~~~~~~
 1:28.59 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:200:17: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.59   200 |         vec_g = load_aligned_float((float*)&igtbl_r[src[1]]);
 1:28.59       |                 ^~~~~~~~~~~~~~~~~~
 1:28.59 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:201:17: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.59   201 |         vec_b = load_aligned_float((float*)&igtbl_r[src[2]]);
 1:28.59       |                 ^~~~~~~~~~~~~~~~~~
 1:28.60 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:233:25: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.60   233 |                 vec_r = load_aligned_float((float*)&igtbl_r[src[0]]);
 1:28.60       |                         ^~~~~~~~~~~~~~~~~~
 1:28.60 /var/tmp/tmp/Pale-Moon/platform/gfx/qcms/transform-altivec.c:234:25: error: incompatible types when assigning to type '__vector float' {aka '__vector(4) float'} from type 'int'
 1:28.60   234 |                 vec_g = load_aligned_float((float*)&igtbl_r[src[1]]);
 1:28.60       |                         ^~~~~~~~~~~~~~~~~~
 

Because, from all my searching, I could not find any flags or vars to actually disable this altivec code, I had to modify the code in a few places to bypass it. `platform/gfx/qcms/transform.c`, commented out the section starting at 1302, and also in `platform/gfx/qcms/moz.build` at line 37.

After these vsx/altivec optimizations have been disabled, palemoon builds successfully. (although it does segfault when running, so that'll be a fun time. same segfault i got in 32.0.1, will debug).

It would be extremely swell if these PPC vector optimizations could have their build issues fixed, but if not, since as they are right now they're non-functional, this non working code would be better disabled/removed.

[o _ o]

While disabling the vsx and crypto would likely result in a successful build (and will do so after I submit this comment), that doesn't solve the issue of this vsx/crypto related code not being functional and causing a build failure. Either the offending code should be removed, or the `undefined references` should be fixed(which I'd personally much prefer)

I'm sorry but that is not how building for multiple target platforms works. We cannot start removing code that works for other target platforms just because it doesn't work on yours. On top, this is in a third party library and we don't have control over what code goes in there (and we really don't want to fork NSS)

To clarify, VSX (and altivec) are vector instructions (similar to SSE2 on intel), only present on PPC processors. This code, the vsx/altivec optimizations, A) do not work for other target platforms as they are PPC specific and B) also do not function on PPC. If you don't want to fix the code, fair enough, but considering how the code doesn't work, shouldn't it be removed/disabled? (which I outlined in the post I just made on how I did that)

[o _ o]

Code: Select all

diff --git a/og.c b/platform/gfx/qcms/transform-altivec.c
index 9dc17bc13f..df896db56c 100644
--- a/og.c
+++ b/platform/gfx/qcms/transform-altivec.c
@@ -29,7 +29,7 @@
 static const ALIGN float floatScaleX4 = FLOATSCALE;
 static const ALIGN float clampMaxValueX4 = CLAMPMAXVAL;
 
-inline vector float load_aligned_float(float *dataPtr)
+static inline vector float load_aligned_float(float *dataPtr)
 {
 	vector float data = vec_lde(0, dataPtr);
 	vector unsigned char moveToStart = vec_lvsl(0, dataPtr);
 

This patch fixes the build failure in the altivec code inside palemoon. I know super simple, this is why I shouldn't do these things at 3am.

[Moonchild]

Also.... considering your desire to *not* fork nss, why are you guys vendoring it by default instead of using the system installed version? Because I have nss installed on my system and it functions perfectly fine with firefox..

Mozilla maintains NSS - they are changing it in tandem with Firefox, so yes, it will always work if you have "latest+latest" but NSS regularly makes breaking changes that do NOT play nice with our code (since we aren't Firefox) and it's a perfect example of how we cannot use system-installed libs for critical components. For NSS it would effectively force us to deal with every "live" version that might be out in the wild as "latest" depending on OS/distro. Also, Windows doesn't have a "system installed NSS" so we need to vendor it there anyway. No idea what the Mac or BSD situation might be. So we use known-good snapshots instead, and vendor those.

For your other issue:
gfx/qcms is third-party code which we have not touched for a long time, and we have not tested PPC building because we don't really target it (although our tree obviously does support building for it) and have not kept up with advances/changes on that architecture. If it needs patches specific for PPC, then feel free to help out by offering a pull request.

[Moonchild]

Patch merged for qcms.

[dbsoft]

Interestingly I had that fix in my code for Mac PPC... just haven't had time to fix the crashes so I hadn't pulled it yet.

https://repo.palemoon.org/dbsoft/UXP/co ... 4017173c66