Is libtremor still needed in the platform? Topic is solved

Discussions about the development and maturation of the platform code (UXP).
Warning: may contain highly-technical topics.

Moderators: trava90, athenian200

User avatar
jobbautista9
Keeps coming back
Keeps coming back
Posts: 780
Joined: 2020-11-03, 06:47
Location: Philippines
Contact:

Is libtremor still needed in the platform?

Unread post by jobbautista9 » 2022-07-08, 14:10

I was looking at the source tree, and noticed this library called tremor. Apparently it's another decoder for Ogg Vorbis, and if the README_MOZILLA is to be believed, it's severely outdated (both we and Mozilla use svn r17547, while upstream has long moved to git). It doesn't seem to be used for anything else than decoding Vorbis audio, and we already use libvorbis for that. Should we remove this library and cleanup the unused MOZ_TREMOR variables in our configure scripts, or am I missing something (like "libvpx being used for WebRTC" stuff)?
Image

merry mimas

XUL add-ons developer. You can find a list of add-ons I manage at http://rw.rs/~job/software.html.

Mima avatar by 絵虎. Pixiv post: https://www.pixiv.net/en/artworks/15431817

Image

User avatar
trava90
Contributing developer
Contributing developer
Posts: 1727
Joined: 2013-05-20, 18:19
Location: Somewhere in Sector 001

Re: Is libtremor still needed in the platform?

Unread post by trava90 » 2022-07-08, 14:25

Tremor was apparently added by Mozilla over a decade ago for use on ARM and other slower fixed-point platforms. With that in mind it's likely not something that is needed for our purposes (though admittedly I have not looked at it in-depth).

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35404
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Is libtremor still needed in the platform?

Unread post by Moonchild » 2022-07-08, 14:51

Keep in mind that as far as I'm aware people do build for slower ARM devices like Raspbian etc. where this would likely be in use?
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
trava90
Contributing developer
Contributing developer
Posts: 1727
Joined: 2013-05-20, 18:19
Location: Somewhere in Sector 001

Re: Is libtremor still needed in the platform?

Unread post by trava90 » 2022-07-08, 15:13

Very good point. I didn't even think about slower non-Android ARM devices. It is possible that Tremor would be used on those platforms.

User avatar
jobbautista9
Keeps coming back
Keeps coming back
Posts: 780
Joined: 2020-11-03, 06:47
Location: Philippines
Contact:

Re: Is libtremor still needed in the platform?

Unread post by jobbautista9 » 2022-07-16, 05:26

In that case we should probably keep the tremor library up-to-date then. Getting the source updated is easy, but how do I enable tremor (which is disabled by default) in build-time? There doesn't seem to be any configure option like --enable-tremor, only variables named MOZ_TREMOR and MOZ_VORBIS. And the platform currently doesn't support enabling both. Do I export MOZ_TREMOR and unset MOZ_VORBIS in .mozconfig?

I might research into whether it's possible to enable both and switch between libraries in runtime, just like how Mozilla does for AV1 with libaom and dav1d (and I will do that too once I manage to get the platform decoder for dav1d working somehow, I unfortunately don't have the C++ skills to write one from scratch. The platform decoder literally is the only thing preventing me from getting dav1d to the platform)
Image

merry mimas

XUL add-ons developer. You can find a list of add-ons I manage at http://rw.rs/~job/software.html.

Mima avatar by 絵虎. Pixiv post: https://www.pixiv.net/en/artworks/15431817

Image

User avatar
jobbautista9
Keeps coming back
Keeps coming back
Posts: 780
Joined: 2020-11-03, 06:47
Location: Philippines
Contact:

Re: Is libtremor still needed in the platform?

Unread post by jobbautista9 » 2022-07-19, 14:16

I've figured out how to make it build tremor instead of the vorbis reference library; I just had to remove the ARM check at line 2689 of old-configure.in. I don't notice any performance penalty using tremor for playing Vorbis audio, and compared to the mainline build which uses libvorbis, my libtremor build seems to use less CPU than mainline while playing Vorbis files. Maybe it's due to the fact that the libtremor build uses signed 16-bit samples instead of 32-bit floating point ones? I wonder if we could get away with just using libtremor for every platform and ditch libvorbis. We just have to get rid of the "MOZ_SAMPLE_TYPE_S16" and "MOZ_SAMPLE_TYPE_FLOAT32" checks as seen in this xref query.
trava90 wrote:
2022-07-08, 15:13
Very good point. I didn't even think about slower non-Android ARM devices. It is possible that Tremor would be used on those platforms.
It's not only possible, but according to line 2689 of old-configure.in, tremor will always be used for ARM:

Code: Select all

dnl Use integers over floats for audio on ARM
dnl because it's more efficient.
if test "$CPU_ARCH" = "arm"; then
    MOZ_SAMPLE_TYPE_S16=1
    AC_DEFINE(MOZ_SAMPLE_TYPE_S16)
    AC_SUBST(MOZ_SAMPLE_TYPE_S16)
else
    MOZ_SAMPLE_TYPE_FLOAT32=1
    AC_DEFINE(MOZ_SAMPLE_TYPE_FLOAT32)
    AC_SUBST(MOZ_SAMPLE_TYPE_FLOAT32)
fi
So yeah, ARM Mac is most likely if not definitely using tremor, and dbsoft may not even be aware of it.
Image

merry mimas

XUL add-ons developer. You can find a list of add-ons I manage at http://rw.rs/~job/software.html.

Mima avatar by 絵虎. Pixiv post: https://www.pixiv.net/en/artworks/15431817

Image

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35404
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Is libtremor still needed in the platform?

Unread post by Moonchild » 2022-07-19, 15:36

jobbautista9 wrote:
2022-07-19, 14:16
Maybe it's due to the fact that the libtremor build uses signed 16-bit samples instead of 32-bit floating point ones? I wonder if we could get away with just using libtremor for every platform and ditch libvorbis.
That all depends on how important it is for sound reproduction. I'm sure there's a good reason why libvorbis uses real numbers for sample values instead of integers. I'm not intimately familiar with the compression and encoding scheme but I'd rather not compromise for a lowest common denominator if there's an actual quality difference. I know for a fact that Ogg files were always inferior in sound, so we may be sacrificing something important if we don't use floats in the algorithm.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
random
Apollo supporter
Apollo supporter
Posts: 33
Joined: 2022-07-22, 06:31

Re: Is libtremor still needed in the platform?

Unread post by random » 2022-07-22, 11:00

Tremor always decodes to 16 bit sample, but I doubt that one can really hear a difference between s16 and f32 output. See
https://people.xiph.org/~xiphmont/demo/neil-young.html

AFAIK Tremor uses 32 bit integer math for decoding. The most likely reason for subpar decoding quality would be libtremor compiled with the _LOW_ACCURACY_ flag.
Moonchild wrote:
2022-07-19, 15:36
I know for a fact that Ogg files were always inferior in sound
Inferior to what?
Btw, Ogg is the container format, which could carry Speex, Vorbis, Opus, Flac and video.
so we may be sacrificing something important if we don't use floats in the algorithm.
Do we know what this something important is?

Both decoders use 32-bit, one uses fixed-point math, the other floating point math. Which is better? I guess it depends how well it is implemented. Their might be advantages in using fixed-point:

If the range of the values to be represented is known in advance and is sufficiently limited, fixed point can make better use of the available bits. For example, if 32 bits are available to represent a number between 0 and 1, a fixed-point representation can have error less than 1.2 × 10−10, whereas the standard floating-point representation may have error up to 596 × 10−10 — because 9 of the bits are wasted with the sign and exponent of the dynamic scaling factor. Specifically, comparing 32-bit fixed-point to floating-point audio, a recording requiring less than 40 dB of headroom has a higher signal-to-noise ratio using 32-bit fixed.
from https://en.wikipedia.org/wiki/Fixed-poi ... ting-point

That said, Tremor was mainly created for CPUs that don't support floating point or are slow with floats. Usually these are not devices that are used for surfing the contemporary web, but rather old embedded and mobile devices. A machine for a desktop browser will most likely have enough CPU power (and float support) to use libvorbis without a sweat.

Is there any version of Palemoon build with Tremor at all?

I'm using Palemoon on a Raspberry Pi 2 and this version (from stevenpusser) doesn't use libvorbisidec1 (tremor).

User avatar
athenian200
Contributing developer
Contributing developer
Posts: 1481
Joined: 2018-10-28, 19:56
Location: Georgia

Re: Is libtremor still needed in the platform?

Unread post by athenian200 » 2022-07-22, 13:57

random wrote:
2022-07-22, 11:00
I'm using Palemoon on a Raspberry Pi 2 and this version (from stevenpusser) doesn't use libvorbisidec1 (tremor).
Is it a Version 1.1 with the Cortex A7 or the newer one with the A53?

The main device we'd be worried about is the older 32-bit Cortex A7 used in the original Raspberry Pi 2. It's right on the edge of what we would be supporting as far as ARM. So ultimately the question comes down to whether the floating point performance on the Cortex A7 is bad enough to justify using libtremor or not. If The Cortex A7 has decent floating point performance, then this may well be an obsolete consideration for even older ARM processors than we were thinking.
"The Athenians, however, represent the unity of these opposites; in them, mind or spirit has emerged from the Theban subjectivity without losing itself in the Spartan objectivity of ethical life. With the Athenians, the rights of the State and of the individual found as perfect a union as was possible at all at the level of the Greek spirit." -- Hegel's philosophy of Mind

dbsoft
Project Contributor
Project Contributor
Posts: 405
Joined: 2020-02-21, 17:35

Re: Is libtremor still needed in the platform?

Unread post by dbsoft » 2022-07-23, 08:08

From what I read the other day when researching this, the 32-bit ARM specification does not have ANY floating point. Some later 32-bit or hybrid 32/64-bit models have floating point extensions, but the ABI does not support floating point at all. So if anyone plans on using 32-bit ARM support need to leave in libtremor.

User avatar
random
Apollo supporter
Apollo supporter
Posts: 33
Joined: 2022-07-22, 06:31

Re: Is libtremor still needed in the platform?

Unread post by random » 2022-07-23, 19:49

My Pi 2 is the 32-bit Cortex A7 version:

Code: Select all

$ cat /proc/cpuinfo 
processor       : 0
model name      : ARMv7 Processor rev 5 (v7l)
BogoMIPS        : 38.40
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm 
Even the Pi 1 (ARMv6) does support floating point (armhf architecture = arm hardware floating-point)

https://en.wikipedia.org/wiki/List_of_A ... hitectures

ffmpeg decodes an ogg/vorbis stereo file 30x faster than real time on a single CPU core (including read and write from the sd-card). When playing the same file in Palemoon htop displays around 40% overall CPU usage (4 cores = 400%). I guess most of it is not from the Vorbis decoder.

I compiled the example cli decoder from Tremor. It is significantly faster on ARMv7 than the floating point decoder, but is the difference big enough and relevant to keep libtremor?

libtremor:

Code: Select all

$ time cat test.ogg | ./ivorbisfile_example > /dev/null
[...]
real    0m6,825s
user    0m6,779s
sys     0m0,086s
libvorbis:

Code: Select all

time ffmpeg -i test.ogg -f wav - > /dev/null
[...]
real    0m10,662s
user    0m10,469s
sys     0m0,162s

Code: Select all

$ time oggdec test.ogg -o - > /dev/null
[...]
real    0m10,727s
user    0m9,806s
sys     0m0,920s
I have no idea if libtremor is faster or slower than libvorbis on ARMv8 (aarch64) or amd64.

User avatar
athenian200
Contributing developer
Contributing developer
Posts: 1481
Joined: 2018-10-28, 19:56
Location: Georgia

Re: Is libtremor still needed in the platform?

Unread post by athenian200 » 2022-07-24, 05:58

random wrote:
2022-07-23, 19:49
My Pi 2 is the 32-bit Cortex A7 version:

Code: Select all

$ cat /proc/cpuinfo 
processor       : 0
model name      : ARMv7 Processor rev 5 (v7l)
BogoMIPS        : 38.40
Features        : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm 
Even the Pi 1 (ARMv6) does support floating point (armhf architecture = arm hardware floating-point)
That's very interesting. I looked into this a little more, and apparently it's not as cut and dried as I thought. There are two APIs for 32-bit ARM... armel, and armhf (and it seems like the A7 and later are usually armhf). Apparently even the Raspberry Pi 1 used arm6hf, and thus it was never built without an FPU. I'm thinking it's sort of like the difference between the 486SX and the 486DX. That is, there are cheaper models that have no hardware floating point, and pricer models that include it, but the API itself doesn't guarantee you will have an FPU in 32-bit ARM, whereas it's baked into the spec with aarch64.
I compiled the example cli decoder from Tremor. It is significantly faster on ARMv7 than the floating point decoder, but is the difference big enough and relevant to keep libtremor?

libtremor:

Code: Select all

$ time cat test.ogg | ./ivorbisfile_example > /dev/null
[...]
real    0m6,825s
user    0m6,779s
sys     0m0,086s
libvorbis:

Code: Select all

time ffmpeg -i test.ogg -f wav - > /dev/null
[...]
real    0m10,662s
user    0m10,469s
sys     0m0,162s
Well, it looks fairly big... it looks like it takes half the time (though admittedly I don't know the units). Then again, aren't most CPU architectures a lot faster at doing non-floating point math? How smooth is libvorbis decoding on your RPi 2? It seems like it's not causing you any substantial problems from what you're saying.
"The Athenians, however, represent the unity of these opposites; in them, mind or spirit has emerged from the Theban subjectivity without losing itself in the Spartan objectivity of ethical life. With the Athenians, the rights of the State and of the individual found as perfect a union as was possible at all at the level of the Greek spirit." -- Hegel's philosophy of Mind

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35404
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Is libtremor still needed in the platform?

Unread post by Moonchild » 2022-07-24, 10:24

random wrote:
2022-07-23, 19:49
ffmpeg decodes an ogg/vorbis stereo file 30x faster than real time on a single CPU core
That's not really relevant here because we aren't a dedicated multimedia application that can focus tightly on maximum performance for one particular task.
Same for a decoding CLI application, you cannot compare that kind of use with use inside a multimedia&internet enabled application.
random wrote:
2022-07-23, 19:49
When playing the same file in Palemoon htop displays around 40% overall CPU usage
Using half of one core is absolutely significant, and not the kind of load I'd like to see with just singular playback. What is the CPU usage when playing the same file when building and using libtremor, instead? That would give us some real comparative numbers here.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
random
Apollo supporter
Apollo supporter
Posts: 33
Joined: 2022-07-22, 06:31

Re: Is libtremor still needed in the platform?

Unread post by random » 2022-07-24, 12:19

athenian200 wrote:
2022-07-24, 05:58
How smooth is libvorbis decoding on your RPi 2?
Smooth enough, if I use a media player like VLC. It's around 5% CPU usage and more or less the same than mp3 or aac decoding (only slightly slower).
It seems like it's not causing you any substantial problems from what you're saying.
I tried Ogg/Vorbis playback in Palemoon (31.1.0) again and now I'm getting around 110% CPU usage (no active websites in other tabs). Again, mp3 and aac looks very much the same (in htop). Weirdly I only get white-noisy distortions with mp3 and aac files on the Pi. You can barely recognize the music in the background. Sound to me like playing the PCM data in the wrong format (signed/unsigned, big/little endian, float/integer). Both files play fine on Palemoon x86 (31.0.0) and it uses much fewer resources (15%, some old slow dual core Intel CPU).

So maybe tremor vs libvorbis is not the biggest issue.

User avatar
random
Apollo supporter
Apollo supporter
Posts: 33
Joined: 2022-07-22, 06:31

Re: Is libtremor still needed in the platform?

Unread post by random » 2022-07-24, 12:39

Moonchild wrote:
2022-07-24, 10:24
random wrote:
2022-07-23, 19:49
ffmpeg decodes an ogg/vorbis stereo file 30x faster than real time on a single CPU core
That's not really relevant here because we aren't a dedicated multimedia application that can focus tightly on maximum performance for one particular task.
Same for a decoding CLI application, you cannot compare that kind of use with use inside a multimedia&internet enabled application.
Is it not? I didn't want to compare the performance of Palemoon with a CLI / media player. It was more about the baseline and the difference in performance of the raw decoding task between tremor and libvorbis.
random wrote:
2022-07-23, 19:49
What is the CPU usage when playing the same file when building and using libtremor, instead? That would give us some real comparative numbers here.
If someone builds Palemoon with libtremor, I'm happy to test it.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35404
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Is libtremor still needed in the platform?

Unread post by Moonchild » 2022-07-24, 13:31

random wrote:
2022-07-24, 12:39
It was more about the baseline and the difference in performance of the raw decoding task between tremor and libvorbis.
you can't make a baseline if you don't compare in the same environment. if you want to make a baseline comparison you have to keep everything except the lib the same, otherwise you're unable to compare anything.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

dbsoft
Project Contributor
Project Contributor
Posts: 405
Joined: 2020-02-21, 17:35

Re: Is libtremor still needed in the platform?

Unread post by dbsoft » 2022-07-24, 21:25

random wrote:
2022-07-23, 19:49
Even the Pi 1 (ARMv6) does support floating point (armhf architecture = arm hardware floating-point)
The problem is the binaries can potentially be run on a platform that does not have floating point, since the 32-bit ARM specification does not guarantee floating point... it could be run on a model that has no FPU. 64-bit ARM floating point is part of the specification, so there are no 64-bit ARM systems without an FPU.

User avatar
jobbautista9
Keeps coming back
Keeps coming back
Posts: 780
Joined: 2020-11-03, 06:47
Location: Philippines
Contact:

Re: Is libtremor still needed in the platform?

Unread post by jobbautista9 » 2022-07-25, 02:34

Thanks everyone especially athenian200 and dbsoft for your input. I think I'm fully convinced now that the tremor library should stay.
Image

merry mimas

XUL add-ons developer. You can find a list of add-ons I manage at http://rw.rs/~job/software.html.

Mima avatar by 絵虎. Pixiv post: https://www.pixiv.net/en/artworks/15431817

Image

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35404
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Is libtremor still needed in the platform?

Unread post by Moonchild » 2022-07-25, 10:07

jobbautista9 wrote:
2022-07-25, 02:34
I think I'm fully convinced now that the tremor library should stay.
At least until 32-bit ARM really isn't a thing any more aside from museum pieces :)
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
random
Apollo supporter
Apollo supporter
Posts: 33
Joined: 2022-07-22, 06:31

Re: Is libtremor still needed in the platform?

Unread post by random » 2022-07-25, 12:21

dbsoft wrote:
2022-07-24, 21:25
The problem is the binaries can potentially be run on a platform that does not have floating point, since the 32-bit ARM specification does not guarantee floating point... it could be run on a model that has no FPU. 64-bit ARM floating point is part of the specification, so there are no 64-bit ARM systems without an FPU.
potentially, theoretically, but not really.

Do you know any computer without hardware floating point support that could run Palemoon reasonably well? I bet there is none.

Is there any Palemoon binary for download that is compiled with tremor?

What about much more popular codecs like mp3 or aac? Are they floating point free?

What are with other parts of Palemoon. Is everything integer math or would the potential desktop machine with floating point free CPU have to emulate floating point instructions?

Which OS and desktop environment would the theoretical floating point disabled computer run?

Locked