Is libtremor still needed in the platform? Topic is solved
Moderators: trava90, athenian200
- jobbautista9
- Keeps coming back
- Posts: 782
- Joined: 2020-11-03, 06:47
- Location: Philippines
- Contact:
Is libtremor still needed in the platform?
I was looking at the source tree, and noticed this library called tremor. Apparently it's another decoder for Ogg Vorbis, and if the README_MOZILLA is to be believed, it's severely outdated (both we and Mozilla use svn r17547, while upstream has long moved to git). It doesn't seem to be used for anything else than decoding Vorbis audio, and we already use libvorbis for that. Should we remove this library and cleanup the unused MOZ_TREMOR variables in our configure scripts, or am I missing something (like "libvpx being used for WebRTC" stuff)?
merry mimas
XUL add-ons developer. You can find a list of add-ons I manage at http://rw.rs/~job/software.html.
Mima avatar by 絵虎. Pixiv post: https://www.pixiv.net/en/artworks/15431817
- trava90
- Contributing developer
- Posts: 1736
- Joined: 2013-05-20, 18:19
- Location: Somewhere in Sector 001
Re: Is libtremor still needed in the platform?
Tremor was apparently added by Mozilla over a decade ago for use on ARM and other slower fixed-point platforms. With that in mind it's likely not something that is needed for our purposes (though admittedly I have not looked at it in-depth).
Re: Is libtremor still needed in the platform?
Keep in mind that as far as I'm aware people do build for slower ARM devices like Raspbian etc. where this would likely be in use?
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
- trava90
- Contributing developer
- Posts: 1736
- Joined: 2013-05-20, 18:19
- Location: Somewhere in Sector 001
Re: Is libtremor still needed in the platform?
Very good point. I didn't even think about slower non-Android ARM devices. It is possible that Tremor would be used on those platforms.
- jobbautista9
- Keeps coming back
- Posts: 782
- Joined: 2020-11-03, 06:47
- Location: Philippines
- Contact:
Re: Is libtremor still needed in the platform?
In that case we should probably keep the tremor library up-to-date then. Getting the source updated is easy, but how do I enable tremor (which is disabled by default) in build-time? There doesn't seem to be any configure option like --enable-tremor, only variables named MOZ_TREMOR and MOZ_VORBIS. And the platform currently doesn't support enabling both. Do I export MOZ_TREMOR and unset MOZ_VORBIS in .mozconfig?
I might research into whether it's possible to enable both and switch between libraries in runtime, just like how Mozilla does for AV1 with libaom and dav1d (and I will do that too once I manage to get the platform decoder for dav1d working somehow, I unfortunately don't have the C++ skills to write one from scratch. The platform decoder literally is the only thing preventing me from getting dav1d to the platform)
I might research into whether it's possible to enable both and switch between libraries in runtime, just like how Mozilla does for AV1 with libaom and dav1d (and I will do that too once I manage to get the platform decoder for dav1d working somehow, I unfortunately don't have the C++ skills to write one from scratch. The platform decoder literally is the only thing preventing me from getting dav1d to the platform)
merry mimas
XUL add-ons developer. You can find a list of add-ons I manage at http://rw.rs/~job/software.html.
Mima avatar by 絵虎. Pixiv post: https://www.pixiv.net/en/artworks/15431817
- jobbautista9
- Keeps coming back
- Posts: 782
- Joined: 2020-11-03, 06:47
- Location: Philippines
- Contact:
Re: Is libtremor still needed in the platform?
I've figured out how to make it build tremor instead of the vorbis reference library; I just had to remove the ARM check at line 2689 of old-configure.in. I don't notice any performance penalty using tremor for playing Vorbis audio, and compared to the mainline build which uses libvorbis, my libtremor build seems to use less CPU than mainline while playing Vorbis files. Maybe it's due to the fact that the libtremor build uses signed 16-bit samples instead of 32-bit floating point ones? I wonder if we could get away with just using libtremor for every platform and ditch libvorbis. We just have to get rid of the "MOZ_SAMPLE_TYPE_S16" and "MOZ_SAMPLE_TYPE_FLOAT32" checks as seen in this xref query.
So yeah, ARM Mac is most likely if not definitely using tremor, and dbsoft may not even be aware of it.
It's not only possible, but according to line 2689 of old-configure.in, tremor will always be used for ARM:
Code: Select all
dnl Use integers over floats for audio on ARM
dnl because it's more efficient.
if test "$CPU_ARCH" = "arm"; then
MOZ_SAMPLE_TYPE_S16=1
AC_DEFINE(MOZ_SAMPLE_TYPE_S16)
AC_SUBST(MOZ_SAMPLE_TYPE_S16)
else
MOZ_SAMPLE_TYPE_FLOAT32=1
AC_DEFINE(MOZ_SAMPLE_TYPE_FLOAT32)
AC_SUBST(MOZ_SAMPLE_TYPE_FLOAT32)
fi
merry mimas
XUL add-ons developer. You can find a list of add-ons I manage at http://rw.rs/~job/software.html.
Mima avatar by 絵虎. Pixiv post: https://www.pixiv.net/en/artworks/15431817
Re: Is libtremor still needed in the platform?
That all depends on how important it is for sound reproduction. I'm sure there's a good reason why libvorbis uses real numbers for sample values instead of integers. I'm not intimately familiar with the compression and encoding scheme but I'd rather not compromise for a lowest common denominator if there's an actual quality difference. I know for a fact that Ogg files were always inferior in sound, so we may be sacrificing something important if we don't use floats in the algorithm.jobbautista9 wrote: ↑2022-07-19, 14:16Maybe it's due to the fact that the libtremor build uses signed 16-bit samples instead of 32-bit floating point ones? I wonder if we could get away with just using libtremor for every platform and ditch libvorbis.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
Re: Is libtremor still needed in the platform?
Tremor always decodes to 16 bit sample, but I doubt that one can really hear a difference between s16 and f32 output. See
https://people.xiph.org/~xiphmont/demo/neil-young.html
AFAIK Tremor uses 32 bit integer math for decoding. The most likely reason for subpar decoding quality would be libtremor compiled with the _LOW_ACCURACY_ flag.
Btw, Ogg is the container format, which could carry Speex, Vorbis, Opus, Flac and video.
Both decoders use 32-bit, one uses fixed-point math, the other floating point math. Which is better? I guess it depends how well it is implemented. Their might be advantages in using fixed-point:
If the range of the values to be represented is known in advance and is sufficiently limited, fixed point can make better use of the available bits. For example, if 32 bits are available to represent a number between 0 and 1, a fixed-point representation can have error less than 1.2 × 10−10, whereas the standard floating-point representation may have error up to 596 × 10−10 — because 9 of the bits are wasted with the sign and exponent of the dynamic scaling factor. Specifically, comparing 32-bit fixed-point to floating-point audio, a recording requiring less than 40 dB of headroom has a higher signal-to-noise ratio using 32-bit fixed.
from https://en.wikipedia.org/wiki/Fixed-poi ... ting-point
That said, Tremor was mainly created for CPUs that don't support floating point or are slow with floats. Usually these are not devices that are used for surfing the contemporary web, but rather old embedded and mobile devices. A machine for a desktop browser will most likely have enough CPU power (and float support) to use libvorbis without a sweat.
Is there any version of Palemoon build with Tremor at all?
I'm using Palemoon on a Raspberry Pi 2 and this version (from stevenpusser) doesn't use libvorbisidec1 (tremor).
https://people.xiph.org/~xiphmont/demo/neil-young.html
AFAIK Tremor uses 32 bit integer math for decoding. The most likely reason for subpar decoding quality would be libtremor compiled with the _LOW_ACCURACY_ flag.
Inferior to what?
Btw, Ogg is the container format, which could carry Speex, Vorbis, Opus, Flac and video.
Do we know what this something important is?so we may be sacrificing something important if we don't use floats in the algorithm.
Both decoders use 32-bit, one uses fixed-point math, the other floating point math. Which is better? I guess it depends how well it is implemented. Their might be advantages in using fixed-point:
If the range of the values to be represented is known in advance and is sufficiently limited, fixed point can make better use of the available bits. For example, if 32 bits are available to represent a number between 0 and 1, a fixed-point representation can have error less than 1.2 × 10−10, whereas the standard floating-point representation may have error up to 596 × 10−10 — because 9 of the bits are wasted with the sign and exponent of the dynamic scaling factor. Specifically, comparing 32-bit fixed-point to floating-point audio, a recording requiring less than 40 dB of headroom has a higher signal-to-noise ratio using 32-bit fixed.
from https://en.wikipedia.org/wiki/Fixed-poi ... ting-point
That said, Tremor was mainly created for CPUs that don't support floating point or are slow with floats. Usually these are not devices that are used for surfing the contemporary web, but rather old embedded and mobile devices. A machine for a desktop browser will most likely have enough CPU power (and float support) to use libvorbis without a sweat.
Is there any version of Palemoon build with Tremor at all?
I'm using Palemoon on a Raspberry Pi 2 and this version (from stevenpusser) doesn't use libvorbisidec1 (tremor).
- athenian200
- Contributing developer
- Posts: 1498
- Joined: 2018-10-28, 19:56
- Location: Georgia
Re: Is libtremor still needed in the platform?
Is it a Version 1.1 with the Cortex A7 or the newer one with the A53?
The main device we'd be worried about is the older 32-bit Cortex A7 used in the original Raspberry Pi 2. It's right on the edge of what we would be supporting as far as ARM. So ultimately the question comes down to whether the floating point performance on the Cortex A7 is bad enough to justify using libtremor or not. If The Cortex A7 has decent floating point performance, then this may well be an obsolete consideration for even older ARM processors than we were thinking.
"The Athenians, however, represent the unity of these opposites; in them, mind or spirit has emerged from the Theban subjectivity without losing itself in the Spartan objectivity of ethical life. With the Athenians, the rights of the State and of the individual found as perfect a union as was possible at all at the level of the Greek spirit." -- Hegel's philosophy of Mind
Re: Is libtremor still needed in the platform?
From what I read the other day when researching this, the 32-bit ARM specification does not have ANY floating point. Some later 32-bit or hybrid 32/64-bit models have floating point extensions, but the ABI does not support floating point at all. So if anyone plans on using 32-bit ARM support need to leave in libtremor.
Re: Is libtremor still needed in the platform?
My Pi 2 is the 32-bit Cortex A7 version:
Even the Pi 1 (ARMv6) does support floating point (armhf architecture = arm hardware floating-point)
https://en.wikipedia.org/wiki/List_of_A ... hitectures
ffmpeg decodes an ogg/vorbis stereo file 30x faster than real time on a single CPU core (including read and write from the sd-card). When playing the same file in Palemoon htop displays around 40% overall CPU usage (4 cores = 400%). I guess most of it is not from the Vorbis decoder.
I compiled the example cli decoder from Tremor. It is significantly faster on ARMv7 than the floating point decoder, but is the difference big enough and relevant to keep libtremor?
libtremor:
libvorbis:
I have no idea if libtremor is faster or slower than libvorbis on ARMv8 (aarch64) or amd64.
Code: Select all
$ cat /proc/cpuinfo
processor : 0
model name : ARMv7 Processor rev 5 (v7l)
BogoMIPS : 38.40
Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm
https://en.wikipedia.org/wiki/List_of_A ... hitectures
ffmpeg decodes an ogg/vorbis stereo file 30x faster than real time on a single CPU core (including read and write from the sd-card). When playing the same file in Palemoon htop displays around 40% overall CPU usage (4 cores = 400%). I guess most of it is not from the Vorbis decoder.
I compiled the example cli decoder from Tremor. It is significantly faster on ARMv7 than the floating point decoder, but is the difference big enough and relevant to keep libtremor?
libtremor:
Code: Select all
$ time cat test.ogg | ./ivorbisfile_example > /dev/null
[...]
real 0m6,825s
user 0m6,779s
sys 0m0,086s
Code: Select all
time ffmpeg -i test.ogg -f wav - > /dev/null
[...]
real 0m10,662s
user 0m10,469s
sys 0m0,162s
Code: Select all
$ time oggdec test.ogg -o - > /dev/null
[...]
real 0m10,727s
user 0m9,806s
sys 0m0,920s
- athenian200
- Contributing developer
- Posts: 1498
- Joined: 2018-10-28, 19:56
- Location: Georgia
Re: Is libtremor still needed in the platform?
That's very interesting. I looked into this a little more, and apparently it's not as cut and dried as I thought. There are two APIs for 32-bit ARM... armel, and armhf (and it seems like the A7 and later are usually armhf). Apparently even the Raspberry Pi 1 used arm6hf, and thus it was never built without an FPU. I'm thinking it's sort of like the difference between the 486SX and the 486DX. That is, there are cheaper models that have no hardware floating point, and pricer models that include it, but the API itself doesn't guarantee you will have an FPU in 32-bit ARM, whereas it's baked into the spec with aarch64.random wrote: ↑2022-07-23, 19:49My Pi 2 is the 32-bit Cortex A7 version:Even the Pi 1 (ARMv6) does support floating point (armhf architecture = arm hardware floating-point)Code: Select all
$ cat /proc/cpuinfo processor : 0 model name : ARMv7 Processor rev 5 (v7l) BogoMIPS : 38.40 Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm
Well, it looks fairly big... it looks like it takes half the time (though admittedly I don't know the units). Then again, aren't most CPU architectures a lot faster at doing non-floating point math? How smooth is libvorbis decoding on your RPi 2? It seems like it's not causing you any substantial problems from what you're saying.I compiled the example cli decoder from Tremor. It is significantly faster on ARMv7 than the floating point decoder, but is the difference big enough and relevant to keep libtremor?
libtremor:libvorbis:Code: Select all
$ time cat test.ogg | ./ivorbisfile_example > /dev/null [...] real 0m6,825s user 0m6,779s sys 0m0,086s
Code: Select all
time ffmpeg -i test.ogg -f wav - > /dev/null [...] real 0m10,662s user 0m10,469s sys 0m0,162s
"The Athenians, however, represent the unity of these opposites; in them, mind or spirit has emerged from the Theban subjectivity without losing itself in the Spartan objectivity of ethical life. With the Athenians, the rights of the State and of the individual found as perfect a union as was possible at all at the level of the Greek spirit." -- Hegel's philosophy of Mind
Re: Is libtremor still needed in the platform?
That's not really relevant here because we aren't a dedicated multimedia application that can focus tightly on maximum performance for one particular task.
Same for a decoding CLI application, you cannot compare that kind of use with use inside a multimedia&internet enabled application.
Using half of one core is absolutely significant, and not the kind of load I'd like to see with just singular playback. What is the CPU usage when playing the same file when building and using libtremor, instead? That would give us some real comparative numbers here.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
Re: Is libtremor still needed in the platform?
Smooth enough, if I use a media player like VLC. It's around 5% CPU usage and more or less the same than mp3 or aac decoding (only slightly slower).
I tried Ogg/Vorbis playback in Palemoon (31.1.0) again and now I'm getting around 110% CPU usage (no active websites in other tabs). Again, mp3 and aac looks very much the same (in htop). Weirdly I only get white-noisy distortions with mp3 and aac files on the Pi. You can barely recognize the music in the background. Sound to me like playing the PCM data in the wrong format (signed/unsigned, big/little endian, float/integer). Both files play fine on Palemoon x86 (31.0.0) and it uses much fewer resources (15%, some old slow dual core Intel CPU).It seems like it's not causing you any substantial problems from what you're saying.
So maybe tremor vs libvorbis is not the biggest issue.
Re: Is libtremor still needed in the platform?
Is it not? I didn't want to compare the performance of Palemoon with a CLI / media player. It was more about the baseline and the difference in performance of the raw decoding task between tremor and libvorbis.Moonchild wrote: ↑2022-07-24, 10:24That's not really relevant here because we aren't a dedicated multimedia application that can focus tightly on maximum performance for one particular task.
Same for a decoding CLI application, you cannot compare that kind of use with use inside a multimedia&internet enabled application.
If someone builds Palemoon with libtremor, I'm happy to test it.
Re: Is libtremor still needed in the platform?
you can't make a baseline if you don't compare in the same environment. if you want to make a baseline comparison you have to keep everything except the lib the same, otherwise you're unable to compare anything.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
Re: Is libtremor still needed in the platform?
The problem is the binaries can potentially be run on a platform that does not have floating point, since the 32-bit ARM specification does not guarantee floating point... it could be run on a model that has no FPU. 64-bit ARM floating point is part of the specification, so there are no 64-bit ARM systems without an FPU.
- jobbautista9
- Keeps coming back
- Posts: 782
- Joined: 2020-11-03, 06:47
- Location: Philippines
- Contact:
Re: Is libtremor still needed in the platform?
Thanks everyone especially athenian200 and dbsoft for your input. I think I'm fully convinced now that the tremor library should stay.
merry mimas
XUL add-ons developer. You can find a list of add-ons I manage at http://rw.rs/~job/software.html.
Mima avatar by 絵虎. Pixiv post: https://www.pixiv.net/en/artworks/15431817
Re: Is libtremor still needed in the platform?
At least until 32-bit ARM really isn't a thing any more aside from museum piecesjobbautista9 wrote: ↑2022-07-25, 02:34I think I'm fully convinced now that the tremor library should stay.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite
Re: Is libtremor still needed in the platform?
potentially, theoretically, but not really.dbsoft wrote: ↑2022-07-24, 21:25The problem is the binaries can potentially be run on a platform that does not have floating point, since the 32-bit ARM specification does not guarantee floating point... it could be run on a model that has no FPU. 64-bit ARM floating point is part of the specification, so there are no 64-bit ARM systems without an FPU.
Do you know any computer without hardware floating point support that could run Palemoon reasonably well? I bet there is none.
Is there any Palemoon binary for download that is compiled with tremor?
What about much more popular codecs like mp3 or aac? Are they floating point free?
What are with other parts of Palemoon. Is everything integer math or would the potential desktop machine with floating point free CPU have to emulate floating point instructions?
Which OS and desktop environment would the theoretical floating point disabled computer run?