Benchmark vs Waterfox on Waterfox Homepage

[Ryrynz]

After seeing the following results on the Waterfox page, the following statement kind of bugged me.

"Please note Palemoon was run on a clean profile, whereas Firefox and Waterfox used heavily used profiles."

So Pale Moon should have had the advantage? Or at least it was intended to imply that.. Anyway I'm kind of skeptical about those benchmarks, except for the Sunspider and Fishbowl which mirror my own results so I made a post on their forum.

MrAlex, how do you get a 500,000+ score on Browsermark with Firefox? And even more so, how do you achieve greater than 550,000 with Pale Moon and Waterfox?

I'm running an i7 3570K (default clock) with Intel HD 4000 and I can only get 424338 with a blank 32bit Firefox profile. I also ran a blank Firefox profile on a Q9550 (default clock) with a Radeon HD5850 and scored 380611, so this really doesn't stack up with the results you have posted. Were you testing x64 builds of Firefox? If so which version? My results should be higher than the i7 960 results as the 3570 is a faster CPU.

Also, how is Fishbowl benchmarked in your results? The only two options I can see is selecting auto and have the browser give an upper limit of fish before it drops below 60FPS (~740 fish on my system with Waterfox) or you can benchmark a set number of fish and measure the FPS (2000 fish ~20FPS, 1750 Fish ~22 FPS)

Just FYI: 64bit browsers score lower in most benchmarks vs 32bit ones so comparing to Firefox 32bit with the typical benchmarks probably isn't a good idea, please read the following post from Moonchild @ the Pale Moon forums Here (32-bit versus 64-bit and tight loops)

The benchmark results should ideally be redone, over multiple systems with different specs for a good cross comparison either with blank profiles only
or two sets of benchmarks one blank the other not across each browser, consistency is key to valid benchmarking.

Although by the looks of it the only other browser you could properly compare to is Pale Moon as that's the only 64bit Firefox browser that's not alpha code, which kind of defeats the purpose of benchmarking Waterfox really as Waterfox is meant to serve as an alternative to Firefox (which you can't benchmark against very well due to the reasons mentioned in the link above from Moonchild)

I've often seen Waterfox on my system benchmark as slower than Firefox (Same goes for 64bit Pale Moon) so it might just be a good idea to remove the benchmarking page altogether as it really doesn't highlight the true strengths of a 64bit Firefox.

Unless of course you just want to have it out with Pale Moon...

[megaman]

This is Waterfox 14?
If Palemoon was using the 14 core, might it have been different?

Let's see how the 15 core works out in PM13, then we can test with WF 15, for now that test with WF 14 is just because Fx still improves. Since they do not say, I am just guesstimating.

[Ryrynz]

Yeah that's 14.0.2 (I do like the new installer and page BTW) Yeah will be an interesting face off Pale Moon 15 vs Waterfox 15. Remember Pale Moon is going straight to version 15 rather than 13 to maintain proper addon compatibility.
If there are any performance enhancements in 13 & 14 that haven't made their way into Pale Moon (I'm not aware of any) there's probably nothing in it.

[dark_moon]

You know, benchmarks dont show the real performance of a browser. Moonchild tell you this

Much browser are optimized for benchmarks, but are slower then other browsers in real usage

[Ryrynz]

Yeah like how the browser performs when opening up multiple tabs for example and other general UI stuff can't be easily compared.

Comparing browsers with the same fundamental code and same instruction set should be comparable in this way tho, Moonchild has mentioned that.
Granted these are limited tools but this is as close as we can get to comparing the two AFAIK.

[Moonchild]

Also keep in mind that Waterfox is built with ICC and benchmarked on an Intel Core processor. That is stacking the odds in Waterfox's favor and biasing the test - try running Waterfox benchmarks on an AMD processor as well, and put them next to it, I would like to see those results! And as pointed out, synthetic benchmarks don't really give a good picture.

Ryrynz: they are also not comparing browsers "with the same fundamental code and same instruction set" because they are comparing Waterfox 14 with (I assume) Pale Moon 12.3 - This is why I have a speed comparison on my own page with the same major/minor version number. You can make an "equal" comparison again with the next major release of Pale Moon (v15)

[Ryrynz]

Also keep in mind that Waterfox is built with ICC and benchmarked on an Intel Core processor. That is stacking the odds in Waterfox's favor and biasing the test - try running Waterfox benchmarks on an AMD processor as well, and put them next to it, I would like to see those results!

I will actually do that at work, we have a few AMD based laptops there (all current gen) I'll run a whole bunch of benches on them when Pale Moon and Waterfox hit 15 final and submit the results for all to enjoy.

they are also not comparing browsers "with the same fundamental code and same instruction set" because they are comparing Waterfox 14 with (I assume) Pale Moon 12.3

Well to be fair it's the current version of Waterfox vs the current version of Pale Moon? which is the best they can do right now.

You can make an "equal" comparison again with the next major release of Pale Moon (v15)

YUP.

[Moonchild]

Well to be fair it's the current version of Waterfox vs the current version of Pale Moon? which is the best they can do right now.

This is very true, but the results are likely much less accurate because (as I already pointed out in my FAQ question) you are measuring performance through the very thing you are actually testing (timing measured through javascript for benchmarking javascript). If the code changes, the you are both changing the thing you are measuring and the way you measure them, which can throw reliability out the window real quick.

If there was a way to measure this performance independently through a system that doesn't change, you could make a better comparison.

[Hoggyy]

I just downloaded Waterfox 14.0.2.. I've also been trying to make in-use comparisons between Pale Moon 12.3, Waterfox 13, and now 14.0.2.. On my Gateway's AMD A8-3500M 64-bit quad-core APU laptop.

It seems as if WFx (both version) could be a bit faster than PM 12.3 on loading many pages and even some user interface response times. However the way smooth scrolling is implemented in PM 12.3 makes it more worth it to me.. I tried to duplicate PM's smooth-scrolling settings in WFx14, but it just wasn't nice like in PM. I also love PM's special staus-bar options.

I still need to remember to check how the interfaces respond with, say, scrolling and page while it firs loads - which has been a problem for quite a while in most of the Fx-based variants (including Fx).. It always seem to pause when the page is loading.


About this ---- didn't Mozilla say they were working on cleaning up the code for faster response times??? I thought I remember reading that a while ago - I think at the time when they introduced SPDY. And considering PM isn't updated as often..... (Which is usually better than too-often.) I remember thinking "Well it's about damn fricken time for starting to speed things up!".


Well - hopefully Waterfox will become faster as the Mozilla code cleanup propagates down to Pale Moon! Still like Pale Moon too much to give it up - and I'm pretty sure it's quite faster on my 32-bit WinXP machines. (I removed Fx completely from the 32-bit machines so I can't compare those anymore. Maybe I should re-install it. )


Keep up the good work!

BTW - would there be any chance that you, yourself, could work on multi-processing?? Wishful thinking on my part.. I suppose that would be quite complicated, especially while remaining future compatible with Fx & add-ons. Maybe some tab handling or add-on handling could be moved to other cores if available??? Or maybe that AMD-64 (& Intel) optimized versions idea I seem to remember being talked about earlier in another thread!! (Wink-wink-nudge-nudge. )

[demoneye]

to keep it short , there are so many variables in each person pc that can impact the results in self test mode , for good or for bad , so what i am trying to say is , screw all tests and use your own surfing experience to decide which browser is best for you