Different toolkit support: SDL Experiments

[mittorn]

Linux/Unix (except Apple) platform code is using gtk now, but it is not much portable.
First, gtk is desktop-only toolkit. While browser mostly targets desktop, this makes difficult for UXP applications to support non-desktop platforms or desktops, not using gtk.
While gtk is complete tolkit, XPCOM only using it to create windows, handle events and apply native theme. Unaccelerated rendering is using cairo, accelerated compositing is targeting opengl, so most core functionality is not depend on gtk
But many linux desktop environments not using gtk, so gtk theme becomes not native
Also, many linux users do not like how gtk development goes now, (this theme for example: https://forum.palemoon.org/viewtopic.php?f=62&t=32251)
gtk2 is deprecated and gtk3 will be deprecated soon, so gtk dependency may cause much work in future

I've tried to get rid of gtk few years ago, but failed because of some issues with npapi code.
Now i started experinments again and got some success.
First, i've successfully built headless UXP:
https://git.disroot.org/mittorn/UXP/src/branch/headless
It is based on uikit nsWindow code (it does not seems to even compile now), so there is many uikit leftovers
While null widgets backend is not fully complete and event handling is undone, this already might be useful for running tests or running some automated tasks. Also, i successfully initialized webgl and compositor on server environment without any X/Wayland/gtk/etc even installed.
Note that this branch is very WIP, it's causing busy-waiting on idle now because there is no event waiting on UI thread

Next, i started making SDL2 widget platform.
While SDL2 is not very optimized for desktop apps (no partial invalidation, no partial window update), this should be fine when running gpu-accelerated browser and using desktop compositing. But main reason i trying SDL because it will allow easily port any UXP app to any platform SDL supports, not relying on X11 support, gtk/gdk
I
Now i successfully built SDL-enabled browser that only depended on SDL and standard C/C++ libraries. LAYERS_BASIC and opengl compositing works
While most desktop toolkits support supports child/popup widgets, SDL does not, and some SDL backends does not support multiple windows at all, so i tweaked nsViewportFrame to handle popup windows.

Keyboard and clipboard handling is not fully done yet, but i want share some screenshots:

palemoon-sdl.png

basilisk-snake-sdl.png

basilisk-no-omtc.png

Native theme is broken now too (invisible text, wrong widget sizes), so i installed custom themes to make it look normally

This is not done to replace GTK/X11 targets yet. I think, it's better to make separate platform for X11 because there are many features (like printing, rich clipboard, xrender, async events, texture sharing, npapi) that already implemeted and rewriting it without gtk should not be difficult. But this might be good start for non-x11 platforms

I wondering if Pale Moon, Basilisk and other UXP apps developers interested in different tookit support. Yes, it's some additional code, but much less than code, targeting X11 directly (because SDL handles most platform interaction). Maybe this helps refactor and simplify other platform code later

[laki2]

I generally agree with your post.

I feel like GTK has gone in such a bad direction, and who knows how long Pale Moon GTK2/3 will still function properly; I wish Linux just stayed on GTK2 and called it a day since that works much better.

[mittorn]

I think it's OK to support gtk now, but only on gnome or gtk/based systems, not as only availiable UNIX GUI toolkit
I doing this for non-gtk systems.
Also, SDL build is not using any x11-related code, so should work as is on haiku, osx and other systems.

[mittorn]

It seems, i missed SDL_UpdateWindowSurfaceRects function, so it should be possible to implement partial updates too.
The only missing feature is damage region on expose events, which will not affect performance too much and only affects non-composited systems

[FranklinDM]

Looks interesting. Just curious, why SDL2 and not SDL3?

[Moonchild]

Native theme is broken now too (invisible text, wrong widget sizes), so i installed custom themes to make it look normally

The default theme goes at length to integrate with whatever os and widget toolkit it is running on. It will require a separate subtheme for use on SDL to integrate its styling and widgets like is done for all the other widget toolkits that are supported. so this styling issue was entirely expected.

While browser mostly targets desktop, this makes difficult for UXP applications to support non-desktop platforms

non-desktop platforms like what?

Just curious, why SDL2 and not SDL3?

And so it begins...

[mittorn]

Looks interesting. Just curious, why SDL2 and not SDL3?

While SDL3 active developed, SDL2 development is done and it will not receive big changes.There is wrapper to run SDL2 apps on SDL3. so it will run on both versions. Some systems still not receive SDL3 packages/ports
It's likely SDL2 code will run on SDL3 as is without changes

It will require a separate subtheme for use on SDL to integrate its styling and widgets like is done for all the other widget toolkits that are supported

Yes, i copied native theme from uikit, but it's missing some requied values and i need fill it with some valid values.
Maybe even make some colors configurable because SDL does not know about system theme colors and fonts. So custom theme is temporary solution to see all text

non-desktop platforms like what?

SailfishOS, PostmarketOS, some game consoles, maybe even Android? Or if someone wants using UXP for their embedded/kiosk app on embedded linux
While desktop UI is not much useful on smartphones, it should be OK on tablets.
And it should be possible to restore old xul firefox UI for smartphones as separate app, it's much better than modern native widgets UI, but i 'm not sure we really need this

[Moonchild]

Maybe even make some colors configurable because SDL does not know about system theme colors and fonts.

If it doesn't/can't know about system themes and fonts then the only reasonable solution is to use hard-coded colours and fonts for all elements. Configurability would be a "nice to have" but shouldn't be a blocker for SDL builds out of the gate.

SailfishOS, PostmarketOS, some game consoles, maybe even Android?

If you're planning on building UXP for Android devices/hardware and other mobile phone OSes like the ones mentioned, UXP isn't a good choice because all Android considerations were removed from the platform as we focus solely on desktop use. Just be aware you may be seeing potential use cases that aren't feasible.

[mittorn]

If you're planning on building UXP for Android devices/hardware and other mobile phone OSes like the ones mentioned, UXP isn't a good choice because all Android considerations were removed from the platform as we focus solely on desktop use. Just be aware you may be seeing potential use cases that aren't feasible.

Firefox for android was useless anyway and i think, such port should be made from scratch. All removed android code was related to android native widgets.
For mobile platforms we do not have any alternative to chromium or firefox webengines, which become veeery heavy and difficult to build.
UXP debug build only takes 3 minutes on my machine, comparing to hours for other browsers
Anyway, for android i only interested in tablet devices and in desktop UI for tablets. In this case UXP does not need any separate UI and platform code. Only need changes in build system, freetype and audio. headless build above already using refactored leftovers of android platform font list.
I did not touched audio code yet, i planning look if it may use SDL and why webrtc code fails without pulseaudio
Sailfish/Postmarket phones is other question, but postmarket becomes popular and i think, some developers might be interested in separate browser with mobile UI for it. It does not need any android platform code and may share platform code with wayland-based desktops (but mobile compositors are limited and usually expect single-window apps)
I using sailfishOS on phone, so i will try build sdl version for it soon.

Also, for desktop platforms: are you interested in restoring non-OMTC opengl layers? Layers interface does not looks to be changed too much.
It seems to be removed somewhere in FF30-40 because of ipc/e10s everywhere and LAYERS_BASIC only left for as fallback for transparent widgets, but is OMTC ever needed when not using e10s and APZ? If main thread rendering is needed, i may try restore it.

[Basilisk-Dev]

This is pretty badass. I'm definitely in support of moving away from GTK on Linux/Unix. I also love the idea of not requiring X11 or Wasteland.

Maybe we'll port our browsers to OS/2 since it has SDL2.

[Moonchild]

Anyway, for android i only interested in tablet devices and in desktop UI for tablets.

Well I have an Android tablet so I'll be happy to be a guinea pig for SDL builds

are you interested in restoring non-OMTC opengl layers?{...} , but is OMTC ever needed when not using e10s and APZ?

OMTC is an essential performance consideration. Doing compositing work on the main thread will make the browser very sluggish on larger or more complex pages, so I'm not really seeing a reason to restore mainthread OpenGL compositing. This has little to nothing to do with APZ or e10s, although it is one of the prerequisites for using either.

[mittorn]

Maybe we'll port our browsers to OS/2 since it has SDL2.

Yes, it's should be useful fr non-x11 platforms like OS2, Haiku, MorphOS, old osx without writing platform GUI code for scratch
Also sdl widget backend may be used as example for new backend because it is very simple compared to gtk, windows and cocoa. For basic port you only need to implement surface locking and event processing, for accelerated port only gl context needed and egl is supported already

Default theme is still the question, maybe we need some code querying desktop theme colors for known platform and some hard-coded fallback for unknown.
That's how profile manager looks now:

profilemanager-no-theme.png

[billmcct]

For mobile platforms we do not have any alternative to chromium or firefox webengines, which become veeery heavy and difficult to build.

I personally use Duck Duck Go browser on my Andriod phone. Uninstalled Chrome.

[laki2]

For mobile platforms we do not have any alternative to chromium or firefox webengines, which become veeery heavy and difficult to build.

I personally use Duck Duck Go browser on my Andriod phone. Uninstalled Chrome.

Off-topic:
Offtopic as has nothing to do with the original thread. I use the cromite browser but honestly you can easily manage to use a very lightweight browser from even 12+ years ago because they all use WebView.

[jarsealer]

I personally use Duck Duck Go browser on my Andriod phone. Uninstalled Chrome.

Off-topic:
I use Cromite (https://cromite.org) and IronFox (https://ironfoxoss.org), based on Mull (Firefox for Android). The only issue I have with IronFox is the Mozilla telemetry, but can be disabled in about:config.

[mittorn]

I more interested in desktop targets, than mobile, but with SDL mobile port might become possible. But supporting android is pain anyway.

Off-topic:
Google always breaking existing android apps, add new limtations and security requirements every version and all new android devices will be without unlockable bootloader, root access, etc (like if you do not really own it), because everybody keep buying this devices. Maybe 2015-2020 was good time for dedicated android port, not even SDL, so it can better integrate to OS, but now it is not interesting. Android as OS is dead and it became some "firmware for services"

So SDL port is not for some dedicated mobile or even android port. It is a way to unify platform code to get new platforms support without writing code from scratch for every platform. So i assume android-only browsers off-topic here, and i do not want focus on android support. But if it build and run without much changes, why not try?

Off-topic:
For "not chrome/firefox" mobile browsers:
Anyway of this is using system webview (which is chromium based too) or bundled chromium/firefox engine
Maybe some of browsers adapted for tablet ui, but at least vanilla firefox even does not display tabs in recent versions. Older versions was better, displaying some tabbar, but ux still very ugly, it does not even have "close other tabs" option.
None of mobile browsers display url before opening it, most of browsers not asking before starting download.... so i call this useless and broken. Even running desktop UI as is on mobile would be better

[mittorn]

For platfrom interaction support:
SDL2 only supports text clipboard (both selection and clipboard buffer supported) and receieving dropped files, so implementing full clipboard support will require platform code
SDL3 adds mime types to clipboard, but still only able to receive drag&drop events in verly limited way:
https://github.com/libsdl-org/SDL/issues/8559
So desktop functionality sill need interact window system directly even with SDL3, until full drag and drop implemented
Also sdl3 adds some "system theme" support, but it's only switches between ligth and dark:
https://wiki.libsdl.org/SDL3/SDL_SystemTheme
At least it allows to hardcode some light and dark themes and switch automaticly

[mittorn]

On x11 it's possible to query system colors with xrdb:

Code: Select all

$ xrdb -query
*Box.background:        #d4d0c8
*Box.foreground:        #000000
*Button.activeBackground:       #d4d0c8
*Button.activeForeground:       #000000
*Button.background:     #d4d0c8
*Button.foreground:     #000000
*Button.highlightBackground:    #d4d0c8
...

That's how some pure x11 apps like xmessage get correct system colors
For wayland, it seems we only can use some dbus interfaces, Which, moreover, missing on my system, as Trinity DE is using DCOP, not dbus and even dconf is empty, so need check on some gnome/kde installations

[mittorn]

It seems, somethinng wrong fith file picker theme in Pale Moon

filepicker-pale-moon.png

On gtk it draws with transparent background, with SDL background is white. In Basilisk theme is ok, but some icons still missing

filepicker-basilisk.png

[mittorn]

With custom themes file picker looks ok, it even shows file/folder icons. It seems, default theme is missing something

fp-theme.png

fp-theme1.png

fp-theme2.png

Broken filepicker theme was caused by Moonscape, it seems to be theme bug, with gtk it's even more broken:

fp-theme3.png

transparent regions are clicked-through, so it somehow caused shaping window