Support <picture> element + some things relating to it

Talk about code development, features, specific bugs, enhancements, patches, and similar things.
Forum rules
Please keep everything here strictly on-topic.
This board is meant for Pale Moon source code development related subjects only like code snippets, patches, specific bugs, git, the repositories, etc.

This is not for tech support! Please do not post tech support questions in the "Development" board!
Please make sure not to use this board for support questions. Please post issues with specific websites, extensions, etc. in the relevant boards for those topics.

Please keep things on-topic as this forum will be used for reference for Pale Moon development. Expect topics that aren't relevant as such to be moved or deleted.
User avatar
__NM64__
Lunatic
Lunatic
Posts: 359
Joined: 2013-10-17, 05:29
Location: Northeast Ohio

Support <picture> element + some things relating to it

Unread post by __NM64__ » 2014-09-16, 05:32

Pretty simple, add support for the <picture> element.

Specification:
http://www.w3.org/html/wg/drafts/html/m ... re-element

Walkthrough:
https://longhandpixels.net/blog/2014/02 ... re-element

Demos & info (including current browser support):
http://responsiveimages.org

Bugzilla listings (Status: RESOLVED FIXED):
https://bugzilla.mozilla.org/show_bug.cgi?id=870022
https://bugzilla.mozilla.org/show_bug.cgi?id=870021
https://bugzilla.mozilla.org/show_bug.cgi?id=1030606




Now it may be optimal that, when right clicking on an image that uses the <picture> element and then selecting "Save As...", the browser would by default save the highest available resolution of that image (rather than saving whatever resolution of the image is displayed). Alternatively, perhaps the "Save as..." menu opiton could be a cascaded context menu for images that use <picture>, thereby allowing the user to specifically save any of the images. It would probably be similarly wise to make the right-click "View Image" option also link to the highest resolution image by default.

Another thing is that it may be useful to have the ability to flat out always load a specific size in a web pages by default, whether it's the highest resolution, the lowest, or a middle resolution. In particular, smaller resolution could be useful for slow and/or bandwidth-metered connections while largest would be good for big high-resolution displays on a fast unlimeted bandwidth connection. Heck, maybe it could even be possible to have a setting to load the lowest resolution images first, then once the page is fully loaded, start loading one of the higher resolutions (this would have a similar result to interlaced images).



One of the main things I'm concerned about is having fallback images that don't use <picture> being of a lower resolution than those made availble via <picture>. This is demonstrated with the following demo: http://responsiveimages.org/demos/on-a-grid

I'm also concerned about images that use <picture> having available larger resolutions than whatever image is displayed, but then getting a medium or low resolution image when saving said image and not even being aware that a higher resolution version even exists.
Last edited by __NM64__ on 2014-09-16, 08:59, edited 6 times in total.

New Tobin Paradigm

Re: Support <picture> element + some things relating to it

Unread post by New Tobin Paradigm » 2014-09-16, 06:01

Can you find a MozCo bug for it too? ;)

User avatar
__NM64__
Lunatic
Lunatic
Posts: 359
Joined: 2013-10-17, 05:29
Location: Northeast Ohio

Re: Support <picture> element + some things relating to it

Unread post by __NM64__ » 2014-09-16, 06:04

Got it:
https://bugzilla.mozilla.org/show_bug.cgi?id=870022

Note that its status is "RESOLVED FIXED".

New Tobin Paradigm

Re: Support <picture> element + some things relating to it

Unread post by New Tobin Paradigm » 2014-09-16, 06:15

Fearless Leader,

Bug 870022 - Implement `picture` element

https://hg.mozilla.org/mozilla-central/rev/6260cb355dcd
https://hg.mozilla.org/mozilla-central/rev/f0dc979d76a2
https://hg.mozilla.org/mozilla-central/rev/4a2c5a92f08f
https://hg.mozilla.org/mozilla-central/rev/86ec95551fd5
https://hg.mozilla.org/mozilla-central/rev/500bca3a7429
https://hg.mozilla.org/mozilla-central/rev/ed37e3099398
https://hg.mozilla.org/mozilla-central/rev/0a8d5c15dcf8
https://hg.mozilla.org/mozilla-central/rev/ea8a9d628260
https://hg.mozilla.org/mozilla-central/rev/5e4c6e158007
https://hg.mozilla.org/mozilla-central/rev/34eebb890abd
https://hg.mozilla.org/mozilla-central/rev/bc34ee9ac52c
https://hg.mozilla.org/mozilla-central/rev/9a0ce7ea232e
https://hg.mozilla.org/mozilla-central/rev/5cb4d46a9873
https://hg.mozilla.org/mozilla-central/rev/179a0eb50196
https://hg.mozilla.org/mozilla-central/rev/3b27161e4101

Depends on Bug 870021 - Implement `srcset` attribute on `img`

https://hg.mozilla.org/mozilla-central/rev/9a8a4dc1220e
https://hg.mozilla.org/mozilla-central/rev/9ea35e4133e5
https://hg.mozilla.org/mozilla-central/rev/eb47e85401b2
https://hg.mozilla.org/mozilla-central/rev/91f19f74704c
https://hg.mozilla.org/mozilla-central/rev/4e440a816661
https://hg.mozilla.org/mozilla-central/rev/a0d119f014e1
https://hg.mozilla.org/mozilla-central/rev/b56811f80128
https://hg.mozilla.org/mozilla-central/rev/4a326b265f41
https://hg.mozilla.org/mozilla-central/rev/ba422a2934f5
https://hg.mozilla.org/mozilla-central/rev/255f01c1a924

And Bug 1030606 - --disable-accessibility build error: "HTMLImageElement.cpp:979:43: error: 'picture' is not a member of 'nsGkAtoms'"

https://hg.mozilla.org/mozilla-central/rev/cf5781ef0d10
Last edited by New Tobin Paradigm on 2014-09-16, 06:27, edited 1 time in total.

User avatar
__NM64__
Lunatic
Lunatic
Posts: 359
Joined: 2013-10-17, 05:29
Location: Northeast Ohio

Re: Support <picture> element + some things relating to it

Unread post by __NM64__ » 2014-09-16, 06:26

Added those Bugzilla listings to the first post.
Matt A Tobin wrote:Fearless
Your choice of words is funny considering that I made this thread specificially because I was paranoid about saving lower-res images when higher-res versions were available and I wasn't even aware of it.

New Tobin Paradigm

Re: Support <picture> element + some things relating to it

Unread post by New Tobin Paradigm » 2014-09-16, 06:28

The question is now is the commit code anywhere near easy to implement in our codebase or will Moonchild have a busy day ahead of him.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35478
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Support <picture> element + some things relating to it

Unread post by Moonchild » 2014-09-16, 08:40

Before I even look at this: can someone please explain to me why we need <picture> when we already have have <img>, and js img.src=URL to change image sources, or even just css to do the same?

Give me a good, pressing reason why this is needed to begin with? (And no, "because my fav site wants to use it" is not a good reason)
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

User avatar
__NM64__
Lunatic
Lunatic
Posts: 359
Joined: 2013-10-17, 05:29
Location: Northeast Ohio

Re: Support <picture> element + some things relating to it

Unread post by __NM64__ » 2014-09-16, 08:45

This article covers the issue of "why not just use Javascript?":
http://arstechnica.com/information-tech ... eb-faster/
When a server sends a page to your browser, the browser first downloads all the HTML on the page and then parses it. Or at least that's what used to happen. Modern Web browsers attempt to speed up page load times by downloading images before parsing the page's body. The browser starts downloading the image long before it knows where that image will be in the page layout or how big it will need to be.

This is simultaneously a very good thing—it means images load faster—and a very tricky thing. It means using JavaScript to manipulate images can actually slow down your page even when your JavaScript is trying to load smaller images (because you end up fighting the prefetcher and downloading two images).


EDIT: Also relevent:
http://usecases.responsiveimages.org/
Limitations of current techniques


Reliance on semantically neutral elements and CSS backgrounds:

Large images incur unnecessary download and processing time, slowing the experience for users. To work around this problem, web developers specify multiple sources of the same image at different resolutions and then pick the image of the correct size based on the viewport size. As web developers lack the markup to achieve what they need, they end up relying on semantically neutral elements, CSS background images, and JavaScript libraries. In other words, developers are being forced to willfully violate the authoring requirements of HTML.


Bypass preload scanner

The reliance on semantically neutral elements (e.g., the div and span elements), instead of semantically meaningful elements such as img, prevents browsers from loading the image resources until after the DOM has (at least partially) loaded and scripts have run. This directly hinders the performance work browser engineers have done over the years to optimize resource loading (e.g., WebKit's HTMLPreloadScanner). Unnecessarily bypassing things like the preload scanner can have measurable performance impact when loading documents. See, for example, The WebKit PreloadScanner by Tony Gentilcore for a small study that demonstrates an up to 20% impact in load time when WebKit's PreloadScanner is disabled. More recent performance tests yield similar results. For more information, see How the Browser Pre-loader Makes Pages Load Faster by Andy Davies.


Reliance on scripts and server-side processing:

The techniques rely on either JavaScript or a server-side solution (or both), which adds complexity and redundant HTTP requests to the development process. Furthermore, the script-based solutions are unavailable to users who have turned off JavaScript.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35478
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Support <picture> element + some things relating to it

Unread post by Moonchild » 2014-09-16, 09:06

Modern Web browsers attempt to speed up page load times by downloading images before parsing the page's body. The browser starts downloading the image long before it knows where that image will be in the page layout or how big it will need to be.
No, they don't. That's silly and not true.
How would a browser know what images are in a page before parsing the html? clairvoyance? :lol:

Second article is also very opinionated and debateable. A properly designed website (especially modern one) will decide in real-time what to download by using dynamic html (either client-side or server-side). Thinking reliance on javascript is bad is strange - it's the main focus of most web browsers out there to render modern web pages using DOM structures. Web developers do not lack the markup to achieve what they need for "responsive design" and saying otherwise is just not true either.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

New Tobin Paradigm

W3C - "Approving solutions for problems we don't have"

Unread post by New Tobin Paradigm » 2014-09-16, 09:10

Modern Web browsers attempt to speed up page load times by downloading images before parsing the page's body. The browser starts downloading the image long before it knows where that image will be in the page layout or how big it will need to be.

Code: Select all

[05:06:51.278] GET http://binaryoutcast.com/ [HTTP/1.1 200 OK 363ms]
[05:06:51.761] GET http://roboto-webfont.googlecode.com/svn/trunk/roboto.all.css [HTTP/1.1 200 OK 204ms]
[05:06:51.762] GET http://binaryoutcast.com/wp-content/themes/binoc-v24/style.css [HTTP/1.1 200 OK 28ms]
[05:06:51.763] GET http://code.jquery.com/jquery-1.7.1.js [HTTP/1.1 200 OK 141ms]
[05:06:51.763] GET http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.16/jquery-ui.min.js [HTTP/1.1 200 OK 285ms]
[05:06:51.764] GET http://platform.twitter.com/widgets.js [HTTP/1.1 200 OK 102ms]
[05:06:51.765] GET http://binaryoutcast.com/wp-content/plugins/multisite-global-search/style.css?ver=3.9.2 [HTTP/1.1 200 OK 29ms]
[05:06:51.765] GET http://binaryoutcast.com/wp-content/plugins/ckeditor-for-wordpress/ckeditor/ckeditor.js?t=CBDD&ver=3.9.2 [HTTP/1.1 200 OK 203ms]
[05:06:51.766] GET http://binaryoutcast.com/wp-includes/js/jquery/jquery.js?ver=1.11.0 [HTTP/1.1 200 OK 230ms]
[05:06:51.767] GET http://binaryoutcast.com/wp-includes/js/jquery/jquery-migrate.min.js?ver=1.2.1 [HTTP/1.1 200 OK 234ms]
[05:06:51.768] GET http://binaryoutcast.com/wp-content/plugins/ckeditor-for-wordpress/includes/ckeditor.utils.js?ver=3.9.2 [HTTP/1.1 200 OK 232ms]
[05:06:51.769] GET http://binaryoutcast.com/media/v24/img/division/ux/icons/icon-home.png [HTTP/1.1 200 OK 286ms]
[05:06:51.769] GET http://binaryoutcast.com/wp-content/uploads/2012/12/binoc-monitor.png [HTTP/1.1 200 OK 440ms]
[05:06:52.258] GET http://binaryoutcast.com/wp-content/themes/binoc-v24/media/obsolete/pre-clarity/img/design/v22/background.jpg [HTTP/1.1 200 OK 237ms]
[05:06:52.259] GET http://binaryoutcast.com/wp-content/themes/binoc-v24/media/v24/img/core/ux/header/logo.png [HTTP/1.1 200 OK 55ms]
[05:06:52.259] GET http://binaryoutcast.com/wp-content/themes/binoc-v24/media/v24/img/core/assets/xlucent-pixel-dark.png [HTTP/1.1 200 OK 55ms]
Seems to me images are the LAST thing that loads.. Also Yes some browsers like chrome use prefetching but that causes a security and privacy issue as well as doesn't cover images as far as I know...

zcorpan

Re: Support <picture> element + some things relating to it

Unread post by zcorpan » 2014-09-17, 05:57

The arstechnica article does a terrible job at describing what actually happens. It was never true that browsers first downloaded everything and then started parsing. It is also not really true that images are downloaded before the markup is parsed.

What used to happen is that images are downloaded as soon as the HTML parses creates the <img> element (subject to download priorities of course). That could well be before the browser has completely fetched the CSS and hence, before it knows the layout.

What happens now is pretty much the same thing, except if there is a <script src> that blocks the HTML parser from continuing (it has to wait in case the script uses document.write), browsers now continue parsing the rest of the page with a lightweight speculative tokenizer in order to find further URLs to start fetching while the blocking script is downloaded. This includes images and it can happen before the browser knows the layout.

It was a requirement that responsive images do not break this optimization. Hence <picture>. If it was not important, you could use JS instead.

HTH

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35478
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Support <picture> element + some things relating to it

Unread post by Moonchild » 2014-09-17, 11:34

So, in other words, it's to counter bad web design. I don't see how that is going to be priority.
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

zcorpan

Re: Support <picture> element + some things relating to it

Unread post by zcorpan » 2014-09-18, 07:29

Actually even if you don't use any blocking <script src>s, including the critical resources in the markup is still a good idea to let the browser properly prioritize its network resources.

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 35478
Joined: 2011-08-28, 17:27
Location: Motala, SE
Contact:

Re: Support <picture> element + some things relating to it

Unread post by Moonchild » 2014-09-18, 08:08

If web designers would actually follow the few simple design rules there are, this wouldn't be a problem to begin with. e.g. put CSS in the <head>, don't use blocking javascripts above the fold, load js asynchronously when you can, and use a lazy loading system on websites that have a very large content and lots of images. Simple rules, easy to follow, and no need for arbitrary elements. It's like this is an ongoing war between "camp scripting" and "camp markup". Of course the W3C is going to be solidly in "camp markup". We have advanced scripting, and modern web browsers who are exceedingly good at it, so why jump through hoops to avoid it?
"Sometimes, the best way to get what you want is to be a good person." -- Louis Rossmann
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

Locked