Customizable Control UI:​ Solving a Multi-Decade Problem​

A presentation at Open UI + CSSWG TPAC Joint Meeting in October 2020 in by Greg Whitworth

Slide 1

Slide 1

Open UI Customizable Control UI: Solving a Multi-Decade Problem Greg Whitworth (@gregwhitworth) - Salesforce Melanie Richards (@soMelanieSaid) - Microsoft

Slide 2

Slide 2

Agenda Presentation (1 hour) • Problem • What is a Control? • Spectrum of Customization • Solving Fully Style-Able Controls • Solving Fully-Extensible Controls • Process Break (15 min) Discussion + Resolutions (1 hour) Break (15 min) Discussion + Resolutions (1.5 hours)

Slide 3

Slide 3

Problem

Slide 4

Slide 4

Slide 5

Slide 5

Slide 6

Slide 6

Survey results: why developers recreate form controls

Slide 7

Slide 7

Survey results: top 10 controls recreated by web developers

Slide 8

Slide 8

Fabric

Slide 9

Slide 9

Recreating controls causes developer pain

Slide 10

Slide 10

Survey results: most frustrating form controls

Slide 11

Slide 11

MDN Web DNA Survey 2019 results: HTML pain points

Slide 12

Slide 12

Twitter complaints about controls, part I

Slide 13

Slide 13

Slide 14

Slide 14

Slide 15

Slide 15

Form Controls Opportunity Analysis: Participants Research Recruiting • Designer: 31 • Fullstack: 29 • Front-end: 14 Twitter • • • • Designer: 44 Front-end: 112 Back-end: 24 Full-stack: 73

Slide 16

Slide 16

Form Controls Opportunity Analysis: Sample Questions How satisfied are you with the experience of fully styling the <select>* popup window on desktop? How important is it for you to be able to fully style the <select> popup window on desktop? * This is just one example of the questions we asked regarding form control styling in our Opportunity Analysis. We inquired generically and about other form factors.

Slide 17

Slide 17

Form Controls Opportunity Analysis Appearance 10.36 Frequency Effort (building your own) (building your own) Interop 10.48 13.54 15.23

Slide 18

Slide 18

Recreating controls causes user experience issues

Slide 19

Slide 19

Survey results: many web developers are not testing for accessibility

Slide 20

Slide 20

A sampling of accessibility issues in the wild Limited keyboard support Limited keyboard or AT support Not focusable via Keyboard Alexa Rank: 1,247 Rank: 454 Alexa Rank: 96 Not focusable via Keyboard nor AT Support Alexa Rank: 1,293

Slide 21

Slide 21

Survey results: browsers tested during development

Slide 22

Slide 22

What is a Control?

Slide 23

Slide 23

What is a form control, anyway?

Slide 24

Slide 24

An MVC model for form controls Model View Data members and capabilities of the control, available to script. Examples: The user interface. Exposes state to the user, enables the user to interact with the control and change state. • • • value Form association Validity state Controller Intermediary between model and view. • • Inform model of interactions with view, via input handlers Inform view of changes to model, via events, CSS pseudo selectors, exposed properties

Slide 25

Slide 25

Our current mental model of a control A control is a type of component that manages user interaction. The control has controller code that manages changes in the component’s states and its model based on user interaction with its parts.

Slide 26

Slide 26

Spectrum of Customization

Slide 27

Slide 27

Controls currently fall along a customization spectrum Less Customizability None More Hint Limited Fully Style-able (In some cases, minor limitations) color picker accent-color date picker color-scheme datalist file picker select picker range file inputs meter option optgroup Progress checkboxes* radios* select text inputs button output Label summary Fully Extensible

Slide 28

Slide 28

Level of customization: none Interop needed: none

Slide 29

Slide 29

Level of customization: hint select { accent-color: #007eff; } Interop needed: none

Slide 30

Slide 30

Level of customization: limited select::select-button { fill: white; background: #007eff } Interop needed: Limited

Slide 31

Slide 31

Level of customization: limited select::select-button { background-image: my-arrow.png; background-color: #007eff } Interop needed: Limited

Slide 32

Slide 32

Level of customization: limited Why is this not possible today with the built-in <select>? • Parts are not standardized • Specific definition of what CSS properties are valid on the various parts Interop needed: Limited

Slide 33

Slide 33

Level of customization: Fully Style-able Why is this not possible today with the built-in <select>? • Listbox is not standardized • Listbox does not allow overflow • Option elements don’t allow layout changes Interop needed: High

Slide 34

Slide 34

Level of customization: Fully Style-able What is needed to enable this? • Standardized parts • Standardized DOM Structure • Standardized Base styles Interop needed: High

Slide 35

Slide 35

Level of customization: Fully Extensible Why is this not possible today with the built-in <select>? • • • • • Interop needed: Full Behaviors States Parts DOM Structure Structural styles

Slide 36

Slide 36

Spectrum of Customizability Less Customizability More None Hint Limited Fully Style-able Fully Extensible Does not allow any style-ability or extensibility to the control or some of its parts Allows the author to provide a value that the UA applies to a component or control that aligns with the spirit of the property Pseudo elements, HTML elements, attributes that provide customizability but are limited in some manner Elements that allow developers to opt-in to standardized parts, DOM structure, and base styles that useragents apply their styles upon Standardization of a control’s anatomy, states, behaviors, with the capability of reusing controller code via defined parts

Slide 37

Slide 37

Solving fully style-able controls

Slide 38

Slide 38

An MVC model for form controls Enable authors to style Enable authors to use the platform Model View Data members and capabilities of the control, available to script. Examples: The user interface. Exposes state to the user, enables the user to interact with the control and change state. • • • value Form association Validity state Enable authors to use the platform Controller Intermediary between model and view. • • Inform model of interactions with view, via input handlers Inform view of changes to model, via events, CSS pseudo selectors, exposed properties

Slide 39

Slide 39

How to enable standardized DOM & Styles

Slide 40

Slide 40

The custom attribute <select> <option></option> </select> The custom attribute modifies the DOM structure to be only the standardized parts, structural styles, and state styles.

Slide 41

Slide 41

The custom attribute <select custom> <div part=“button-container”> <div part=“selected-value”></div> </div> <div part=“listbox-container”> <option></option> </div> </select>

Slide 42

Slide 42

The custom attribute enables the structural stylesheet

Slide 43

Slide 43

The custom attribute enables the structural stylesheet select[custom] { } select[custom][open] { }

Slide 44

Slide 44

How to enable standardized DOM & Styles Author Stylesheet User Agent Stylesheet Structural CSS “UAs should include in their user agent stylesheet style rules to give widgets a recognizable shape when appearance is none.” And then there’s all: unset ☺ as well

Slide 45

Slide 45

Solving fully extensible controls

Slide 46

Slide 46

An MVC model for form controls Enable authors to extend Enable authors to use the platform Model View Data members and capabilities of the control, available to script. Examples: The user interface. Exposes state to the user, enables the user to interact with the control and change state. • • • value Form association Validity state Enable authors to use the platform Controller Intermediary between model and view. • • Inform model of interactions with view, via input handlers Inform view of changes to model, via events, CSS pseudo selectors, exposed properties

Slide 47

Slide 47

Web devs can update slots to replace the content of control parts…

<div slot=”listbox” part=”listbox” class=”my-box”> <option> <img src=”annie.jpg” alt=”” /> <p class=”name”>Annie Lindqvist</p> <p class=”title”>Designer</p> <span class=”status”>Online</span> </option> … </div>

Slide 48

Slide 48

…or they can replace the entire view with their own Shadow DOM let customSelect = document.createElement(‘select’); customSelect.setAttribute(“custom”, “”); let selectShadow = customSelect.attachShadow({ mode: ‘open’ }); selectShadow.innerHTML = My custom select UI; document.body.appendChild(customSelect); Core parts of the author’s shadow DOM must be labeled with the part attribute: part=”button”, part=”listbox”

Slide 49

Slide 49

Parts help the web dev leverage the platform Controller code can: 1. Inform the model of user interactions with the view, e.g. selected option 2. Apply the right intrinsic accessibility semantics, e.g. part-appropriate roles, states, and properties 3. Wire up the correct behaviors, e.g. keyboard interactions for opening and closing the select popup, traversing through options, etc.

Slide 50

Slide 50

Extensibility requires standardized DOM structure

Slide 51

Slide 51

Extensibility requires standardized states

Slide 52

Slide 52

Extensibility requires standardized behaviors

Slide 53

Slide 53

Process for holistic control standardization

Slide 54

Slide 54

Fabric Design Blueprint

Slide 55

Slide 55

Fabric Open UI Blueprint

Slide 56

Slide 56

Discussion

Slide 57

Slide 57

Discussion Agenda Break (15 min) Problem (15 min) Control Definition (15 min) Spectrum (30 min) Break (15 min) Solutions for fully style-able/extensible, standardization (1 hour) Process (30 min)

Slide 58

Slide 58

Discussion: The Problem

Slide 59

Slide 59

Proposed resolution: The Problem Web developers needing to re-create a browser’s form controls is a problem.

Slide 60

Slide 60

Discussion: Definition of a control

Slide 61

Slide 61

Proposed resolution: Control Definition A control is a type of component that manages user interaction. The control has controller code that manages changes in the component’s states and its model based on user interaction with its parts.

Slide 62

Slide 62

Discussion: Spectrum of customization

Slide 63

Slide 63

Spectrum of Customizability Less Customizability More None Hint Limited Fully Style-able Fully Extensible Does not allow any style-ability or extensibility to the control or some of its parts Allows the author to provide a value that the UA applies to a component or control that aligns with the spirit of the property Pseudo elements, HTML elements, attributes that provide customizability but are limited in some manner Elements that allow developers to opt-in to standardized anatomy, DOM structure, and base styles that user-agents apply their styles upon Standardization of a control’s anatomy, states, behaviors, with the capability of reusing controller code via defined parts Proposed resolution: we will normatively define a spectrum of customizability for controls

Slide 64

Slide 64

What parts do you feel are necessary to explore to solve this problem? What is necessary to avoid re-creating controls? Less Customizability More None Hint Limited Fully Style-able Fully Extensible Does not allow any style-ability or extensibility to the control or some of its parts Allows the author to provide a value that the UA applies to a component or control that aligns with the spirit of the property Pseudo elements, HTML elements, attributes that provide customizability but are limited in some manner Elements that allow developers to opt-in to standardized parts, DOM structure, and base styles that useragents apply their styles upon Standardization of a control’s anatomy, states, behaviors, with the capability of reusing controller code via defined parts A B C D E

Slide 65

Slide 65

Proposed resolution We will standardize control anatomy (slots and parts), states, and behaviors.

Slide 66

Slide 66

Fabric Open UI Blueprint

Slide 67

Slide 67

Proposed resolution Control definitions will begin in Open UI to have a complete specification created for all parts, states and behaviors. New CSS pseudo elements, classes or primitives will be standardized in the CSSWG. New elements, attributes or DOM events will be standardized in WHATWG. New ARIA roles will be standardized in the ARIA WG.

Slide 68

Slide 68

Appendix

Slide 69

Slide 69

Resources • Initial thoughts on standardizing form controls • Can we please style the <select> control?! • Customizing control UI explainer • Recording of the presentation by Melanie Richards and Greg Whitworth

Slide 70

Slide 70