Open UI Customizable Control UI: Solving a Multi-Decade Problem Greg Whitworth (@gregwhitworth) - Salesforce Melanie Richards (@soMelanieSaid) - Microsoft

Agenda Presentation (1 hour) • Problem • What is a Control? • Spectrum of Customization • Solving Fully Style-Able Controls • Solving Fully-Extensible Controls • Process Break (15 min) Discussion + Resolutions (1 hour) Break (15 min) Discussion + Resolutions (1.5 hours)

Problem

Survey results: why developers recreate form controls

Survey results: top 10 controls recreated by web developers

Fabric

Recreating controls causes developer pain

Survey results: most frustrating form controls

MDN Web DNA Survey 2019 results: HTML pain points

Twitter complaints about controls, part I

Form Controls Opportunity Analysis: Participants Research Recruiting • Designer: 31 • Fullstack: 29 • Front-end: 14 Twitter • • • • Designer: 44 Front-end: 112 Back-end: 24 Full-stack: 73

Form Controls Opportunity Analysis: Sample Questions How satisfied are you with the experience of fully styling the <select>* popup window on desktop? How important is it for you to be able to fully style the <select> popup window on desktop? * This is just one example of the questions we asked regarding form control styling in our Opportunity Analysis. We inquired generically and about other form factors.

Form Controls Opportunity Analysis Appearance 10.36 Frequency Effort (building your own) (building your own) Interop 10.48 13.54 15.23

Recreating controls causes user experience issues

Survey results: many web developers are not testing for accessibility

A sampling of accessibility issues in the wild Limited keyboard support Limited keyboard or AT support Not focusable via Keyboard Alexa Rank: 1,247 Rank: 454 Alexa Rank: 96 Not focusable via Keyboard nor AT Support Alexa Rank: 1,293

Survey results: browsers tested during development

What is a Control?

What is a form control, anyway?

An MVC model for form controls Model View Data members and capabilities of the control, available to script. Examples: The user interface. Exposes state to the user, enables the user to interact with the control and change state. • • • value Form association Validity state Controller Intermediary between model and view. • • Inform model of interactions with view, via input handlers Inform view of changes to model, via events, CSS pseudo selectors, exposed properties

Our current mental model of a control A control is a type of component that manages user interaction. The control has controller code that manages changes in the component’s states and its model based on user interaction with its parts.

Spectrum of Customization

Controls currently fall along a customization spectrum Less Customizability None More Hint Limited Fully Style-able (In some cases, minor limitations) color picker accent-color date picker color-scheme datalist file picker select picker range file inputs meter option optgroup Progress checkboxes* radios* select text inputs button output Label summary Fully Extensible

Level of customization: none Interop needed: none

Level of customization: hint select { accent-color: #007eff; } Interop needed: none

Level of customization: limited select::select-button { fill: white; background: #007eff } Interop needed: Limited

Level of customization: limited select::select-button { background-image: my-arrow.png; background-color: #007eff } Interop needed: Limited

Level of customization: limited Why is this not possible today with the built-in <select>? • Parts are not standardized • Specific definition of what CSS properties are valid on the various parts Interop needed: Limited

Level of customization: Fully Style-able Why is this not possible today with the built-in <select>? • Listbox is not standardized • Listbox does not allow overflow • Option elements don’t allow layout changes Interop needed: High

Level of customization: Fully Style-able What is needed to enable this? • Standardized parts • Standardized DOM Structure • Standardized Base styles Interop needed: High

Level of customization: Fully Extensible Why is this not possible today with the built-in <select>? • • • • • Interop needed: Full Behaviors States Parts DOM Structure Structural styles

Spectrum of Customizability Less Customizability More None Hint Limited Fully Style-able Fully Extensible Does not allow any style-ability or extensibility to the control or some of its parts Allows the author to provide a value that the UA applies to a component or control that aligns with the spirit of the property Pseudo elements, HTML elements, attributes that provide customizability but are limited in some manner Elements that allow developers to opt-in to standardized parts, DOM structure, and base styles that useragents apply their styles upon Standardization of a control’s anatomy, states, behaviors, with the capability of reusing controller code via defined parts

Solving fully style-able controls

An MVC model for form controls Enable authors to style Enable authors to use the platform Model View Data members and capabilities of the control, available to script. Examples: The user interface. Exposes state to the user, enables the user to interact with the control and change state. • • • value Form association Validity state Enable authors to use the platform Controller Intermediary between model and view. • • Inform model of interactions with view, via input handlers Inform view of changes to model, via events, CSS pseudo selectors, exposed properties

How to enable standardized DOM & Styles

The custom attribute <select> <option></option> </select> The custom attribute modifies the DOM structure to be only the standardized parts, structural styles, and state styles.

The custom attribute <select custom> <div part=“button-container”> <div part=“selected-value”></div> </div> <div part=“listbox-container”> <option></option> </div> </select>

The custom attribute enables the structural stylesheet

The custom attribute enables the structural stylesheet select[custom] { } select[custom][open] { }

How to enable standardized DOM & Styles Author Stylesheet User Agent Stylesheet Structural CSS “UAs should include in their user agent stylesheet style rules to give widgets a recognizable shape when appearance is none.” And then there’s all: unset ☺ as well

Solving fully extensible controls

An MVC model for form controls Enable authors to extend Enable authors to use the platform Model View Data members and capabilities of the control, available to script. Examples: The user interface. Exposes state to the user, enables the user to interact with the control and change state. • • • value Form association Validity state Enable authors to use the platform Controller Intermediary between model and view. • • Inform model of interactions with view, via input handlers Inform view of changes to model, via events, CSS pseudo selectors, exposed properties

Web devs can update slots to replace the content of control parts…

<div slot=”listbox” part=”listbox” class=”my-box”> <option> <img src=”annie.jpg” alt=”” /> <p class=”name”>Annie Lindqvist</p> <p class=”title”>Designer</p> <span class=”status”>Online</span> </option> … </div>

…or they can replace the entire view with their own Shadow DOM let customSelect = document.createElement(‘select’); customSelect.setAttribute(“custom”, “”); let selectShadow = customSelect.attachShadow({ mode: ‘open’ }); selectShadow.innerHTML = My custom select UI; document.body.appendChild(customSelect); Core parts of the author’s shadow DOM must be labeled with the part attribute: part=”button”, part=”listbox”

Parts help the web dev leverage the platform Controller code can: 1. Inform the model of user interactions with the view, e.g. selected option 2. Apply the right intrinsic accessibility semantics, e.g. part-appropriate roles, states, and properties 3. Wire up the correct behaviors, e.g. keyboard interactions for opening and closing the select popup, traversing through options, etc.

Extensibility requires standardized DOM structure

Extensibility requires standardized states

Extensibility requires standardized behaviors

Process for holistic control standardization

Fabric Design Blueprint

Fabric Open UI Blueprint

Discussion

Discussion Agenda Break (15 min) Problem (15 min) Control Definition (15 min) Spectrum (30 min) Break (15 min) Solutions for fully style-able/extensible, standardization (1 hour) Process (30 min)

Discussion: The Problem

Proposed resolution: The Problem Web developers needing to re-create a browser’s form controls is a problem.

Discussion: Definition of a control

Proposed resolution: Control Definition A control is a type of component that manages user interaction. The control has controller code that manages changes in the component’s states and its model based on user interaction with its parts.

Discussion: Spectrum of customization

Spectrum of Customizability Less Customizability More None Hint Limited Fully Style-able Fully Extensible Does not allow any style-ability or extensibility to the control or some of its parts Allows the author to provide a value that the UA applies to a component or control that aligns with the spirit of the property Pseudo elements, HTML elements, attributes that provide customizability but are limited in some manner Elements that allow developers to opt-in to standardized anatomy, DOM structure, and base styles that user-agents apply their styles upon Standardization of a control’s anatomy, states, behaviors, with the capability of reusing controller code via defined parts Proposed resolution: we will normatively define a spectrum of customizability for controls

What parts do you feel are necessary to explore to solve this problem? What is necessary to avoid re-creating controls? Less Customizability More None Hint Limited Fully Style-able Fully Extensible Does not allow any style-ability or extensibility to the control or some of its parts Allows the author to provide a value that the UA applies to a component or control that aligns with the spirit of the property Pseudo elements, HTML elements, attributes that provide customizability but are limited in some manner Elements that allow developers to opt-in to standardized parts, DOM structure, and base styles that useragents apply their styles upon Standardization of a control’s anatomy, states, behaviors, with the capability of reusing controller code via defined parts A B C D E

Proposed resolution We will standardize control anatomy (slots and parts), states, and behaviors.

Fabric Open UI Blueprint

Proposed resolution Control definitions will begin in Open UI to have a complete specification created for all parts, states and behaviors. New CSS pseudo elements, classes or primitives will be standardized in the CSSWG. New elements, attributes or DOM events will be standardized in WHATWG. New ARIA roles will be standardized in the ARIA WG.

Appendix

Resources • Initial thoughts on standardizing form controls • Can we please style the <select> control?! • Customizing control UI explainer • Recording of the presentation by Melanie Richards and Greg Whitworth