A presentation at RubyConf Kenya in in Nairobi, Kenya by Caleb Hearth
These slides intentionally left blank. Yes, I know these slides are blank. No, they’re not broken.
These slides intentionally left blank. Yes, I know these slides are blank. No, they’re not broken.
How I Built Software to Kill People A cautionary tale by @calebthompson I’ve been a developer for about ten years, have a degree in SWE, and have worked with all sorts of businesses and customers in web and native development, including now where I work at Heroku. I’m going to tell you about how I took a job building software to kill people.
But don't get distracted by that; I didn't know at the time.
Even before I'd walked across the stage for graduation, I accepted an offer for an internship. I'd be joining a contracting company for the Department of Defense, or DOD. That’s the part of the government made up by the military in the United States. The DOD outsources all sorts of things, from P-8A Multi-mission Maritime aircraft to blue, shade 451, Poly/wool cloth. They also outsource a lot of software projects.
At the time, I thought nothing of the fact that I'd be supporting the military. Even if I had, they're the good folks right? My dad was in the military. So was my grandfather. I have great respect for those in the military. It was good money, a great opportunity, and a close friend of mine had gotten me the gig. Life was good.
I joined the team after they'd already gotten started on the project. The gist of what we were building was an app that look at how WiFi signal strength changed as your phone moved around. If the signal strength got stronger, you were getting closer. If it got weaker, you were moving away. To find them, we'd collect two pieces of information for each wifi access point in range. The phone's location information and the WiFi signal strength.
It seemed pretty cool compared to what I'd built to up that point. The most complicated thing I’d created was an inventory management system. The app didn't concern itself overmuch with persistence or databases. Who needs to stop and restart a program anyhow? The data was right there in memory and if you forgot how many Aerosmith CDs you had, who cares?
Honestly, the idea of finding WiFi routers based on the signal strength seemed pretty intimidating at the time. The idea impressed me.
But don't get distracted by all this; the software was intended to kill people.
To predict the actual location of the WiFi signal, we used a convolution of two algorithms. This is going to get math-y and technical but don’t worry too much about understanding things.
Both of them relied on the Free Space Path Loss equation. For our purposes, FSPL calculates how far away a phone is from a WiFi signal based on loss in signal strength. It assumes that there is only empty air, or "free space", between the access point and the phone.
R 2 ⌘ 1 � P i ( y i � f i ) 2 P i ( y i � ¯ y ) 2 <latexit sha1_base64="uV6A8sEfYhhvheNCvWMQJDteJSA=">AAACO3icdVDLSgMxFL3js9ZX1aWbYBF0YZkpPuqu6salirVCp5ZMmrHBzMMkIwxh/suN4De4ENy4caGIW/dmWgUVvZDL4Zxzk9zjxZxJZdsP1tDwyOjYeGGiODk1PTNbmps/kVEiCG2QiEfi1MOSchbShmKK09NYUBx4nDa9i71cb15RIVkUHqs0pu0An4fMZwQrQ3VKR0dnupq59DJhV8hZ09qVSYA6mmUrad7X/Lyv5qYMuZG5Cv12aNfDAuk0+7R1SmW7sm1v1tbXUQ62bKeGBoyzgZyK3a9yfef++BYADjqlO7cbkSSgoSIcS9ly7Fi1NRaKEU6zoptIGmNygc9py8AQB1S2dX/3DC0bpov8SJgTKtRnv09oHEiZBp5xBlj15G8tJ//SWonya23NwjhRNCSDh/yEIxWhPEjUZYISxVMDMBHM/BWRHhaYKBN30YTwtSn6H5xUK45dcQ5NGrswqAIswhKsgANbUId9OIAGELiGR3iGF+vGerJerbeBdcj6nFmAH2W9fwAZNbGh</latexit> <latexit sha1_base64="F9nzIegExF7ioCCnhMI4xCYEaFE=">AAACO3icdVDLSgMxFM34rPVVdekmWARdWGak2rqrunFZxdZCp5ZMmrHBzMMkUxjC/Ja60J9wIbhx40IRF27cm2kVtOiFXA7nnJvkHidkVEjTfDBGRsfGJyYzU9npmdm5+dzCYl0EEcekhgMW8IaDBGHUJzVJJSONkBPkOYycOOf7qX7SI1zQwD+WcUhaHjrzqUsxkppq546OTtVmYpOLiPagtaGULSIPthVN1uK0b7hpX09NCbQDfRUcdijbQRyqOPmytXN5s7BjbpeLRZiCkmmV4YCxtqBVMPuVr+zeH9+8X15V27k7uxPgyCO+xAwJ0bTMULYU4pJiRpKsHQkSInyOzkhTQx95RLRUf/cErmqmA92A6+NL2Gd/TijkCRF7jnZ6SHbFsJaSf2nNSLrllqJ+GEni48FDbsSgDGAaJOxQTrBksQYIc6r/CnEXcYSljjurQ/jeFP4P6psFyyxYhzqNPTCoDFgGK2ANWKAEKuAAVEENYHANHsEzeDFujSfj1XgbWEeMr5kl8KuMj088ebP6</latexit> <latexit sha1_base64="F9nzIegExF7ioCCnhMI4xCYEaFE=">AAACO3icdVDLSgMxFM34rPVVdekmWARdWGak2rqrunFZxdZCp5ZMmrHBzMMkUxjC/Ja60J9wIbhx40IRF27cm2kVtOiFXA7nnJvkHidkVEjTfDBGRsfGJyYzU9npmdm5+dzCYl0EEcekhgMW8IaDBGHUJzVJJSONkBPkOYycOOf7qX7SI1zQwD+WcUhaHjrzqUsxkppq546OTtVmYpOLiPagtaGULSIPthVN1uK0b7hpX09NCbQDfRUcdijbQRyqOPmytXN5s7BjbpeLRZiCkmmV4YCxtqBVMPuVr+zeH9+8X15V27k7uxPgyCO+xAwJ0bTMULYU4pJiRpKsHQkSInyOzkhTQx95RLRUf/cErmqmA92A6+NL2Gd/TijkCRF7jnZ6SHbFsJaSf2nNSLrllqJ+GEni48FDbsSgDGAaJOxQTrBksQYIc6r/CnEXcYSljjurQ/jeFP4P6psFyyxYhzqNPTCoDFgGK2ANWKAEKuAAVEENYHANHsEzeDFujSfj1XgbWEeMr5kl8KuMj088ebP6</latexit> <latexit sha1_base64="TlDf1W5SPBOARhLlfPKdsAl9Zbg=">AAACO3icdVDLSgMxFM34rPU16tJNsAh10ZIRH3VXdONSxT6gU0smzWho5mGSEYYw/+XGn3Dnxo0LRdy6N9NW0KIXcjmcc26Se7yYM6kQerKmpmdm5+YLC8XFpeWVVXttvSmjRBDaIBGPRNvDknIW0oZiitN2LCgOPE5b3uAk11t3VEgWhZcqjWk3wNch8xnBylA9++LiSu9mLr1N2B10Klq7MglgT7OsnOa94ud9Jzdl0I3MVXDSoV0PC6jTbGzr2SVUPUIHtb09mIND5NTgiHH2oVNFwyqBcZ317Ee3H5EkoKEiHEvZcVCsuhoLxQinWdFNJI0xGeBr2jEwxAGVXT3cPYPbhulDPxLmhAoO2Z8TGgdSpoFnnAFWN3JSy8m/tE6i/FpXszBOFA3J6CE/4VBFMA8S9pmgRPHUAEwEM3+F5AYLTJSJu2hC+N4U/g+au1UHVZ1zVKofj+MogE2wBcrAAYegDk7BGWgAAu7BM3gFb9aD9WK9Wx8j65Q1ntkAv8r6/ALVbq82</latexit> The first algorithm was R^2. It measured the difference between the signal strength we'd observed and the expected signal strength at each distance in a search grid. Locations with the lowest R^2 error rate were the most likely to be the source of the WiFi signal.
✓ 4ˆ � 5 3 n ◆ 1 5 ⇡ 1 . 06 ˆ � n � 1 / 5 , <latexit sha1_base64="RtGGtO8ILOPIq2pyroW4mEk+qu0=">AAACRnicdVBNixMxGH6nfq31q+rRS3ARVtCa0Xa3HoRVLx5XsLsLnWnJpJlO2ExmSN4RS8iv8id48ezNm1cvHhTxaqajsIo+EHh4PkjyZLWSFin9GPXOnD13/sLWxf6ly1euXhtcv3Foq8ZwMeWVqsxxxqxQUospSlTiuDaClZkSR9nJ89Y/ei2MlZV+hetapCVbaZlLzjBIi0FakCckUSLHnSQ3jLtRUjB0iZWrkvn52LtH2idGrgq8O3ddJPZu7D1JWF2b6g2Jh3SXnG4RPXf34wdjf28x2KbDx3R3MhqRluzReEI6JR63xQ2295/St58A4GAx+JAsK96UQiNXzNpZTGtMHTMouRK+nzRW1IyfsJWYBapZKWzqNjN4cicoS5JXJhyNZKOebjhWWrsus5AsGRb2b68V/+XNGswnqZO6blBo3l2UN4pgRdpNyVIawVGtA2HcyPBWwgsWpsKwfD+M8Pun5P/k8OEwpsP4ZVjjGXTYgltwG3Yghj3YhxdwAFPg8A4+w1f4Fr2PvkTfox9dtBf96tyEP9CDn6ofs3c=</latexit> <latexit sha1_base64="50AFhZ12WguaQSaO7crJI4F5CK8=">AAACRnicdVC9bxMxHPWFrxK+0nZksagqFQmCDUmbDkgFFsYikbZS7hL5HF/Oqs93sn9XEVn+q9hZWJjZGPgDWBhAiBVfjkoFwZMsPb0P2X5ppaQFQj5FnUuXr1y9tna9e+Pmrdt3eusbR7asDRdjXqrSnKTMCiW1GIMEJU4qI1iRKnGcnr5o/OMzYaws9WtYViIp2ELLTHIGQZr1khw/xbESGezEmWHcDeKcgYutXBTMT4fePdE+NnKRw/2payPUu6H3OGZVZco3mPbJLr7YwnrqHtJHQ/9g1tsi/X2yOxoMcEP2CB3hVqHDprjC1sEz8vbz+jt9OOt9jOclrwuhgStm7YSSChLHDEiuhO/GtRUV46dsISaBalYIm7jVDB5vB2WOs9KEowGv1IsNxwprl0UakgWD3P7tNeK/vEkN2ShxUlc1CM3bi7JaYShxsymeSyM4qGUgjBsZ3op5zsJUEJbvhhHOf4r/T44e9ynp01dhjeeoxRq6i+6hHUTRHjpAL9EhGiOO3qMv6Bv6Hn2IvkY/op9ttBP97myiP9BBvwBNVrSx</latexit> <latexit sha1_base64="50AFhZ12WguaQSaO7crJI4F5CK8=">AAACRnicdVC9bxMxHPWFrxK+0nZksagqFQmCDUmbDkgFFsYikbZS7hL5HF/Oqs93sn9XEVn+q9hZWJjZGPgDWBhAiBVfjkoFwZMsPb0P2X5ppaQFQj5FnUuXr1y9tna9e+Pmrdt3eusbR7asDRdjXqrSnKTMCiW1GIMEJU4qI1iRKnGcnr5o/OMzYaws9WtYViIp2ELLTHIGQZr1khw/xbESGezEmWHcDeKcgYutXBTMT4fePdE+NnKRw/2payPUu6H3OGZVZco3mPbJLr7YwnrqHtJHQ/9g1tsi/X2yOxoMcEP2CB3hVqHDprjC1sEz8vbz+jt9OOt9jOclrwuhgStm7YSSChLHDEiuhO/GtRUV46dsISaBalYIm7jVDB5vB2WOs9KEowGv1IsNxwprl0UakgWD3P7tNeK/vEkN2ShxUlc1CM3bi7JaYShxsymeSyM4qGUgjBsZ3op5zsJUEJbvhhHOf4r/T44e9ynp01dhjeeoxRq6i+6hHUTRHjpAL9EhGiOO3qMv6Bv6Hn2IvkY/op9ttBP97myiP9BBvwBNVrSx</latexit> <latexit sha1_base64="7ddxkNEdCFac8KSYj1fgOIzuhVI=">AAACRnicdVDNaxQxHM2sVev6tdWjl9BFqKBrorvtehCKvXhswW0LO7NLJpvZCc1khuQ3xSXkr/PSc2/+CV48KNKrmd0tVNEHgcf7IMlLKyUtEPI1at3auH3n7ua99v0HDx897mw9ObZlbbgY8VKV5jRlViipxQgkKHFaGcGKVImT9Oyg8U/OhbGy1J9gUYmkYHMtM8kZBGnaSXL8HsdKZLATZ4Zx149zBi62cl4wPxl491b72Mh5Di8mbhWh3g28xzGrKlN+xrRHdvHNFtYT94q+HviX006X9N6R3WG/jxuyR+gQrxQ6aIpLdNEah9POZTwreV0IDVwxa8eUVJA4ZkByJXw7rq2oGD9jczEOVLNC2MQtZ/D4eVBmOCtNOBrwUr3ZcKywdlGkIVkwyO3fXiP+yxvXkA0TJ3VVg9B8dVFWKwwlbjbFM2kEB7UIhHEjw1sxz1mYCsLy7TDC9U/x/8nxmx4lPXpEuvsf1nNsomdoG+0givbQPvqIDtEIcfQFfUM/0M/oIvoe/YquVtFWtO48RX+ghX4Dm7yxMw==</latexit> We'd combine that calculation with a Gaussian estimate. It was a graph of the sum of two bell curves: probabilities that a signal was originating at given distances. The first was negative and said “you’re probably not trying to find a signal if you’re standing next to it” and the second was positive and said “this is the most likely distance for the signal to originate”
The algorithm adjusted the width and height of these two curves by consulting past measurements. It created a heat map of probabilities for the signal source.
✓ 4ˆ � 5 3 n ◆ 1 5 ⇡ 1 . 06 ˆ � n � 1 / 5 , <latexit sha1_base64="RtGGtO8ILOPIq2pyroW4mEk+qu0=">AAACRnicdVBNixMxGH6nfq31q+rRS3ARVtCa0Xa3HoRVLx5XsLsLnWnJpJlO2ExmSN4RS8iv8id48ezNm1cvHhTxaqajsIo+EHh4PkjyZLWSFin9GPXOnD13/sLWxf6ly1euXhtcv3Foq8ZwMeWVqsxxxqxQUospSlTiuDaClZkSR9nJ89Y/ei2MlZV+hetapCVbaZlLzjBIi0FakCckUSLHnSQ3jLtRUjB0iZWrkvn52LtH2idGrgq8O3ddJPZu7D1JWF2b6g2Jh3SXnG4RPXf34wdjf28x2KbDx3R3MhqRluzReEI6JR63xQ2295/St58A4GAx+JAsK96UQiNXzNpZTGtMHTMouRK+nzRW1IyfsJWYBapZKWzqNjN4cicoS5JXJhyNZKOebjhWWrsus5AsGRb2b68V/+XNGswnqZO6blBo3l2UN4pgRdpNyVIawVGtA2HcyPBWwgsWpsKwfD+M8Pun5P/k8OEwpsP4ZVjjGXTYgltwG3Yghj3YhxdwAFPg8A4+w1f4Fr2PvkTfox9dtBf96tyEP9CDn6ofs3c=</latexit> <latexit sha1_base64="50AFhZ12WguaQSaO7crJI4F5CK8=">AAACRnicdVC9bxMxHPWFrxK+0nZksagqFQmCDUmbDkgFFsYikbZS7hL5HF/Oqs93sn9XEVn+q9hZWJjZGPgDWBhAiBVfjkoFwZMsPb0P2X5ppaQFQj5FnUuXr1y9tna9e+Pmrdt3eusbR7asDRdjXqrSnKTMCiW1GIMEJU4qI1iRKnGcnr5o/OMzYaws9WtYViIp2ELLTHIGQZr1khw/xbESGezEmWHcDeKcgYutXBTMT4fePdE+NnKRw/2payPUu6H3OGZVZco3mPbJLr7YwnrqHtJHQ/9g1tsi/X2yOxoMcEP2CB3hVqHDprjC1sEz8vbz+jt9OOt9jOclrwuhgStm7YSSChLHDEiuhO/GtRUV46dsISaBalYIm7jVDB5vB2WOs9KEowGv1IsNxwprl0UakgWD3P7tNeK/vEkN2ShxUlc1CM3bi7JaYShxsymeSyM4qGUgjBsZ3op5zsJUEJbvhhHOf4r/T44e9ynp01dhjeeoxRq6i+6hHUTRHjpAL9EhGiOO3qMv6Bv6Hn2IvkY/op9ttBP97myiP9BBvwBNVrSx</latexit> <latexit sha1_base64="50AFhZ12WguaQSaO7crJI4F5CK8=">AAACRnicdVC9bxMxHPWFrxK+0nZksagqFQmCDUmbDkgFFsYikbZS7hL5HF/Oqs93sn9XEVn+q9hZWJjZGPgDWBhAiBVfjkoFwZMsPb0P2X5ppaQFQj5FnUuXr1y9tna9e+Pmrdt3eusbR7asDRdjXqrSnKTMCiW1GIMEJU4qI1iRKnGcnr5o/OMzYaws9WtYViIp2ELLTHIGQZr1khw/xbESGezEmWHcDeKcgYutXBTMT4fePdE+NnKRw/2payPUu6H3OGZVZco3mPbJLr7YwnrqHtJHQ/9g1tsi/X2yOxoMcEP2CB3hVqHDprjC1sEz8vbz+jt9OOt9jOclrwuhgStm7YSSChLHDEiuhO/GtRUV46dsISaBalYIm7jVDB5vB2WOs9KEowGv1IsNxwprl0UakgWD3P7tNeK/vEkN2ShxUlc1CM3bi7JaYShxsymeSyM4qGUgjBsZ3op5zsJUEJbvhhHOf4r/T44e9ynp01dhjeeoxRq6i+6hHUTRHjpAL9EhGiOO3qMv6Bv6Hn2IvkY/op9ttBP97myiP9BBvwBNVrSx</latexit> <latexit sha1_base64="7ddxkNEdCFac8KSYj1fgOIzuhVI=">AAACRnicdVDNaxQxHM2sVev6tdWjl9BFqKBrorvtehCKvXhswW0LO7NLJpvZCc1khuQ3xSXkr/PSc2/+CV48KNKrmd0tVNEHgcf7IMlLKyUtEPI1at3auH3n7ua99v0HDx897mw9ObZlbbgY8VKV5jRlViipxQgkKHFaGcGKVImT9Oyg8U/OhbGy1J9gUYmkYHMtM8kZBGnaSXL8HsdKZLATZ4Zx149zBi62cl4wPxl491b72Mh5Di8mbhWh3g28xzGrKlN+xrRHdvHNFtYT94q+HviX006X9N6R3WG/jxuyR+gQrxQ6aIpLdNEah9POZTwreV0IDVwxa8eUVJA4ZkByJXw7rq2oGD9jczEOVLNC2MQtZ/D4eVBmOCtNOBrwUr3ZcKywdlGkIVkwyO3fXiP+yxvXkA0TJ3VVg9B8dVFWKwwlbjbFM2kEB7UIhHEjw1sxz1mYCsLy7TDC9U/x/8nxmx4lPXpEuvsf1nNsomdoG+0givbQPvqIDtEIcfQFfUM/0M/oIvoe/YquVtFWtO48RX+ghX4Dm7yxMw==</latexit> We'd normalize the two algorithm’s probabilities for each location in the search grid. The combination of probabilities was more correct than either algorithm itself.
✓ 4ˆ � 5 3 n ◆ 1 5 ⇡ 1 . 06 ˆ � n � 1 / 5 , <latexit sha1_base64="RtGGtO8ILOPIq2pyroW4mEk+qu0=">AAACRnicdVBNixMxGH6nfq31q+rRS3ARVtCa0Xa3HoRVLx5XsLsLnWnJpJlO2ExmSN4RS8iv8id48ezNm1cvHhTxaqajsIo+EHh4PkjyZLWSFin9GPXOnD13/sLWxf6ly1euXhtcv3Foq8ZwMeWVqsxxxqxQUospSlTiuDaClZkSR9nJ89Y/ei2MlZV+hetapCVbaZlLzjBIi0FakCckUSLHnSQ3jLtRUjB0iZWrkvn52LtH2idGrgq8O3ddJPZu7D1JWF2b6g2Jh3SXnG4RPXf34wdjf28x2KbDx3R3MhqRluzReEI6JR63xQ2295/St58A4GAx+JAsK96UQiNXzNpZTGtMHTMouRK+nzRW1IyfsJWYBapZKWzqNjN4cicoS5JXJhyNZKOebjhWWrsus5AsGRb2b68V/+XNGswnqZO6blBo3l2UN4pgRdpNyVIawVGtA2HcyPBWwgsWpsKwfD+M8Pun5P/k8OEwpsP4ZVjjGXTYgltwG3Yghj3YhxdwAFPg8A4+w1f4Fr2PvkTfox9dtBf96tyEP9CDn6ofs3c=</latexit> <latexit sha1_base64="50AFhZ12WguaQSaO7crJI4F5CK8=">AAACRnicdVC9bxMxHPWFrxK+0nZksagqFQmCDUmbDkgFFsYikbZS7hL5HF/Oqs93sn9XEVn+q9hZWJjZGPgDWBhAiBVfjkoFwZMsPb0P2X5ppaQFQj5FnUuXr1y9tna9e+Pmrdt3eusbR7asDRdjXqrSnKTMCiW1GIMEJU4qI1iRKnGcnr5o/OMzYaws9WtYViIp2ELLTHIGQZr1khw/xbESGezEmWHcDeKcgYutXBTMT4fePdE+NnKRw/2payPUu6H3OGZVZco3mPbJLr7YwnrqHtJHQ/9g1tsi/X2yOxoMcEP2CB3hVqHDprjC1sEz8vbz+jt9OOt9jOclrwuhgStm7YSSChLHDEiuhO/GtRUV46dsISaBalYIm7jVDB5vB2WOs9KEowGv1IsNxwprl0UakgWD3P7tNeK/vEkN2ShxUlc1CM3bi7JaYShxsymeSyM4qGUgjBsZ3op5zsJUEJbvhhHOf4r/T44e9ynp01dhjeeoxRq6i+6hHUTRHjpAL9EhGiOO3qMv6Bv6Hn2IvkY/op9ttBP97myiP9BBvwBNVrSx</latexit> <latexit sha1_base64="50AFhZ12WguaQSaO7crJI4F5CK8=">AAACRnicdVC9bxMxHPWFrxK+0nZksagqFQmCDUmbDkgFFsYikbZS7hL5HF/Oqs93sn9XEVn+q9hZWJjZGPgDWBhAiBVfjkoFwZMsPb0P2X5ppaQFQj5FnUuXr1y9tna9e+Pmrdt3eusbR7asDRdjXqrSnKTMCiW1GIMEJU4qI1iRKnGcnr5o/OMzYaws9WtYViIp2ELLTHIGQZr1khw/xbESGezEmWHcDeKcgYutXBTMT4fePdE+NnKRw/2payPUu6H3OGZVZco3mPbJLr7YwnrqHtJHQ/9g1tsi/X2yOxoMcEP2CB3hVqHDprjC1sEz8vbz+jt9OOt9jOclrwuhgStm7YSSChLHDEiuhO/GtRUV46dsISaBalYIm7jVDB5vB2WOs9KEowGv1IsNxwprl0UakgWD3P7tNeK/vEkN2ShxUlc1CM3bi7JaYShxsymeSyM4qGUgjBsZ3op5zsJUEJbvhhHOf4r/T44e9ynp01dhjeeoxRq6i+6hHUTRHjpAL9EhGiOO3qMv6Bv6Hn2IvkY/op9ttBP97myiP9BBvwBNVrSx</latexit> <latexit sha1_base64="7ddxkNEdCFac8KSYj1fgOIzuhVI=">AAACRnicdVDNaxQxHM2sVev6tdWjl9BFqKBrorvtehCKvXhswW0LO7NLJpvZCc1khuQ3xSXkr/PSc2/+CV48KNKrmd0tVNEHgcf7IMlLKyUtEPI1at3auH3n7ua99v0HDx897mw9ObZlbbgY8VKV5jRlViipxQgkKHFaGcGKVImT9Oyg8U/OhbGy1J9gUYmkYHMtM8kZBGnaSXL8HsdKZLATZ4Zx149zBi62cl4wPxl491b72Mh5Di8mbhWh3g28xzGrKlN+xrRHdvHNFtYT94q+HviX006X9N6R3WG/jxuyR+gQrxQ6aIpLdNEah9POZTwreV0IDVwxa8eUVJA4ZkByJXw7rq2oGD9jczEOVLNC2MQtZ/D4eVBmOCtNOBrwUr3ZcKywdlGkIVkwyO3fXiP+yxvXkA0TJ3VVg9B8dVFWKwwlbjbFM2kEB7UIhHEjw1sxz1mYCsLy7TDC9U/x/8nxmx4lPXpEuvsf1nNsomdoG+0givbQPvqIDtEIcfQFfUM/0M/oIvoe/YquVtFWtO48RX+ghX4Dm7yxMw==</latexit> If we collected readings while moving in a straight line, we could tell you how far away the WiFi was. If you turned a corner, we could also add direction, so you could find it in 2D space. If you climbed some stairs, we'd show you altitude as well. The technology was the most interesting project I'd ever worked on. It might still be the most interesting.
But don't let that distract you; it was designed to kill people.
I spent a lot of my time pairing on performance improvements to the code. The app worked, but it took about seven minutes to do so. This was partially because we translated code from MatLab, which is optimized for working with matrices of numbers. The Java code we wrote originally used nested loops instead. We're talking quadruply nested here, with some expensive calculations inside.
� � = arctan q (cos � 2 · sin( � � )) 2
� � = arctan q (cos � 2 · sin( � � )) 2
But don't get distracted; that performance increase made it faster to kill people.
The accuracy of the locations wasn't fantastic. I don't remember exactly what it was before we focused on improving this, but an average error about 45 feet, or 14 meters, sticks in my head.
That’s about the length of a shipping container.
That's significant when the wifi range for 802.11n is only about 100 feet or 30 meters. That means we could be up to almost half the range of the router away from where the router actually was.
I talked about the Gaussian estimation, the two curves from the second algorithm. We hard-coded numbers that defined this curve. They were only starting points, but they were starting points every time we made the calculation.
During my software education, I’d learned about a tool I never expected to use in a job. It is called a Genetic Algorithm, and it's a type of machine learning program that produces a set of values that optimize for a desired result. It takes a set of genes - our constants from the gaussian estimation, and looks at how they perform in a fitness function - the location algorithm’s accuracy.
For the dataset of readings I was using in the GA, I knew the actual location of the access points. That meant I could run the geolocation algorithm with each gene’s values in place of the old constants. I could look at the distance between the actual location and the one calculated by the GA-derived values. The shorter the distance, the better the genes were performing.
Doing this a lot of times with a population of genomes, each time making small changes to the top performers and replacing poor performers with new random genes, we can find the top performers across all populations. When the GA ended I could take a look at the values and select the best performer.
I let this genetic algorithm run over the weekend. It was able to increase the accuracy from 40-odd feet to about 10.3 feet, 25% of the error from the original. That is less than the GPS accuracy on the smartphones collecting data. This is too accurate, so it may have been over-optimized against the test data and might not be as accurate against other data sets. This is called overfitting and the way around it is to have separate sets of training and test data.
✓ 4ˆ � 5 3 n ◆ 1 5 ⇡ 1 . 06 ˆ � n � 1 / 5 , <latexit sha1_base64="RtGGtO8ILOPIq2pyroW4mEk+qu0=">AAACRnicdVBNixMxGH6nfq31q+rRS3ARVtCa0Xa3HoRVLx5XsLsLnWnJpJlO2ExmSN4RS8iv8id48ezNm1cvHhTxaqajsIo+EHh4PkjyZLWSFin9GPXOnD13/sLWxf6ly1euXhtcv3Foq8ZwMeWVqsxxxqxQUospSlTiuDaClZkSR9nJ89Y/ei2MlZV+hetapCVbaZlLzjBIi0FakCckUSLHnSQ3jLtRUjB0iZWrkvn52LtH2idGrgq8O3ddJPZu7D1JWF2b6g2Jh3SXnG4RPXf34wdjf28x2KbDx3R3MhqRluzReEI6JR63xQ2295/St58A4GAx+JAsK96UQiNXzNpZTGtMHTMouRK+nzRW1IyfsJWYBapZKWzqNjN4cicoS5JXJhyNZKOebjhWWrsus5AsGRb2b68V/+XNGswnqZO6blBo3l2UN4pgRdpNyVIawVGtA2HcyPBWwgsWpsKwfD+M8Pun5P/k8OEwpsP4ZVjjGXTYgltwG3Yghj3YhxdwAFPg8A4+w1f4Fr2PvkTfox9dtBf96tyEP9CDn6ofs3c=</latexit> <latexit sha1_base64="50AFhZ12WguaQSaO7crJI4F5CK8=">AAACRnicdVC9bxMxHPWFrxK+0nZksagqFQmCDUmbDkgFFsYikbZS7hL5HF/Oqs93sn9XEVn+q9hZWJjZGPgDWBhAiBVfjkoFwZMsPb0P2X5ppaQFQj5FnUuXr1y9tna9e+Pmrdt3eusbR7asDRdjXqrSnKTMCiW1GIMEJU4qI1iRKnGcnr5o/OMzYaws9WtYViIp2ELLTHIGQZr1khw/xbESGezEmWHcDeKcgYutXBTMT4fePdE+NnKRw/2payPUu6H3OGZVZco3mPbJLr7YwnrqHtJHQ/9g1tsi/X2yOxoMcEP2CB3hVqHDprjC1sEz8vbz+jt9OOt9jOclrwuhgStm7YSSChLHDEiuhO/GtRUV46dsISaBalYIm7jVDB5vB2WOs9KEowGv1IsNxwprl0UakgWD3P7tNeK/vEkN2ShxUlc1CM3bi7JaYShxsymeSyM4qGUgjBsZ3op5zsJUEJbvhhHOf4r/T44e9ynp01dhjeeoxRq6i+6hHUTRHjpAL9EhGiOO3qMv6Bv6Hn2IvkY/op9ttBP97myiP9BBvwBNVrSx</latexit> <latexit sha1_base64="50AFhZ12WguaQSaO7crJI4F5CK8=">AAACRnicdVC9bxMxHPWFrxK+0nZksagqFQmCDUmbDkgFFsYikbZS7hL5HF/Oqs93sn9XEVn+q9hZWJjZGPgDWBhAiBVfjkoFwZMsPb0P2X5ppaQFQj5FnUuXr1y9tna9e+Pmrdt3eusbR7asDRdjXqrSnKTMCiW1GIMEJU4qI1iRKnGcnr5o/OMzYaws9WtYViIp2ELLTHIGQZr1khw/xbESGezEmWHcDeKcgYutXBTMT4fePdE+NnKRw/2payPUu6H3OGZVZco3mPbJLr7YwnrqHtJHQ/9g1tsi/X2yOxoMcEP2CB3hVqHDprjC1sEz8vbz+jt9OOt9jOclrwuhgStm7YSSChLHDEiuhO/GtRUV46dsISaBalYIm7jVDB5vB2WOs9KEowGv1IsNxwprl0UakgWD3P7tNeK/vEkN2ShxUlc1CM3bi7JaYShxsymeSyM4qGUgjBsZ3op5zsJUEJbvhhHOf4r/T44e9ynp01dhjeeoxRq6i+6hHUTRHjpAL9EhGiOO3qMv6Bv6Hn2IvkY/op9ttBP97myiP9BBvwBNVrSx</latexit> <latexit sha1_base64="7ddxkNEdCFac8KSYj1fgOIzuhVI=">AAACRnicdVDNaxQxHM2sVev6tdWjl9BFqKBrorvtehCKvXhswW0LO7NLJpvZCc1khuQ3xSXkr/PSc2/+CV48KNKrmd0tVNEHgcf7IMlLKyUtEPI1at3auH3n7ua99v0HDx897mw9ObZlbbgY8VKV5jRlViipxQgkKHFaGcGKVImT9Oyg8U/OhbGy1J9gUYmkYHMtM8kZBGnaSXL8HsdKZLATZ4Zx149zBi62cl4wPxl491b72Mh5Di8mbhWh3g28xzGrKlN+xrRHdvHNFtYT94q+HviX006X9N6R3WG/jxuyR+gQrxQ6aIpLdNEah9POZTwreV0IDVwxa8eUVJA4ZkByJXw7rq2oGD9jczEOVLNC2MQtZ/D4eVBmOCtNOBrwUr3ZcKywdlGkIVkwyO3fXiP+yxvXkA0TJ3VVg9B8dVFWKwwlbjbFM2kEB7UIhHEjw1sxz1mYCsLy7TDC9U/x/8nxmx4lPXpEuvsf1nNsomdoG+0givbQPvqIDtEIcfQFfUM/0M/oIvoe/YquVtFWtO48RX+ghX4Dm7yxMw==</latexit> I loved this. Genetic algorithms, R^2, and Gaussian estimation are the kind of thing that they tell you you'll never need to use again once you graduate. But we were using them for a real world project! It was great.
But don't let that distract you; this accuracy made it easier for the software to kill people.
We'd been working with the project owner throughout this process.
Whenever we hit one of these milestones, we'd tell him. He'd be happy about it, but a question always came up. He wanted it to sniff for the signals put out by phones in addition to wifi hotspots. This is a much harder problem from a technical perspective. The functionality necessary to do this is a setting on the wireless network controller. Neither iPhone nor Android supported that option. We'd need to jailbreak or root the phone. The closest solution we ever found was a SourceForge project that seemed promising. We didn't fully understand its use and it wasn't well documented.
We told the project owner that we'd get to it later. None of us thought it was that important: we had the technology to find WiFi Access Points working. That was the goal. Each time we'd demonstrate the new exciting tech we'd built though, the same question came up.
We got wifi hotspots located! Great, does it find phones?
It's taking seconds instead of minutes! Great, does it find phones?
We looked into it finding phones, it seems unlikely but maybe! Ok, we'll come back to it.
Look at this genetic algorithm, it’s so much more accurate now! Great, does it find phones?
I had been distracted.
✓ 4ˆ � 5 3 n ◆ 1 5 ⇡ 1 . 06 ˆ � n � 1 / 5 , <latexit sha1_base64="RtGGtO8ILOPIq2pyroW4mEk+qu0=">AAACRnicdVBNixMxGH6nfq31q+rRS3ARVtCa0Xa3HoRVLx5XsLsLnWnJpJlO2ExmSN4RS8iv8id48ezNm1cvHhTxaqajsIo+EHh4PkjyZLWSFin9GPXOnD13/sLWxf6ly1euXhtcv3Foq8ZwMeWVqsxxxqxQUospSlTiuDaClZkSR9nJ89Y/ei2MlZV+hetapCVbaZlLzjBIi0FakCckUSLHnSQ3jLtRUjB0iZWrkvn52LtH2idGrgq8O3ddJPZu7D1JWF2b6g2Jh3SXnG4RPXf34wdjf28x2KbDx3R3MhqRluzReEI6JR63xQ2295/St58A4GAx+JAsK96UQiNXzNpZTGtMHTMouRK+nzRW1IyfsJWYBapZKWzqNjN4cicoS5JXJhyNZKOebjhWWrsus5AsGRb2b68V/+XNGswnqZO6blBo3l2UN4pgRdpNyVIawVGtA2HcyPBWwgsWpsKwfD+M8Pun5P/k8OEwpsP4ZVjjGXTYgltwG3Yghj3YhxdwAFPg8A4+w1f4Fr2PvkTfox9dtBf96tyEP9CDn6ofs3c=</latexit> <latexit sha1_base64="50AFhZ12WguaQSaO7crJI4F5CK8=">AAACRnicdVC9bxMxHPWFrxK+0nZksagqFQmCDUmbDkgFFsYikbZS7hL5HF/Oqs93sn9XEVn+q9hZWJjZGPgDWBhAiBVfjkoFwZMsPb0P2X5ppaQFQj5FnUuXr1y9tna9e+Pmrdt3eusbR7asDRdjXqrSnKTMCiW1GIMEJU4qI1iRKnGcnr5o/OMzYaws9WtYViIp2ELLTHIGQZr1khw/xbESGezEmWHcDeKcgYutXBTMT4fePdE+NnKRw/2payPUu6H3OGZVZco3mPbJLr7YwnrqHtJHQ/9g1tsi/X2yOxoMcEP2CB3hVqHDprjC1sEz8vbz+jt9OOt9jOclrwuhgStm7YSSChLHDEiuhO/GtRUV46dsISaBalYIm7jVDB5vB2WOs9KEowGv1IsNxwprl0UakgWD3P7tNeK/vEkN2ShxUlc1CM3bi7JaYShxsymeSyM4qGUgjBsZ3op5zsJUEJbvhhHOf4r/T44e9ynp01dhjeeoxRq6i+6hHUTRHjpAL9EhGiOO3qMv6Bv6Hn2IvkY/op9ttBP97myiP9BBvwBNVrSx</latexit> <latexit sha1_base64="50AFhZ12WguaQSaO7crJI4F5CK8=">AAACRnicdVC9bxMxHPWFrxK+0nZksagqFQmCDUmbDkgFFsYikbZS7hL5HF/Oqs93sn9XEVn+q9hZWJjZGPgDWBhAiBVfjkoFwZMsPb0P2X5ppaQFQj5FnUuXr1y9tna9e+Pmrdt3eusbR7asDRdjXqrSnKTMCiW1GIMEJU4qI1iRKnGcnr5o/OMzYaws9WtYViIp2ELLTHIGQZr1khw/xbESGezEmWHcDeKcgYutXBTMT4fePdE+NnKRw/2payPUu6H3OGZVZco3mPbJLr7YwnrqHtJHQ/9g1tsi/X2yOxoMcEP2CB3hVqHDprjC1sEz8vbz+jt9OOt9jOclrwuhgStm7YSSChLHDEiuhO/GtRUV46dsISaBalYIm7jVDB5vB2WOs9KEowGv1IsNxwprl0UakgWD3P7tNeK/vEkN2ShxUlc1CM3bi7JaYShxsymeSyM4qGUgjBsZ3op5zsJUEJbvhhHOf4r/T44e9ynp01dhjeeoxRq6i+6hHUTRHjpAL9EhGiOO3qMv6Bv6Hn2IvkY/op9ttBP97myiP9BBvwBNVrSx</latexit> <latexit sha1_base64="7ddxkNEdCFac8KSYj1fgOIzuhVI=">AAACRnicdVDNaxQxHM2sVev6tdWjl9BFqKBrorvtehCKvXhswW0LO7NLJpvZCc1khuQ3xSXkr/PSc2/+CV48KNKrmd0tVNEHgcf7IMlLKyUtEPI1at3auH3n7ua99v0HDx897mw9ObZlbbgY8VKV5jRlViipxQgkKHFaGcGKVImT9Oyg8U/OhbGy1J9gUYmkYHMtM8kZBGnaSXL8HsdKZLATZ4Zx149zBi62cl4wPxl491b72Mh5Di8mbhWh3g28xzGrKlN+xrRHdvHNFtYT94q+HviX006X9N6R3WG/jxuyR+gQrxQ6aIpLdNEah9POZTwreV0IDVwxa8eUVJA4ZkByJXw7rq2oGD9jczEOVLNC2MQtZ/D4eVBmOCtNOBrwUr3ZcKywdlGkIVkwyO3fXiP+yxvXkA0TJ3VVg9B8dVFWKwwlbjbFM2kEB7UIhHEjw1sxz1mYCsLy7TDC9U/x/8nxmx4lPXpEuvsf1nNsomdoG+0givbQPvqIDtEIcfQFfUM/0M/oIvoe/YquVtFWtO48RX+ghX4Dm7yxMw==</latexit> � � = arctan q (cos � 2 · sin( � � )) 2
Does it find phones.
This was never about finding better wifi. It was always about finding phones. Phones carried by people. I was building a tool for the military to find people based on where their phones where, and shoot them.
I tried to rationalize this then. The military is in place to protect Truth, Justice, and the American Way. But this was the same time that we found out the government had been spying on Americans in the US with drones. The military is a tool of the government, and it seemed like we couldn’t trust the government as well as we’d thought.
I didn't want to be a part of building something used to kill people, especially since I knew I'd never know who used against or have a say in that decision.
I rationalize it now too. We were interns, and we didn't even have clearance. The projects this company did for the government were classified Top Secret. I wasn't allowed to know what they were. My code probably got thrown away and forgotten. Probably.
This was an extreme example of code used in a way that the creator did not intend it. The project owner conveniently left out its purpose when explaining the goals. I conveniently didn't focus too much on that part. It was great pay for me at the time. It was a great project. Maybe I just didn't want to know what it would be used for. I got distracted.
Distracted by the technology. It would be just as easy to get distracted by the cool framework a company used, the great design of the app that would look good on your portfolio, or some really nice o ffi ce amenities, or that others are doing this so it’s probably ok.
There are other examples of when code is used in ways it wasn't intended, and of code that does bad things. Most aren't as black and white. The unifying factor in all of them is that a developer wrote the code that did these unethical or immoral things. As a profession, we have a superpower: we can make computers do things.
We build tools, and ultimately some responsibility lies with us to think through how those tools will be used.
Not just what their intention is, but also what misuses might come out of them. None of us wants to build things that will be used for evil.
So how can we carefully consider the potential impacts of our software projects? Honestly, I don't have any answers to this.
I don't think that there really is a universal answer yet, because if we had it I have to believe we'd not be building these dangerous pieces of software.
I do have a couple of ideas though.
The first is to add to the planning process a step where we come up with the worst possible uses of our software.
In opting in folks to an email list by default, the worst case might be that we send them a bunch of unwanted email and they unsubscribe. Maybe they even stop being a customer.
"Am I willing to sell my hypothetical startup's soul for a bigger mailing list, when that might be all that keeps the company afloat? Yeah, no problem." That makes sense to me. I don't think it's the best practice, but in the end it's not physically hurting anyone.
If I had sat down and thought about what the wifi location app could be used for in the worst case, I would have come to a very different conclusion.
Another, the thing that I think I should have done, and that we can all do more of, is to simply not take requests at face value.
The project owner at the Defense contractor I worked at didn't spell out what the reason for the code was. But at least in retrospect, it wasn't a big leap of logic. "We're going to build an app to find wifi signals" is all true, but it's not the whole truth. Asking them, or myself, "why" enough times probably would have led me to a much earlier understanding. Why? To find the sources. Why? To go to them. Why?
It’s a major problem when we’re given so much power in tech, but we’re not doing anything to ensure that we use it safely. Thinking about what we’re doing and being careful not to build things that can be used maliciously is really important.
Don't get distracted by deadlines and feature requests. Think about the consequences of what you're building. Build in safeguards to prevent misuse, or don't build it at all because it's too dangerous.
I'm asking you to do something about this. It's only fair that we talk a bit about how and when to take a stand.
Before saying no then, there are some things to consider:
Can I afford to leave here without a job financially?
Am I likely to be able to rely on my network to get me another job?
Have I built up a trust with my employer where I can go to them with this type of thing and feel confident that I'll be heard out instead of summarily shown the door?
The answer to these questions was no for me in 2011. Sometimes, something is important enough to you that you should still do something.There's a lot that goes into these decisions and they have consequences. I'd like to think that I would still say no.
Different scenarios need di ff erent ways of analyzing what you should do. In some cases, the right thing to do say nothing and build the product. It isn't a simple thing to make this decision.
But don't get distracted by having to think through it. Sometimes your code can kill people.
How I Built Software to Kill People A cautionary tale by @calebthompson
https://www.calebthompson.io/talks/dont-get-distracted Obviously I don’t have slides, so I’ve uploaded basically the script of this talk, along with some suggestions for further reading, to this URL. If you’re looking for something to share, I’d love it if you would use this link.
In 2011, with a team of interns at a Department of Defense contractor, I created a Wi-Fi geolocation app to locate hotspots. It could find the location in 3D space of every hotspot near you in seconds. We made formulas to model signal strength and probable distances. We used machine learning to optimize completion time and accuracy.
I was so caught up in the details that it took me months to see it would be used to kill people. What do we do when we discover that we’re building something immoral or unethical? How can we think through the uses of our software to avoid this problem entirely?
for free. You
can too.