The security issue that killed a financial product launch

A presentation at Agile Testing Days 2018 in November 2018 in Potsdam, Germany by nicola sedgwick

https://www.agiletestingdays.com https://twitter.com/nicolasedgwick http://www.nicola-sedgwick.com

crowd
Photo credit Rob Curran on Unsplash

crowd - bounty.
Image credit https://internetbugbounty.org/

crowd - reward
Photo credit Christian Dubovan on Unsplash

crowd - professionals
Photo credit Hello I'm Nik on Unsplash

story
Photo credit Sharon McCutcheon on Unsplash

story - ethics
Photo credit Cristian Newman on Unsplash

story - vulnerable
Photo credit Mihály Köles on Unsplash

challenge
Photo credit Luke van Zyl on Unsplash

challenge - shopping
Photo credit rawpixel on Unsplash

challenge - practicalities
Photo credit Fancycrave on Unsplash

challenge - reputation
Photo credit Jon Tyson on Unsplash

challenge - competition
Photo credit Patryk Sobczak on Unsplash

situation
Photo credit Matt Botsford on Unsplash

situation - owasp
credit https://www.owasp.org

situation - tools
Photo credit Adam Sherez on Unsplash

situation - understanding
Photo credit John Carlisle on Unsplash

analysis
Photo credit Luke van Zyl on Unsplash

analysis - protested
Photo credit Robert Hickerson on Unsp

analysis - impenetrable
Photo credit Ben Hershey on Unsplash

analysis - landscape
Photo credit Luo Lei on Unsplash

analysis - maze
Photo credit Wim Arys on Unsplash

analysis - sense
Photo credit Vladislav Klapin on Unspla

hacking
Photo credit Markus Spiske on Unsplash

hacking - vulnerability
Photo credit Hans-Peter Gauster on Unsplash

hacking - system
Photo credit rawpixel on Unsplash

hacking - transmission
Photo credit Jack Price-Burns on Unsplash

hacking - breach
Photo credit Ben Hershey on Unsplash

disbelief
Photo credit Jonathan Hoxmark on Unsplash

disbelief - halt
Photo credit Kai Pilger on Unsplash

disbelief - denied
Photo credit B J on Unsplash

disbelief - perhaps
Photo credit Mike Wilson on Unsplash

repetition
Photo credit Tine Ivanič on Unsplash

repetition - footsteps
Photo credit eberhard grossgasteiger o

repetition - payments
Photo credit Ales Nesetril on Unsplash

advice
Photo credit Melinda Gimpel on Unsplash

advice - trouble
Photo credit Ye Jinghan on Unsplash

advice - gifts
Photo credit freestocks.org on Unsplash

realisation
Photo credit Jez Timms on Unsplash

realisation - broken
Photo credit Stephanie Watters Flores on Unsplash

outcome
Photo credit Taskin Ashiq on Unsplash

outcome - bounty
Photo credit Brian Mann on Unsplash

summary
this is not an isolated situation
tools assist testing; tools don’t test
think critically during design
always return to the beginning

takeaways
• You already have the skill you need to find security issues … your brain! • Critical thinking skills are perfect for locating security problems. • Engage security assessment as part of architecture planning and throughout development. www.agiletestingdays.com |
A g i l eT D
nicolasedgwick | www.nicola-sedgwick.com

nicola sedgwick
@nicolasedgwick

1 / 46

That was missed by the professional penetration testers and security 'experts'.

Security issues can be identified using the stock-and-trade critical thinking skills of a tester.

Some time ago I had the pleasure of taking part in a security bug hunt for a new financial product. This was a product ready to go to market, a product that had passed all penetration tests and was now being handed to a crowd of external testers for a final attempt to 'hack' the product.

Against all their confidence I was able to 'hack' that product and use funds to which I should not have had access. However, once I reported the vulnerability, I wasn't believed and I was asked to repeat the 'hack' multiple times until the 'experts' believed I was achieving what I was reporting - they simply couldn't believe that their penetration test result was wrong.

Like many security talks I will tell you all about the tool I used to perform this 'hack'; Unlike many security talks this is not a tool you can install, rent or puchase - because it's my brain, but your brain is capable of doing the same.

Key Learnings

You already have the skills to find security problems
Critical thinking skills are perfect for locating security problems
Engage security assessment as part of architecture planning, not just at the end of the release

Resources

The following resources were mentioned during the presentation or are useful additional information.

Agile Testing Days 2018
Unsplash

Source of free-to-use high resolution imagery

Buzz and feedback

Here’s what was said about this presentation on social media.

Wonderful talk, slides and storytelling thank you @nicolasedgwick #AgileTD pic.twitter.com/1Mndr8nWJp
— Anne-Marie Charrett (@charrett) November 15, 2018
Seeing @nicolasedgwick speak for the first time! Rofl she’s a rocket scientist! Heh. 🚀🧪 💻 Rocket science is easy, it’s rocket recovery that’s hard. #AgileTD
— Lanette Creamer (@lanettecream) November 15, 2018
So many great storytellers at #agiletd, @nicolasedgwick told her story so incredibly well, was a pleasure to listen to it and a strong reminder that everyone with a critical thinking can play an important role in security testing, thank you! pic.twitter.com/hj1MF13u1X
— Gerald Mücke (@gmuecke) November 15, 2018
@nicolasedgwick is on ;) welcome to a story on how a product launch could be "killed" by a security issue :)@AgileTD #agileTD pic.twitter.com/QZxijcanRE
— alt the tester (@alt_lv) November 15, 2018
@nicolasedgwick talking about #bug #bounties and #security #AgileTD pic.twitter.com/8sGm2Y2dh9
— Rhian Lewis (@rhian_is) November 15, 2018
Security testers are a bit scary to those of us who aren’t security testers. She was picked to be “fresh eyes”. Payday loans are being described well by @nicolasedgwick & how ethically hard it is to test some things. #AgileTD
— Lanette Creamer (@lanettecream) November 15, 2018
@nicolasedgwick taking about how a security issue killed a financial product. We all have the skills needed for security testing ! @AgileTD #agileTD pic.twitter.com/2w5mXCngIK
— Claire Reckless (@clairereckless) November 15, 2018
You were amazing and such great storyteller, please give more talks :)
— Anne-Marie Charrett (@charrett) November 15, 2018
I’m blessed to be friends with so many fantastic #womenintesting @nicolasedgwick is one of them! She talks us through her issues when reporting security issues as part of a crowd sourced bug bounty @AgileTD #agiletd pic.twitter.com/YZ3KTWaXPX
— Dan Billing (@TheTestDoctor) November 15, 2018
Her husband thinks she is hacking & plugging into the matrix when she says she found a security bug. @nicolasedgwick #AgileTD
— Lanette Creamer (@lanettecream) November 15, 2018
The ethics of testing Pay Day loan software for @nicolasedgwick was rife with conflict and challenges, as these companies predate on vulnerable and poor people. It’s a big deal in the U.K. in these days of #austerity @agiletd #agiletd pic.twitter.com/0Z5UYzsniB
— Dan Billing (@TheTestDoctor) November 15, 2018
The exposure of personal and payment data was a real risk to the product that @nicolasedgwick was testing @agiletd #agiletd pic.twitter.com/a745SuY6bV
— Dan Billing (@TheTestDoctor) November 15, 2018
So important: if you're security-testing a financial product in the wild with your own details, make sure it doesn't impact your credit report! @nicolasedgwick at #AgileTD pic.twitter.com/0QnLfzdLqd
— Rhian Lewis (@rhian_is) November 15, 2018
Testing in the wild with real credit checks and actually impacting their own accounts. If the purchase info can be seen or can be reused, it’s a bug. Security crowd testing is like a herd of seagulls! Competition & not collab. #AgileTD @nicolasedgwick
— Lanette Creamer (@lanettecream) November 15, 2018
Rather than focusing on obvious security vulnerabilities, @nicolasedgwick focussed on the gaps where code can be flawed, by modelling her situations. Others could focus on the OWASP Top 10. Think of it as feeling and senses, personas, rather than bugs and flaws. @AgileTD #agiletd pic.twitter.com/zXPR0ohCke
— Dan Billing (@TheTestDoctor) November 15, 2018
If I could understand the system, I could find the possible miscommunications & gaps, or workarounds. I had no admin or server side info. The payments side is pretty mature. The clients weren’t going to pay for bugs in the payment side. Impenetrable? @nicolasedgwick #AgileTD
— Lanette Creamer (@lanettecream) November 15, 2018
By not using the payment gateway, the breach exploited the browser which was vulnerable to observing the credit card info via a browser extension. @nicolasedgwick @AgileTD #agiletd pic.twitter.com/VVSI9rOff1
— Dan Billing (@TheTestDoctor) November 15, 2018
They weren’t hooked into the payment gateway! She just had to show all form fields. They said her probable bug wasn’t a bug. She had to wait for a physical shipment to get past their denial. The company would not listen to her. They doubted her. @nicolasedgwick #AgileTD
— Lanette Creamer (@lanettecream) November 15, 2018
@nicolasedgwick had to get everything in writing to avoid possible legal trouble. Eventually only after a pro reproduced it would they believe her. This is so often the case for me too. My bugs are valid only if 2 men can reproduce it first. #AgileTD She won the bounty & a bonus!
— Lanette Creamer (@lanettecream) November 15, 2018
Because she wasn’t a “security expert”, @nicolasedgwick wasn’t believed. Nicola’s lawyer husband supported her on the legal implications of what and how she was testing, as there were potential legal risks and personal financial loss @AgileTD #agiletd pic.twitter.com/C5fQzyK0Ov
— Dan Billing (@TheTestDoctor) November 15, 2018
'I wanted to know when I was crossing the line into illegal behaviour.. I had to get everything in writing' - great #security #test talk by @nicolasedgwick whose winning #bug bounty discovery stopped a product launch! #AgileTD
— Rhian Lewis (@rhian_is) November 15, 2018
A fundamental flaw in the architecture led to the product not being launched. @nicolasedgwick helped to prevent an insecure application from being released! @AgileTD #agiletd pic.twitter.com/3JA3QXDthU
— Dan Billing (@TheTestDoctor) November 15, 2018
@nicolasedgwick had to reproduce this 5x!!! The steps weren’t changing. I know this feeling she describes so well. She repeated with a video but got more & more uncomfortable with each repeat. Trust was a problem here. @nicolasedgwick #AgileTD
— Lanette Creamer (@lanettecream) November 15, 2018
Don’t just focus on the tools, use your brain, focus on the problem, think critically, return to first principles of product design with security. @nicolasedgwick @AgileTD #agiletd pic.twitter.com/UFPqxQBDU9
— Dan Billing (@TheTestDoctor) November 15, 2018
Testing, even security testing, requires critical thinking! Pull your head out of your tools. Sanity check that you have the big picture. You have the skills to do security testing! It’s not a phase-security Testing should be continuous. @nicolasedgwick #AgileTD
— Lanette Creamer (@lanettecream) November 15, 2018
@nicolasedgwick - It's all is going to be fine xD @AgileTD #AgileTD pic.twitter.com/Nq5vJuIJac
— alt the tester (@alt_lv) November 15, 2018
Thanks @nicolasedgwick for the great story :)@AgileTD #AgileTD pic.twitter.com/X7qhJCGoAV
— alt the tester (@alt_lv) November 15, 2018

The security issue that killed a financial product launch

crowd

crowd - bounty.

crowd - reward

crowd - professionals

story

story - ethics

story - vulnerable

challenge

challenge - shopping

challenge - practicalities

challenge - reputation

challenge - competition

situation

situation - owasp

situation - tools

situation - understanding

analysis

analysis - protested

analysis - impenetrable

analysis - landscape

analysis - maze

analysis - sense

hacking

hacking - vulnerability

hacking - system

hacking - transmission

hacking - breach

disbelief

disbelief - halt

disbelief - denied

disbelief - perhaps

repetition

repetition - footsteps

repetition - payments

advice

advice - trouble

advice - gifts

realisation

realisation - broken

outcome

outcome - bounty

summary

takeaways

A g i l eT D

Link for this presentation:

HTML code for embedding:

Share on social media:

Agile Testing Days 2018

Unsplash