A presentation at PyGotham in in New York, NY, USA by Aaron Bassett
Can you keep a secret? @aaronbassett
Aaron Bassett Developer Advocate
import nexmo client = nexmo.Client( key=”a925db1ar392”, secret=”01nd637fn29oe31mc721” ) client.send_message({ “from”: “Python”, “to”: “447700900981”, “text”: “Hello world” })
import nexmo client = nexmo.Client( key=”a925db1ar392”, secret=”01nd637fn29oe31mc721” ) client.send_message({ “from”: “Python”, “to”: “447700900981”, “text”: “Hello world” })
import nexmo NEXMO_KEY = “a925db1ar392” NEXMO_SECRET = “01nd637fn29oe31mc721” MY_NUMBER = “447700900981” client = nexmo.Client(key=NEXMO_KEY, secret=NEXMO_SECRET) client.send_message({ “from”: “Python”, “to”: MY_NUMBER, “text”: “Hello world” })
import nexmo NEXMO_KEY = “a925db1ar392” NEXMO_SECRET = “01nd637fn29oe31mc721” MY_NUMBER = “447700900981” client = nexmo.Client(key=NEXMO_KEY, secret=NEXMO_SECRET) client.send_message({ “from”: “Python”, “to”: MY_NUMBER, “text”: “Hello world” })
import nexmo NEXMO_KEY = “a925db1ar392” NEXMO_SECRET = “01nd637fn29oe31mc721” MY_NUMBER = “447700900981” client = nexmo.Client(key=NEXMO_KEY, secret=NEXMO_SECRET) client.send_message({ “from”: “Python”, “to”: MY_NUMBER, “text”: “Hello world” })
git add . git commit -m “wip” git push
“the median time to discovery was 20 seconds, with times ranging from half a second to over 4 minutes” – How Bad Can It Git? Characterizing Secret Leakage in Public GitHub Repositories
Not safe, BUT VERY easy
Functionality Security Usability
Low Friction
Easy to implement
12 Factor Apps
I. Codebase II. Dependencies III. Config IV. Backing services V. Build, release, run VI. Processes VII. Port binding VIII. Concurrency IX. Disposability X. Dev/prod parity XI. Logs XII. Admin processes
III. Config
“A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials.” –The twelve-factor app
environment variables
import os import nexmo client = nexmo.Client( key=os.environ[“NEXMO_KEY”], secret=os.environ[“NEXMO_SECRET”] ) client.send_message({ “from”: “Python”, “to”: os.environ[“MY_NUMBER”], “text”: “Hello world” })
import os import nexmo client = nexmo.Client( key=os.environ[“NEXMO_KEY”], secret=os.environ[“NEXMO_SECRET”] ) client.send_message({ “from”: “Python”, “to”: os.environ[“MY_NUMBER”], “text”: “Hello world” })
import os import nexmo client = nexmo.Client( key=os.environ.get(“NEXMO_KEY”), secret=os.environ.get(“NEXMO_SECRET”) ) client.send_message({ “from”: “Python”, “to”: os.environ.get(“MY_NUMBER”), “text”: “Hello world” })
import os import nexmo client = nexmo.Client( key=os.environ.get(“NEXMO_KEY”), secret=os.environ.get(“NEXMO_SECRET”) ) client.send_message({ “from”: “Python”, “to”: os.environ.get(“MY_NUMBER”), “text”: “Hello world” })
import os import nexmo client = nexmo.Client( key=os.getenv(“NEXMO_KEY”), secret=os.getenv(“NEXMO_SECRET”), ) client.send_message( { “from”: “Python”, “to”: os.getenv(“MY_NUMBER”), “text”: “Hello world”, } )
import os import nexmo client = nexmo.Client( key=os.getenv(“NEXMO_KEY”), secret=os.getenv(“NEXMO_SECRET”), ) client.send_message( { “from”: “Python”, “to”: os.getenv(“MY_NUMBER”), “text”: “Hello world”, } )
Creating environment variables
}
}
}
“direnv is an extension for your shell. It augments existing shells with a new feature that can load and unload environment variables depending on the current directory.” – direnv.net
$ echo export NEXMO_KEY=a925db1ar392 > .envrc .envrc is not allowed $ direnv allow . direnv: reloading direnv: loading .envrc direnv export: +NEXMO_KEY $ cd $.. direnv: unloading
$ echo export NEXMO_KEY=a925db1ar392 > .envrc .envrc is not allowed $ direnv allow . direnv: reloading direnv: loading .envrc direnv export: +NEXMO_KEY $ cd $.. direnv: unloading
$ echo export NEXMO_KEY=a925db1ar392 > .envrc .envrc is not allowed $ direnv allow . direnv: reloading direnv: loading .envrc direnv export: +NEXMO_KEY $ cd $.. direnv: unloading
$ echo export NEXMO_KEY=a925db1ar392 > .envrc .envrc is not allowed $ direnv allow . direnv: reloading direnv: loading .envrc direnv export: +NEXMO_KEY $ cd $.. direnv: unloading
$ echo export NEXMO_KEY=a925db1ar392 > .envrc .envrc is not allowed $ direnv allow . direnv: reloading direnv: loading .envrc direnv export: +NEXMO_KEY $ cd $.. direnv: unloading
$ echo export NEXMO_KEY=a925db1ar392 > .envrc .envrc is not allowed $ direnv allow . direnv: reloading direnv: loading .envrc direnv export: +NEXMO_KEY $ cd $.. direnv: unloading
Ignore .envrc echo .envrc > ~/.gitignore git config $—global core.excludesfile ~/.gitignore
Example .envrc grep -ohr “^export .*=” .envrc > .envrc.example
Sharing values & Files
import os import nexmo client = nexmo.Client( application_id=os.getenv(“NEXMO_APPLICATION_ID”), private_key=os.getenv(“NEXMO_PRIVATE_KEY”, “./private.key”), ) response = client.create_call( { “to”: [{“type”: “phone”, “number”: os.getenv(“TO_NUMBER”)}], “from”: {“type”: “phone”, “number”: os.getenv(“NEXMO_NUMBER”)}, “answer_url”: [f”{os.getenv(‘VAPI_URL’)}/answer”], } )
import os import nexmo client = nexmo.Client( application_id=os.getenv(“NEXMO_APPLICATION_ID”), private_key=os.getenv(“NEXMO_PRIVATE_KEY”, “./private.key”), ) response = client.create_call( { “to”: [{“type”: “phone”, “number”: os.getenv(“TO_NUMBER”)}], “from”: {“type”: “phone”, “number”: os.getenv(“NEXMO_NUMBER”)}, “answer_url”: [f”{os.getenv(‘VAPI_URL’)}/answer”], } )
import os import nexmo client = nexmo.Client( application_id=os.getenv(“NEXMO_APPLICATION_ID”), private_key=os.getenv(“NEXMO_PRIVATE_KEY”, “./private.key”), ) response = client.create_call( { “to”: [{“type”: “phone”, “number”: os.getenv(“TO_NUMBER”)}], “from”: {“type”: “phone”, “number”: os.getenv(“NEXMO_NUMBER”)}, “answer_url”: [f”{os.getenv(‘VAPI_URL’)}/answer”], } )
git-secret.io
“git-secret encrypts files and stores them inside the git repository, so you will have all the changes for every commit.” –git-secret.io
Safe & Easy
Safe & Easy-ish
PGP
$ git secret init git-secret: init created: ‘/myproject/.gitsecret/’
$ git secret tell me@aaronbassett.com gpg: marginals needed: 3 completes needed: 1 trust model: pgp gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u gpg: next trustdb check due at 2021-09-30 gpg: keybox ‘/myproject/.gitsecret/keys/pubring.kbx’ created gpg: /myproject/.gitsecret/keys/trustdb.gpg: trustdb created git-secret: done. me@aaronbassett.com added as user(s) who know the secret.
$ git secret add private.key git-secret: these files are not in .gitignore: private.key git-secret: auto adding them to .gitignore git-secret: 1 item(s) added.
$ cat .gitignore .gitsecret/keys/random_seed !*.secret private.key
$ ls .git .gitignore .gitsecret private.key
$ git secret hide git-secret: done. 1 of 1 files are hidden.
$ ls .git .gitignore .gitsecret private.key private.key.secret
$ git secret reveal File ‘/myproject/private.key’ exists. Overwrite? (y/N) y git-secret: done. 1 of 1 files are revealed.
$ git secret whoknows me@aaronbassett.com $ git secret list private.key
$ git secret killperson me@aaronbassett.com git-secret: removed keys. git-secret: now [me@aaronbassett.com] do not have an access to the repository. git-secret: make sure to hide the existing secrets again. $ git secret reveal git-secret: abort: no public keys for users found. run ‘git secret tell email@address.
Git Secrets
“git-secrets scans commits, commit messages, and —no-ff merges to prevent adding secrets into your git repositories. If a commit, commit message, or any commit in a —no-ff merge history matches one of your configured prohibited regular expression patterns, then the commit is rejected.” – awslabs/git-secrets
$ git secrets $—register-aws $—global OK $ git secrets $—install ~/.git-templates/git-secrets ✓ Installed commit-msg hook to /Users/aaronbassett/.git-templates/gitsecrets/hooks/commit-msg ✓ Installed pre-commit hook to /Users/aaronbassett/.git-templates/gitsecrets/hooks/pre-commit ✓ Installed prepare-commit-msg hook to /Users/aaronbassett/.git-templates/ git-secrets/hooks/prepare-commit-msg $ git config $—global init.templateDir ~/.git-templates/git-secrets
$ git secrets $—register-aws $—global OK $ git secrets $—install ~/.git-templates/git-secrets ✓ Installed commit-msg hook to /Users/aaronbassett/.git-templates/gitsecrets/hooks/commit-msg ✓ Installed pre-commit hook to /Users/aaronbassett/.git-templates/gitsecrets/hooks/pre-commit ✓ Installed prepare-commit-msg hook to /Users/aaronbassett/.git-templates/ git-secrets/hooks/prepare-commit-msg $ git config $—global init.templateDir ~/.git-templates/git-secrets
$ git secrets $—register-aws $—global OK $ git secrets $—install ~/.git-templates/git-secrets ✓ Installed commit-msg hook to /Users/aaronbassett/.git-templates/gitsecrets/hooks/commit-msg ✓ Installed pre-commit hook to /Users/aaronbassett/.git-templates/gitsecrets/hooks/pre-commit ✓ Installed prepare-commit-msg hook to /Users/aaronbassett/.git-templates/ git-secrets/hooks/prepare-commit-msg $ git config $—global init.templateDir ~/.git-templates/git-secrets
$ git secrets $—register-aws $—global OK $ git secrets $—install ~/.git-templates/git-secrets ✓ Installed commit-msg hook to /Users/aaronbassett/.git-templates/gitsecrets/hooks/commit-msg ✓ Installed pre-commit hook to /Users/aaronbassett/.git-templates/gitsecrets/hooks/pre-commit ✓ Installed prepare-commit-msg hook to /Users/aaronbassett/.git-templates/ git-secrets/hooks/prepare-commit-msg $ git config $—global init.templateDir ~/.git-templates/git-secrets
providers
^[5KL][1-9A-HJ-NP-Za-km-z]{50,51}$ (xox[p|b|o|a]-[0-9]{12}-[0-9]{12}-[0-9]{12}-[a-z0-9]{32}) https:$//hooks.slack.com/services/T[a-zA-Z0-9_]{8}/B[a-zA-Z0-9_]{8}/[a-zA-Z0-9_]{24} EAACEdEose0cBA[0-9A-Za-z]+ [t|T][w|W][i|I][t|T][t|T][e|E][r|R].[1-9][0-9]+-[0-9a-zA-Z]{40} [t|T][w|W][i|I][t|T][t|T][e|E][r|R].[‘|”][0-9a-zA-Z]{35,44}[‘|”] AIza[0-9A-Za-z*-]{35} [0-9]+-[0-9A-Za-z]{32}*.apps*.googleusercontent*.com ya29*.[0-9A-Za-z*-_]+ [h|H][e|E][r|R][o|O][k|K][u|U].*[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12} (——-(BEGIN|END) PRIVATE KEY——-) (——-(BEGIN|END) RSA PRIVATE KEY——-)
$ git secrets $—add-provider $— cat /secret/file/patterns
(——-(BEGIN|END) RSA PRIVATE KEY——-)
ya29*.[0-9A-Za-z*-_]+
EAACEdEose0cBA[0-9A-Za-z]+
$ cd /secret/file/patterns $ ls crypto keys vendors
git secrets $—add-provider $— egrep -rhv “(^#|^$)” /secret/file/patterns
^[5KL][1-9A-HJ-NP-Za-km-z]{50,51}$ (xox[p|b|o|a]-[0-9]{12}-[0-9]{12}-[0-9]{12}-[a-z0-9]{32}) https:$//hooks.slack.com/services/T[a-zA-Z0-9_]{8}/B[a-zA-Z0-9_]{8}/[a-zA-Z0-9_]{24} EAACEdEose0cBA[0-9A-Za-z]+ [t|T][w|W][i|I][t|T][t|T][e|E][r|R].[1-9][0-9]+-[0-9a-zA-Z]{40} [t|T][w|W][i|I][t|T][t|T][e|E][r|R].[‘|”][0-9a-zA-Z]{35,44}[‘|”] AIza[0-9A-Za-z*-]{35} [0-9]+-[0-9A-Za-z]{32}*.apps*.googleusercontent*.com ya29*.[0-9A-Za-z*-_]+ [h|H][e|E][r|R][o|O][k|K][u|U].*[0-9A-F]{8}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{4}-[0-9A-F]{12} (——-(BEGIN|END) PRIVATE KEY——-) (——-(BEGIN|END) RSA PRIVATE KEY——-)
$ git add ‘private.key’ $ git commit -m “Adding some files I shouldn’t” private.key:1:——-BEGIN PRIVATE KEY——[ERROR] Matched one or more prohibited patterns Possible mitigations: - Mark false positives as allowed using: git config $—add secrets.allowed $$… - Mark false positives as allowed by adding regular expressions to .gitallowed at repository’s root directory - List your configured patterns: git config $—get-all secrets.patterns - List your configured allowed patterns: git config $—get-all secrets.allowed - List your configured allowed patterns in .gitallowed at repository’s root directory - Use $—no-verify if this is a one-time false positive
GITLEAKS
“Audit git repos for secrets. Gitleaks provides a way for you to find unencrypted secrets and other unwanted data types in git source code repositories” – zricethezav/gitleaks
I. A GIT Repo II. Github user III. Github organization IV. Github PR V. GitLab user VI. GitLab group
{ } “line”: “——-BEGIN PRIVATE KEY——-“, “commit”: “37f19780583f12fd2fb687f2d7d3840880e79c76”, “offender”: “——-BEGIN PRIVATE KEY——-“, “rule”: “PKCS8”, “info”: “——-BEGIN PRIVATE KEY——- regex match”, “commitMsg”: “wip backup “, “author”: “Joe Bloggs”, “email”: “jbloggs@example.com”, “file”: “app/private.key”, “repo”: “my-awesome-app”, “date”: “2019-09-14T12:26:10+08:00”, “tags”: “key, PKCS8”, “severity”: “”
gitleaks —github-org=MyOrg > /dev/null if [ $? -eq 0 ]; then echo “No Leaks” else nexmo sms 447700900235 Git is leaking $—confirm fi
gitleaks —github-org=MyOrg > /dev/null if [ $? -eq 0 ]; then echo “No Leaks” else nexmo sms 447700900235 Git is leaking $—confirm fi
gitleaks —github-org=MyOrg > /dev/null if [ $? -eq 0 ]; then echo “No Leaks” else nexmo sms 447700900235 Git is leaking $—confirm fi
gitleaks —github-org=MyOrg > /dev/null if [ $? -eq 0 ]; then echo “No Leaks” else nexmo sms 447700900235 Git is leaking $—confirm fi
@aaronbassett noti.st/aaronbassett
We’ve all been guilty of hard-coding secrets at some point. It’s just a quick hack, and you’ll definitely go back and tidy it up later. But then you forget, and it’s all too easy to git push your API keys to GitHub.
This easy to make mistake could end up costing you thousands of dollars, and with the median time to discovery for a secret key leaked to GitHub being 20 seconds you could end up compromised before you have a chance to correct your error.
In this talk, we’ll look at techniques that you can use personally and within your development teams to properly store, share, and manage your secrets, as easily as possible without disrupting your workflow.
The following resources were mentioned during the presentation or are useful additional information.
The following code examples from the presentation can be tried out live.
Here’s what was said about this presentation on social media.
for free. You
can too.