Are logs a software engineer’s best friend? Yes — follow these best practices

Are logs a software engineer’s best friend? Yes — follow these best practices Geshan Manandhar Senior Software Engineer (THE ICONIC) @geshan

whoami ◉ Senior Software Engineer ○ Tech solution provider ◉ Agile follower ◉ Conditional microservices believer

I am from Kathmandu, Nepal

I work at THE ICONIC

Agenda ◉ ◉ ◉ ◉ ◉ ◉ What is application log? Why log from the application level Tools of the trade — How? Logging Best practices Key Takeaways Demo

What is application log?

“Logs are the stream of aggregated, time-ordered events collected from the output streams of all running processes and backing services.” - 12Factor.net “ Source: https://www.12factor.net/logs

“An application log is a file of events that are logged by a software application. It contains errors, informational events and warnings.” - Technopedia “ Source: https://www.techopedia.com/definition/1819/application-log

Application logs are like having the right reading glasses

Why log from application level?

“The feature I deployed to production last week was working fine till yesterday, but now I have no idea why it is not working!” - One of us “

Logs help to pinpoint the exact place where the software is not working as expected

To stop speculating and figure out the real data and metrics on any env

Logs are one of the ways your app communicates back to you

Even if you land into a mess, you know a way to sort things fast (MTTR)

Having proper logs is like having a torch in a dark place

Tools of the trade — How?

Please don’t hammer the screw :)

Simplified components of logging Application generates logs Local File / Stdout Logs are ingested Using a library or over https Software that parses, analyzes and shows logs Logs are shipped/ingested with an agent or log shipper Software Engineer views/searches logs

Use a trusted logging library Language Library Github stars PHP Monolog ~16.0k TypeScript/JS Winston ~14.2k Native N/A Python Note: Don’t forget about monolog handlers and formatters :)

Use a trustworthy software to view and search your logs Self Hosted SAAS Cloud Provider Based

No tail -f …, a good UI to view/search logs will make your life a lot easier

Logging best practices

Log information optimally Too Much = noise, too less = inadequate information

RFC-5425 (8 Levels) Emergency Alert Critical Error Warning Notice Info Debug Always follow logs severity standards Emergency = phone rings at 1 AM, Info = good to know

Structure your logs Structured logs are easier to parse and search, always provide context.

Don’t hamper speed and performance Write logs locally or async as far as possible, may be use a queue and consumer.

Add traceability for microservices Same request ID travels across services. Ability for distributed tracing.

Do not log sensitive information Like passwords, credit card information, phone numbers etc

Key Takeaways

Start logging today

It is easier than you think and benefits outweigh the cost

Logs can reveal things you didn’t know existed at unexpected sources

Logging > error tracking, it keeps a track of your application health

Find the right tools, invest in a log management software that suits your needs

Logs with optimally configured alerts are very powerful

Demo ● https://github.com/geshan/laravel6-logging ● Pull Request - PR

Logs are an integral part of building happy, robust and scalable software

Thanks! Any questions? Later :) THE ICONIC is hiring - Engineering Manager You can find me at ◉ ◉ ◉ @geshan (geshan@gmail.com) Geshan.com.np Slides: bit.ly/logging-lau Presentation template by SlidesCarnival

Credits/References ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ ◉ https://pixabay.com/en/primate-ape-thinking-mimic-view-1019101/ https://pixabay.com/photos/batch-dry-firewood-forestry-logs-1868104/ https://www.techopedia.com/definition/1819/application-log https://www.scalyr.com/blog/application-logging-practices-adopt/ https://unsplash.com/photos/4Bs9kSDJsdc https://unsplash.com/photos/FtZL0r4DZYk https://pixabay.com/photos/calm-water-clear-drink-glass-2315559/ https://pixabay.com/photos/backdrop-block-brick-building-21534/ https://www.afcent.af.mil/News/Art/igphoto/2000289219/ https://unsplash.com/photos/MS0q8GqjVrY https://pixabay.com/photos/tools-work-repair-hammer-2145770/ https://pixabay.com/photos/needle-in-a-haystack-needle-haystack-1752846/ https://pixabay.com/photos/building-zen-man-boy-pyramid-game-2665812/ https://unsplash.com/photos/0vsk2_9dkqo https://unsplash.com/photos/pUT2Ujm6FP4 https://unsplash.com/photos/ZaGcU6BxJEc https://peter.bourgon.org/blog/2016/02/07/logging-v-instrumentation.html https://www.loggly.com/blog/30-best-practices-logging-scale/ https://geshan.com.np/blog/2015/08/importance-of-logging-in-your-applications/ https://www.scalyr.com/media-gallery-4/ https://aws.amazon.com/blogs/developer/php-application-logging-with-amazon-cloudwatch-logs-and-monolog/ https://raygun.com/blog/what-is-mttr/ https://pixabay.com/photos/pc-computer-stethoscope-technology-3540378/ https://krebsonsecurity.com/2019/03/facebook-stored-hundreds-of-millions-of-user-passwords-in-plain-text-for-years/