Release Your Refactoring Superpower

A presentation at ZendCon 2018 in October 2018 in Las Vegas, NV, USA by Adam Culp

Slide 1

Slide 1

Release Your Refactoring Superpower By: Adam Culp Twitter: @adamculp

Slide 2

Slide 2

Release Your Refactoring Superpower About me ● – OSS Contributor – PHP Certified – Zend Certification Advisory Board – PHP-Fig voting member (IBM i Toolkit) – Consultant at Zend Technologies – Organizer SoFloPHP (South Florida) – Organizer SunshinePHP (Miami) – Long distance (ultra) runner – Photography Enthusiast – 2 Judo Black Belt Instructor

Slide 3

Slide 3

Release Your Refactoring Superpower About me ● – OSS Contributor – PHP Certified – Zend Certification Advisory Board – PHP-Fig voting member (IBM i Toolkit) – Consultant at Zend Technologies – Organizer SoFloPHP (South Florida) – Organizer SunshinePHP (Miami) – Long distance (ultra) runner – – 3 I am the Photography Enthusiast PHP Ninja!!! Judo Black Belt Instructor

Slide 4

Slide 4

Release Your Refactoring Superpower ● Fan of iteration – 4 Pretty much everything requires iteration to do well: ● Long distance running ● Judo ● Development ● Evading project managers ● Refactoring!

Slide 5

Slide 5

Release Your Refactoring Superpower ● What Can I Do? – 5 Estimation

Slide 6

Slide 6

Release Your Refactoring Superpower ● 6 What Can I Do? – Estimation – Coding (actual refactoring)

Slide 7

Slide 7

Release Your Refactoring Superpower ● 7 What Can I Do? – Estimation – Coding (actual refactoring) – Algorithms

Slide 8

Slide 8

Release Your Refactoring Superpower ● 8 What Can I Do? – Estimation – Coding (actual refactoring) – Algorithms – Convince Business

Slide 9

Slide 9

Release Your Refactoring Superpower ● 9 What Can I Do? – Estimation – Coding (actual refactoring) – Algorithms – Convince Business – Silver Bullet

Slide 10

Slide 10

Release Your Refactoring Superpower ● Modernization? – How? ● 10 New infrastructure (servers, technology, etc.)

Slide 11

Slide 11

Release Your Refactoring Superpower ● Modernization? – 11 How? ● New infrastructure (servers, technology, etc.) ● New frameworks or libraries

Slide 12

Slide 12

Release Your Refactoring Superpower ● Modernization? – 12 How? ● New infrastructure (servers, technology, etc.) ● New frameworks or libraries ● New programming language

Slide 13

Slide 13

Release Your Refactoring Superpower ● Modernization? – 13 How? ● New infrastructure (servers, technology, etc.) ● New frameworks or libraries ● New programming language ● New DB

Slide 14

Slide 14

Release Your Refactoring Superpower ● Modernization? – – How? ● New infrastructure (servers, technology, etc.) ● New frameworks or libraries ● New programming language ● New DB Why? ● 14 Desire

Slide 15

Slide 15

Release Your Refactoring Superpower ● Modernization? – – 15 How? ● New infrastructure (servers, technology, etc.) ● New frameworks or libraries ● New programming language ● New DB Why? ● Desire ● Bored

Slide 16

Slide 16

Release Your Refactoring Superpower ● Modernization? – – 16 How? ● New infrastructure (servers, technology, etc.) ● New frameworks or libraries ● New programming language ● New DB Why? ● Desire ● Bored ● Perceived need

Slide 17

Slide 17

Release Your Refactoring Superpower ● Modernization? – – 17 How? ● New infrastructure (servers, technology, etc.) ● New frameworks or libraries ● New programming language ● New DB Why? ● Desire ● Bored ● Perceived need ● To gain something – Speed – Functionality

Slide 18

Slide 18

Release Your Refactoring Superpower ● Modernization? – – – How? ● New infrastructure (servers, technology, etc.) ● New frameworks or libraries ● New programming language ● New DB Why? ● Desire ● Bored ● Perceived need ● To gain something Speed – Functionality When? ● 18 – Next 6 months, year(s), decade

Slide 19

Slide 19

Release Your Refactoring Superpower ● Modernization? – – – 19 How? ● New infrastructure (servers, technology, etc.) ● New frameworks or libraries ● New programming language ● New DB Why? ● Desire ● Bored ● Perceived need ● To gain something – Speed – Functionality When? ● Next 6 months, year(s), decade ● Realistic time

Slide 20

Slide 20

Release Your Refactoring Superpower ● Modernization? – – – 20 How? ● New infrastructure (servers, technology, etc.) ● New frameworks or libraries ● New programming language ● New DB Why? ● Desire ● Bored ● Perceived need ● To gain something – Speed – Functionality When? ● Next 6 months, year(s), decade ● Realistic time ● NOW!

Slide 21

Slide 21

Release Your Refactoring Superpower ● 21 Rewrite FTW!

Slide 22

Slide 22

Release Your Refactoring Superpower ● Typical Loop – Business Responses ● 22 No time

Slide 23

Slide 23

Release Your Refactoring Superpower ● Typical Loop – 23 Business Responses ● No time ● No money

Slide 24

Slide 24

Release Your Refactoring Superpower ● Typical Loop – 24 Business Responses ● No time ● No money ● No need

Slide 25

Slide 25

Release Your Refactoring Superpower ● Typical Loop – 25 Business Responses ● No time ● No money ● No need ● Things are “good enough”

Slide 26

Slide 26

Release Your Refactoring Superpower ● 26 The Fix

Slide 27

Slide 27

Release Your Refactoring Superpower ● Case Study – Managing legacy system costs: A case study of a meta-assessment model to identify solutions in a large financial services company – 2017 (by James Crotty, Ivan Horrocks) https://www.sciencedirect.com/science/article/pii/S221083271630126 0#b0025 ● 2001 Brooke and Ramage defined legacy as: – 27 Old information system remaining in operation within an Organization

Slide 28

Slide 28

Release Your Refactoring Superpower ● Case Study – Managing legacy system costs: A case study of a meta-assessment model to identify solutions in a large financial services company – 2017 (by James Crotty, Ivan Horrocks) https://www.sciencedirect.com/science/article/pii/S221083271630126 0#b0025 ● 28 2001 Brooke and Ramage defined legacy as: – Old information system remaining in operation within an Organization – Business critical, resisting modification as failure would cause significant impact on business

Slide 29

Slide 29

Release Your Refactoring Superpower ● Case Study – Managing legacy system costs: A case study of a meta-assessment model to identify solutions in a large financial services company – 2017 (by James Crotty, Ivan Horrocks) https://www.sciencedirect.com/science/article/pii/S221083271630126 0#b0025 ● 29 2001 Brooke and Ramage defined legacy as: – Old information system remaining in operation within an Organization – Business critical, resisting modification as failure would cause significant impact on business – Based on outdated technology but critical day-to-day operations

Slide 30

Slide 30

Release Your Refactoring Superpower ● Case Study – Managing legacy system costs: A case study of a meta-assessment model to identify solutions in a large financial services company – 2017 (by James Crotty, Ivan Horrocks) https://www.sciencedirect.com/science/article/pii/S221083271630126 0#b0025 ● 30 2001 Brooke and Ramage defined legacy as: – Old information system remaining in operation within an Organization – Business critical, resisting modification as failure would cause significant impact on business – Based on outdated technology but critical day-to-day operations – Built when processing and storage was much more expensive

Slide 31

Slide 31

Release Your Refactoring Superpower ● Case Study – Managing legacy system costs: A case study of a meta-assessment model to identify solutions in a large financial services company – 2017 (by James Crotty, Ivan Horrocks) https://www.sciencedirect.com/science/article/pii/S221083271630126 0#b0025 ● 31 2001 Brooke and Ramage defined legacy as: – Old information system remaining in operation within an Organization – Business critical, resisting modification as failure would cause significant impact on business – Based on outdated technology but critical day-to-day operations – Built when processing and storage was much more expensive – Poorly documented

Slide 32

Slide 32

Release Your Refactoring Superpower ● Case Study – Managing legacy system costs: A case study of a meta-assessment model to identify solutions in a large financial services company – 2017 (by James Crotty, Ivan Horrocks) https://www.sciencedirect.com/science/article/pii/S221083271630126 0#b0025 ● 32 2001 Brooke and Ramage defined legacy as: – Old information system remaining in operation within an Organization – Business critical, resisting modification as failure would cause significant impact on business – Based on outdated technology but critical day-to-day operations – Built when processing and storage was much more expensive – Poorly documented – Lack of design

Slide 33

Slide 33

Release Your Refactoring Superpower ● Case Study – Modernization Drivers ● 33 Skillset shortages (old technologies)

Slide 34

Slide 34

Release Your Refactoring Superpower ● Case Study – 34 Modernization Drivers ● Skillset shortages (old technologies) ● Technical needs

Slide 35

Slide 35

Release Your Refactoring Superpower ● Case Study – 35 Modernization Drivers ● Skillset shortages (old technologies) ● Technical needs ● Business needs

Slide 36

Slide 36

Release Your Refactoring Superpower ● Case Study – 36 Modernization Drivers ● Skillset shortages (old technologies) ● Technical needs ● Business needs ● Personal bias

Slide 37

Slide 37

Release Your Refactoring Superpower ● Case Study – Cost Reduction Strategies ● 37 Ordinary maintenance

Slide 38

Slide 38

Release Your Refactoring Superpower ● Case Study – 38 Cost Reduction Strategies ● Ordinary maintenance ● Reverse engineering

Slide 39

Slide 39

Release Your Refactoring Superpower ● Case Study – 39 Cost Reduction Strategies ● Ordinary maintenance ● Reverse engineering ● Restructuring

Slide 40

Slide 40

Release Your Refactoring Superpower ● Case Study – 40 Cost Reduction Strategies ● Ordinary maintenance ● Reverse engineering ● Restructuring ● Re-engineering

Slide 41

Slide 41

Release Your Refactoring Superpower ● Case Study – 41 Cost Reduction Strategies ● Ordinary maintenance ● Reverse engineering ● Restructuring ● Re-engineering ● Migration

Slide 42

Slide 42

Release Your Refactoring Superpower ● Case Study – 42 Cost Reduction Strategies ● Ordinary maintenance ● Reverse engineering ● Restructuring ● Re-engineering ● Migration ● Discard

Slide 43

Slide 43

Release Your Refactoring Superpower ● Case Study – 43 Cost Reduction Strategies ● Ordinary maintenance ● Reverse engineering ● Restructuring ● Re-engineering ● Migration ● Discard ● Wrapping

Slide 44

Slide 44

Release Your Refactoring Superpower ● Case Study – 44 Cost Reduction Strategies ● Ordinary maintenance ● Reverse engineering ● Restructuring ● Re-engineering ● Migration ● Discard ● Wrapping ● Outsource?

Slide 45

Slide 45

Release Your Refactoring Superpower ● Case Study – 45 Cost Reduction Strategies ● Ordinary maintenance ● Reverse engineering ● Restructuring ● Re-engineering ● Migration ● Discard ● Wrapping ● Outsource? ● Freeze

Slide 46

Slide 46

Release Your Refactoring Superpower ● Case Study – 46 Cost Reduction Strategies ● Ordinary maintenance ● Reverse engineering ● Restructuring ● Re-engineering ● Migration ● Discard ● Wrapping ● Outsource? ● Freeze ● Carry On

Slide 47

Slide 47

Release Your Refactoring Superpower ● Case Study – 47 Cost Reduction Strategies ● Ordinary maintenance ● Reverse engineering ● Restructuring ● Re-engineering ● Migration ● Discard ● Wrapping ● Outsource? ● Freeze ● Carry On ● Replacement with commercial off-the-shelf software and discarding

Slide 48

Slide 48

Release Your Refactoring Superpower ● How Do We Know? – Measurement ● A Method for Assessing Legacy Systems for Evolution – 1998 (by Jane Ransom, Ian Sommerville, and Ian Warren) http://citeseerx.ist.psu.edu/viewdoc/download? doi=10.1.1.128.9889&rep=rep1&type=pdf – 48 Legacy = business critical = cost not justifiable

Slide 49

Slide 49

Release Your Refactoring Superpower ● How Do We Know? – Measurement ● 49 A Method for Assessing Legacy Systems for Evolution – 1998 (by Jane Ransom, Ian Sommerville, and Ian Warren) http://citeseerx.ist.psu.edu/viewdoc/download? doi=10.1.1.128.9889&rep=rep1&type=pdf – Legacy = business critical = cost not justifiable – Company and project specific

Slide 50

Slide 50

Release Your Refactoring Superpower ● How Do We Know? – Measurement ● 50 A Method for Assessing Legacy Systems for Evolution – 1998 (by Jane Ransom, Ian Sommerville, and Ian Warren) http://citeseerx.ist.psu.edu/viewdoc/download? doi=10.1.1.128.9889&rep=rep1&type=pdf – Legacy = business critical = cost not justifiable – Company and project specific – Continuously refined

Slide 51

Slide 51

Release Your Refactoring Superpower ● How Do We Know? – Measurement ● 51 A Method for Assessing Legacy Systems for Evolution – 1998 (by Jane Ransom, Ian Sommerville, and Ian Warren) http://citeseerx.ist.psu.edu/viewdoc/download? doi=10.1.1.128.9889&rep=rep1&type=pdf – Legacy = business critical = cost not justifiable – Company and project specific – Continuously refined – Gains depth of understanding of business

Slide 52

Slide 52

Release Your Refactoring Superpower ● How Do We Know? – Criteria ● Decision Model for Legacy Systems - 1999 (by . H. Bennett, M. Ramage, and M. Munro) - ftp://ftp.inf.pucrio.br/pub/docs/FomularioSolicitacoes/mariliaGFerreira-09-13-7.pdf – Based more upon organizational points ● 52 Boundary: the unit of analysis

Slide 53

Slide 53

Release Your Refactoring Superpower ● How Do We Know? – Criteria ● Decision Model for Legacy Systems - 1999 (by . H. Bennett, M. Ramage, and M. Munro) - ftp://ftp.inf.pucrio.br/pub/docs/FomularioSolicitacoes/mariliaGFerreira-09-13-7.pdf – Based more upon organizational points ● ● 53 Boundary: the unit of analysis Vision: global summary of the unit

Slide 54

Slide 54

Release Your Refactoring Superpower ● How Do We Know? – Criteria ● Decision Model for Legacy Systems - 1999 (by . H. Bennett, M. Ramage, and M. Munro) - ftp://ftp.inf.pucrio.br/pub/docs/FomularioSolicitacoes/mariliaGFerreira-09-13-7.pdf – Based more upon organizational points ● ● ● 54 Boundary: the unit of analysis Vision: global summary of the unit Logic: rationale for vision

Slide 55

Slide 55

Release Your Refactoring Superpower ● How Do We Know? – Criteria ● Decision Model for Legacy Systems - 1999 (by . H. Bennett, M. Ramage, and M. Munro) - ftp://ftp.inf.pucrio.br/pub/docs/FomularioSolicitacoes/mariliaGFerreira-09-13-7.pdf – Based more upon organizational points ● ● ● ● 55 Boundary: the unit of analysis Vision: global summary of the unit Logic: rationale for vision Structure: of the organisation

Slide 56

Slide 56

Release Your Refactoring Superpower ● How Do We Know? – Criteria ● Decision Model for Legacy Systems - 1999 (by . H. Bennett, M. Ramage, and M. Munro) - ftp://ftp.inf.pucrio.br/pub/docs/FomularioSolicitacoes/mariliaGFerreira-09-13-7.pdf – Based more upon organizational points ● ● ● ● ● 56 Boundary: the unit of analysis Vision: global summary of the unit Logic: rationale for vision Structure: of the organisation Roles: organizational roles of people

Slide 57

Slide 57

Release Your Refactoring Superpower ● How Do We Know? – Criteria ● Decision Model for Legacy Systems - 1999 (by . H. Bennett, M. Ramage, and M. Munro) - ftp://ftp.inf.pucrio.br/pub/docs/FomularioSolicitacoes/mariliaGFerreira-09-13-7.pdf – Based more upon organizational points ● ● ● ● ● ● 57 Boundary: the unit of analysis Vision: global summary of the unit Logic: rationale for vision Structure: of the organisation Roles: organizational roles of people View of information: resource analysis

Slide 58

Slide 58

Release Your Refactoring Superpower ● How Do We Know? – Criteria ● Decision Model for Legacy Systems - 1999 (by . H. Bennett, M. Ramage, and M. Munro) - ftp://ftp.inf.pucrio.br/pub/docs/FomularioSolicitacoes/mariliaGFerreira-09-13-7.pdf – Based more upon organizational points ● ● ● ● ● ● ● 58 Boundary: the unit of analysis Vision: global summary of the unit Logic: rationale for vision Structure: of the organisation Roles: organizational roles of people View of information: resource analysis Costs: major costs, both financial and nonfinancial

Slide 59

Slide 59

Release Your Refactoring Superpower ● How Do We Know? – Criteria ● Decision Model for Legacy Systems - 1999 (by . H. Bennett, M. Ramage, and M. Munro) - ftp://ftp.inf.pucrio.br/pub/docs/FomularioSolicitacoes/mariliaGFerreira-09-13-7.pdf – Based more upon organizational points ● ● ● ● ● ● ● ● 59 Boundary: the unit of analysis Vision: global summary of the unit Logic: rationale for vision Structure: of the organisation Roles: organizational roles of people View of information: resource analysis Costs: major costs, both financial and nonfinancial Benefits: both financial and nonfinancial

Slide 60

Slide 60

Release Your Refactoring Superpower ● How Do We Know? – Criteria ● Decision Model for Legacy Systems - 1999 (by . H. Bennett, M. Ramage, and M. Munro) - ftp://ftp.inf.pucrio.br/pub/docs/FomularioSolicitacoes/mariliaGFerreira-09-13-7.pdf – Based more upon organizational points ● ● ● ● ● ● ● ● ● 60 Boundary: the unit of analysis Vision: global summary of the unit Logic: rationale for vision Structure: of the organisation Roles: organizational roles of people View of information: resource analysis Costs: major costs, both financial and nonfinancial Benefits: both financial and nonfinancial Risks: major sources of risk

Slide 61

Slide 61

Release Your Refactoring Superpower ● How Do We Know? – Structure ● A Framework to Assess Legacy Software Systems - 2014 (by Basem Y. Alkazemi) https://pdfs.semanticscholar.org/da50/7665a6c3bacb5559996bd436a9f76aa 4e5a7.pdf – Strategies ● 61 Replacing

Slide 62

Slide 62

Release Your Refactoring Superpower ● How Do We Know? – Structure ● A Framework to Assess Legacy Software Systems - 2014 (by Basem Y. Alkazemi) https://pdfs.semanticscholar.org/da50/7665a6c3bacb5559996bd436a9f76aa 4e5a7.pdf – Strategies ● ● 62 Replacing Maintaining

Slide 63

Slide 63

Release Your Refactoring Superpower ● How Do We Know? – Structure ● A Framework to Assess Legacy Software Systems - 2014 (by Basem Y. Alkazemi) https://pdfs.semanticscholar.org/da50/7665a6c3bacb5559996bd436a9f76aa 4e5a7.pdf – Strategies ● ● ● 63 Replacing Maintaining Re-architecting

Slide 64

Slide 64

Release Your Refactoring Superpower ● How Do We Know? – Structure ● A Framework to Assess Legacy Software Systems - 2014 (by Basem Y. Alkazemi) https://pdfs.semanticscholar.org/da50/7665a6c3bacb5559996bd436a9f76aa 4e5a7.pdf – Strategies ● ● ● ● 64 Replacing Maintaining Re-architecting Extending by wrapping

Slide 65

Slide 65

Release Your Refactoring Superpower ● How Do We Know? – Application ● A Decisional Framework to Measure System Dimensions of Legacy Application for Rejuvenation through Reengineering - 2011 (by Er. Anand Rajavat, Dr. (Mrs.) Vrinda Tokekar) https://www.ijcaonline.org/volume16/number2/pxc3872674.pdf – System domain ● ● ● ● 65 Customer requirements Orgs strategic goals Operational env Risk management i. Organizational ii. Resource iii.Development iv.Personal v. User Requirement vi.Specialization vii.Team viii.Communication

Slide 66

Slide 66

Release Your Refactoring Superpower

66

Slide 67

Slide 67

Release Your Refactoring Superpower

67

Slide 68

Slide 68

Release Your Refactoring Superpower ● Case Study – 68 Example Assessment ● Step #1 - Does application meet or exceed definition of “Legacy”? ● Answers: Yes, No, Maybe, Don’t know – Business Critical – Old – Changed to meet organizational needs – Degrades as changes made – Maintenance cost increase as changes made – Obsolete languages – Poor, if any, documentation – Inadequate data management – Limited support capability – Limited support capacity – Lacks architecture to evolve to meet emerging requirements

Slide 69

Slide 69

Release Your Refactoring Superpower ● Case Study – Example Assessment ● Step #1 – Technical value attribute assessment ● Answers: Yes, No, Don’t know – Maintainability ● ● ● ● – Decompostability/Architecture ● ● ● ● ● ● 69 LOC Control Flow Cyclomatic complexity Dead code fate Modularity % of modules with separation of concerns Consumption Extensibility Style Interoperability

Slide 70

Slide 70

Release Your Refactoring Superpower ● Case Study – Example Assessment ● Step #1 – Cont’d ● Answers: Yes, No, Don’t know – Deterioration ● ● ● ● – Obsolescence ● ● ● ● ● ● ● 70 Backlog increase Defect rate increase Response-time increase Maintanance time per request increase System age Operating system version Hardware version Technical support availability Security Legality System evolution required for business goals?

Slide 71

Slide 71

Release Your Refactoring Superpower ● Case Study – Example Assessment ● Step #2 – Business value attribute assessment ● Answers: Yes, No, Don’t know – Economic value ● ● ● – Data value ● ● – ● ● Business function coverage rate Actual usage frequency Customer/user satisfaction metric Specialization ● ● 71 % of mission critical archives % of application dependent archives Utility ● – Market value Profitability index IRR % of highly specialized functions % of generic functions

Slide 72

Slide 72

Release Your Refactoring Superpower ● Case Study – 72 Example Assessment ● Step #3 – Organizational infrastructure attribute assessment ● Answers: Yes, No, Don’t know – Development & maintenance internal or outsourced? – Technical maturity – Commitment to training – Skill level of system support – Response to change

Slide 73

Slide 73

Release Your Refactoring Superpower ● Case Study – Example Assessment ● 73 Step #4 – Calculations – Calculate all responses to 1 – 5 values (don’t know = 0) – To easily plot on decisional matrix

Slide 74

Slide 74

Release Your Refactoring Superpower ● Case Study – Example Assessment ● Step #5 – Conversion – 74 Convert Y, N, DK to numeric values and Avg %

Slide 75

Slide 75

Release Your Refactoring Superpower ● Case Study – Example Assessment ● Step #6 – Plotting – 75 Display points on the decisional matrix

Slide 76

Slide 76

Release Your Refactoring Superpower ● Superhero – Status Granted ● 76 It’s YOU!

Slide 77

Slide 77

Release Your Refactoring Superpower ● Supervillain – Professor LOC ● 77 Fighting tools – PHPLoc – PHPmd (codesize)

Slide 78

Slide 78

Release Your Refactoring Superpower ● The Data – 78 PHPLoc

Slide 79

Slide 79

Release Your Refactoring Superpower ● The Data – 79 PHPmd

Slide 80

Slide 80

Release Your Refactoring Superpower ● Ray Gun – 80 Docker Images

Slide 81

Slide 81

Release Your Refactoring Superpower ● Arsenal – 81 1 million commits ● Rename Variable/Method/Class 77% ● Extract Constant ● Make Type Global ● Rename Refactoring Command ● Move/Extract Class 1% ● Move/Extract Method 13% ● Modify Method Parameters

Slide 82

Slide 82

● Thank you! – Code: https://github.com/adamculp/ Adam Culp http://www.geekyboy.com http://RunGeekRadio.com Twitter @adamculp Questions?