Innovation Enabled, Value Driven Architecture Evolution

In this talk we describe our experience applying somewhat unconventional lean-inspired innovation techniques to make major impacts in challenging environments. Unlike the generally accepted approach of slowly evolving the old system to the next generic technology and practices; e.g. Procedural to OO, OO to FP, waterfall to agile; monolith to microservices, migrate to cloud etc. we describe an alternative.  We explain and illustrate by concrete example a lean value-based approach to target and implement short-term high-impact changes at specific bottlenecks in the architecture. Experience with classic legacy architecture transformation is well known to be expensive; slow; high risk and often has a disappointing impact.  In part, this isn’t surprising because the classic Agile approach seeks to improve primarily by systemic change across the organization often force-fitting essentially common work practices and engineering techniques across the business.  Although Agile is substantially inspired by Lean, it has been unfortunately obsessed with specific ceremonies and tooling.   Despite almost 25 years of “Agile/XP” most large organizations are still applying the same techniques!  Executives often sign onto Agile to obtain Agility, but the Agile fanboys never bothered to mention that productivity, nor agility is actually not included in the Agile contract per se.  The Agile promise is Predictability and Quality which it can deliver.  In my experience, large single-tech approaches take way too long and have too much risk; and require too many people hence they may be “agile in the small”l, but not have the impact that business seeks/needs in the large. In contrast to common code-centric approaches for large-scale transformation, we discuss our experiences applying a value-based approach where Agility is substantially due to the application of a specific innovation at a narrowly focused point of high leverage. The approaches to my knowledge are not discussed in the legacy literature which focuses on evolving large code bases a la "Working with Legacy Code."   Each intervention applies a different strong specific solution, often using new or radically different approaches in each improvement.  A carefully targeted approach allows more effective use of scarce skilled resources, substantially reducing training and staff expenses. Unlike code-centric approaches, we focus on data and data flow. Agility and Productivity typically come from major innovations in the value chain. In our experience, the shortest path to such innovations is their selective, and appropriate insertion in the value chain.  In order to make this talk very tangible the majority of the talk discusses several concrete mini-case studies which illustrate what one might call "innovation patterns" applied in practice.