Алиса Бобовникова
Agile Transformation in IT-organizations
INTRODUCTION
Current market conditions stimulate organizations to introduce innovative product improvement solutions. Environment changes responsiveness became an important factor in market competition. This trend is especially typical for the IT industry as this field is characterized by a lot of innovation. Transforming companies find it difficult to adapt to continuous change. This is often caused by lack of organizational flexibility and changes responsiveness practice. In order to address this issue, companies need to continuously search, evaluate, analyze and develop new tools for organizational culture.
This narrative is for those who have chosen the path of transformation to gain speed, flexibility and effectiveness. We will talk about how to improve the teams effectiveness with agile techniques and also give you an idea how to start changes and how to measure them.
Chapter 1 discusses the approach to understanding agile methodology and its main provisions. Special attention is paid to the relationship between the type of thinking and the team effectiveness. Continuous customer focus and the need for regular teams interaction is revealed from practical examples.
Chapter 2 reflects agile management features and waterfall approach, which is also considered classic. This chapter provides these methodologies comparative analysis, highlighting distinctive features and opportunities for their use.
Chapter 3 describes agile transformation stages, most common agile frameworks, their distinctive features as well as their pros & cons.
Chapter 4 provides an overview of the main existing agile methodologies with implementation examples, as well as a step-by-step agile implementing roadmap also considering agile driven project documentation required.
Chapter 5 and Chapter 6 describes agile implementation effectiveness assessment methods, analyzes direct and indirect implementation process metrics as well as the technique implementation success criteria in different companies practice, and provides key agile projects KPIs.
Chapter 7 is dedicated to the agile transformation challenges and explanation what agile coaching is, describing Agile coaching levels and tools as well as how coaching can help you with transformation.
The narrative is filled with living examples of companies implementing transformation in the IT sphere. Thanks to organizations practice, we deliver experience that can be useful for you along this difficult path. New management practices introduction in an organization is a way that requires changes not only in actions but also in attitude. We hope you will follow this path with us.
CHAPTER 1. WHAT IS THE AGILE METHODOLOGY?
In 2021, the Agile Manifesto celebrated its 20th anniversary. The approach originated as a revolt of developers against clumsy IT corporations.
Let's figure out what Agile is, learn something from its history, and what is its difference from other software development approaches. We will also dive into some of the Agile frameworks, find out which option is most suitable for you, and what Agile coaching is, so this knowledge will allow you to apply it in your team or company.
The material may be useful and informative for wide range of specialists dealing with software development: front-end and back-end developers, DevOps specialists and software architects as well as for team leaders, project managers, product owners, CTOs, CPOs etc.
To begin with, lets recall some facts. In 1970, Dr. Winston Royce issued the document that defined Waterfall1. Probably it was already being done by others at that time, but it is considered now to be definitive. He marked 5 principles that must be added to reduce most of the risks of doing waterfall:
program design comes first;
document the design;
do it twice;
plan, control, and monitor testing;
involve the customer.
Dr. Royces last lines about waterfall were:
[This] summarizes the five steps that I feel necessary to transform a risky development process into one that will provide the desired product. I would emphasize that each item costs some additional sum of money. If the relatively simpler process without the five complexities described here would work successfully, then of course the additional
money is not well spent. In my experience, the simpler method has never worked on large software development efforts and the costs to recover far exceeded those required to finance the five-step process listed.
In other words, Dr. Royce suggested writing a lot of documentation, handling risks, repeating some stages several times. The result was a heavy, in contemporary opinion, waterfall model. In an ideal world, we would go through all levels of this system from top to bottom, like a waterfall flows, and in the end we would have a good system.
The waterfall development model quickly gained popularity in the West, and the vast majority of software development teams applied it some time ago.
Currently, Royce's name is strongly associated with the cascade methodology, while few recall that he criticized this approach, pointing out that the software development process should not resemble the conveyor line operation. IInstead of waiting for all the steps (phases) to be completed, Royce suggested using a phase-based approach with a short completion cycle of 1-2 weeks. Its essence is that initially all the requirements necessary for the project are collected, then divided into sub-projects, the target architecture is developed, the design is created, code is written in short iterations etc. The waterfall model may have
concretized and formalized the development methods existed, but it was not without its drawbacks.
Due to traditional cascade methodology imperfection, clumsiness and heaviness developers at that time already experimented with so called iterative-incremental approach2, which gave them increased development flexibility based on regular feedback from customers and users.
The main idea of this method is to develop the system iteratively using repetitive cycles at smaller time periods (incrementally), allowing software developers to take the advantage of what was learned during the earlier parts or versions of the system development. Learning occurs both in the development and use of the system, where possible key process steps begin with the simple implementation of a subset of software requirements and iteratively improve the evolving versions until the full system is implemented. At each iteration, design changes and new features are added.
Experiments continued in the 70s, and in the 80s of the 20th century, laying the groundwork for what would become the Agile methodology. By the beginning of the 90s the iterative-incremental approach was world-famous and studied.
Now it is clear that for the Agile appearance at that time only the human dimension inquiring strive to understand human traits and how to incorporate that understanding into management planning and actions was missing, exactly teamwork and stake on people and their interaction. By the beginning of the 90s, two key aspects were trained and tested, which will form the basis of Agile: iterative-incremental approach and team work, when the project is executed by a group of people interacting with each other as efficiently as possible, which also allows to implement projects of any scale.
The 80-90s were marked by personal computers (Apple Macintosh, PC-based and others) mass distribution. Millions of new computers emerged in ordinary people and office workers use leading to the fact that the software development market began to focus on mass demand, where it was necessary to solve not engineering, but user and business tasks.
For example, one of these tasks is to create a text editor. Before MS Word began to dominate, a lot of trial and error was done, a lot of processors and text editors appeared, many of which only few people remember now: WordStar, WordPerfect, Lexicon, XyWriter, Lotus notes, Makurita and others. This kind of product development has to be conducted with a constant attention to user's reaction, quickly responding to his needs with new versions.
Approaches to development based on the cascade model, with clear sequential stages, and comprehensive documentation have received the conditional name of "heavy methods" (heavy methodologies)3. At the same time, various specialists from the IT world attempted to work differently, inventing new tools, techniques, or even developing whole methodologies in order to increase software adaptability and development speed as well as deliver valuable product to the user faster.
These new approaches used a similar set of principles and tools:
iterative-incremental development;
regular customer or user feedback;
teamwork;
maximum transparency.
Several so-called "lightweight" methods (lightweight methodologies)4 emerged in the 90s of the 20th century: Crystal Clear, Extreme programming (XP), Rapid application development (RAD) and Dynamic System Development method (DSDM), ICONIX, Scrum5, Adaptive Software Development (ASD), Function-driven Development (FDD).
Jim Highsmith, author of the book "Adaptive Software Development" recalled that time: I think that at that moment we were all looking for legitimacy. Each of us could worked alone and did similar things in own way, but it could not have success and recognition in the community [of software developers].
Classical "heavy-methods" were often officially fixed as standards for large-scale software development projects (for example, in the US Department of Defense). "Lightweight methods" have long been perceived as exotic, or specific ways of working in a specific company on a specific project. They did not receive wide recognition, and were promoted only by their founders efforts.
Many people came up with the idea that it was necessary to organize a general meeting at which various "lightweight" methodologies founders and supporters would form a common document proclaiming a new software development paradigm. Then they could act as a united front, as an organized force, as opposed to "heavy-methods" dominance.
In early 2001, 17 people gathered not far from Wasatch Mountains in Snowbird, Utah, to discuss the future of software development. The participants of this group were united by concern about the current state of affairs in the industry, when heavy-methodology driven projects increasingly failed and flexible approaches were in great demand of legitimization and recognition. At the same time, they were not afraid that their thoughts about optimal solution differed.
The long weekend meeting resulted in Agile Manifesto of Software Development and became an answer to all these questions. This concise and expressive document consisted of only 68 words and changed software development forever. For almost two decades since its creation, these words (and the 12 principles that followed) have been adopted (to one degree or another) by a huge number of people, teams and companies.
Agile emerged as a mindset, a thought process that involves understanding, collaborating, learning, and staying flexible to achieve high-performing results, as a counterbalance to outdated approaches and excessive bureaucracy in IT field. Silicon Valley Residents realized that it is impossible to create innovative products in a conservative environment.
Here are Agile values with more detailed explanation.
There are four of them:
if you want to build an agile process, you need to interact and communicate with each other. You can (and definitely will) use some tools, for example, trackers JIRA, Redmine, etc. But the whole process should be based on various meetings and interaction, and not on trackers settings or TFS (Microsoft stack);
the working product that we make is much more important than the documentation for it. An example was given above with two companies. Documentation, the user cannot apply because the product is not ready, does not bring value to this user. If we learn to work minimizing software development steps, or by making them smaller, then we will have a more flexible process;
cooperation and interaction with the customer is more important than strict contract following. It is usually named Fixed Price when you sign an agreement and fulfill agreed amount of work. The time, amount of work and deadlines are fixed. This approach is not very good if you want to work for the long term and be flexible. To be agile, it is more correct to build partnerships with the customer. The most important thing is that the search for a partnership and a win-win situation begins here, when both the customer and his contractor win;
readiness for changes with following the original plan. Agile development approach requires plan, estimates and forecasts. If you have an initial annual project plan and provide some working product version in some months or requirements change while developing you can change the whole plan taking changes and feedback into consideration.
12 Agile development principles, also the result of the Snowbird meeting, expand these several value-defining proposals.
It's all. Since then, the website with the Agile Manifesto has hardly changed (or maybe it hasn't changed at all), which can't be said about the world around Agile.
Agile methodologies have enjoyed overwhelming popularity: first mentioned in the PMBOK (the US Project Management Body of Knowledge standard) 5th edition (2013), then they were fully adopted in the PMBOK version 7, released in 2021 which took all the Agile principles and became its direct ambassador.
To conclude with Agile introduction, it is necessary to state the main goal of this approach delivering value to the consumer. According to Agile methodology, it is achieved using three characteristics for a software product:
Thus, the Agile method is applied to:
accelerate the product launch to the market. If you want to develop software faster, you need to apply Agile. For example, two similar business companies. The first one creates the technical task for software development, designs the structure and interface consistently, this is a waterfall model, implementing it can take several months. Another team can already release a website and software applying Agile, start earning money and hijack the market;
manage priority changes. An unpleasant challenge for almost all companies. Your requirements will definitely change if you are doing the project that lasts at least a few months.What about commercial development, the problem is that programmers, analysts and designers never know what customer who pays us and users need. The usual approach is: until the user tries the site or application, you do not know whether it is needed or not;