A core challеnge in software development
Usually, at the beginning of most software projects features are being developed fast, by a small number of software developers. Over time, as more people are being added to the project, productivity per person is going down.
This is somewhat expected as there is more time invested in communication and synchronisation within the team. But the next figure (Figure 1), based on a study from a real company, shows something really concerning and at the same time familiar to most of us in the software delivery business.
Figure 1: Thousands of lines of code produced by the different number of developers
As the number of developers grows from 1 to 8, the number of lines of code is a little bit more than doubled. This resulted in a dramatic increase in the cost per development (Figure 2).
Figure 2: Cost per lines of code over time
Someone might argue that Lines of Code is not the best indicator of productivity, but it still shows that the pace of development per person is reduced.
One of the main reasons for this issue is the software architecture and design and the compromises made with it known as “technical debt”. The development team usually is pushed by the business to release to market as soon as possible. And usually, the promise is “Let’s get this feature to market fast, and then we will take some time to clear the mess”.
But we all know that this never happens – after this feature, there are already a bunch of other features waiting to be addressed. And they are as important (if not more) as the ones before them. So this creeping mess slows development down, and every new release becomes like adding a new layer in a castle of cards – it takes more time and effort, and a wrong move can affect all the layers beneath it. So as Robert Martin said:
“The only way to go fast is to go well.”
Getting software done in the right way is not an easy task. It takes knowledge and skills that a five years university (or six months of training in a software academy) is not going to give you right away. It requires a level of dedication and discipline that most of us are not used to. But above all the most important is the desire to be a professional. Everyone can write a code that just works, but only an expert can write code that is clean, understandable, testable, simple, optimal, and extensible.
Why is the good software architecture the key to keeping the productivity of the team in the long run?
As Robert Martin said in his book Clean Architecture, every software product has two main values: behaviour and architecture. The behaviour is how the (commercial) software is saving money (or gaining more money) for the stakeholders.
We, as software engineers, help the business define the requirements and then implement them and eventually make a system that behaves as needed for the business. From a business perspective, this is what we get paid for and what we should be doing 100% of the time. Many programmers also think the same wrong way.
But the other value, the architecture value, has to do with the nature of writing software. Software is something flexible and was intended to change the behaviour of machines easily. If it loses this ability, then it can no longer reflect the changes in the business needs, and so it becomes not only useless but also a burden.
That’s why we can state that this second value, the architecture value, is more important than the behaviour value. If you have a software that behaves by the current requirements but is hard to change, then it will become obsolete because of the first rule of business: “If you do not adapt to the changes in the market you are doing business in, you will fail”(1). On the other hand, if you have a system that does not fulfill the requirements (does not behave as expected), but it is easy to change then you can change it and make it work.
We, as professional software engineers, have to fight for the software architecture because we are the ones that understand the value of it. The business will insist that they are interested only in the behaviour of the system, and sometimes (if not often) they will push the team to deliver faster and to make compromises.
But after 1-2 years when the cost of a feature becomes unrealistically high (because the system lost its second value, the ability to change) then you can be sure that the software engineers will be the first to blame for allowing the system to reach this stage.
“Therefore is the responsibility of the development team to assess the importance of architecture over the urgency of features.” – Rober C. Martin
What can we do to take responsibility for the architecture?
Does Scrum give guidelines?
Let’s start first with the agile development processes and what is their contribution to keeping -a nice and clean architecture and good quality of the software product in general.
Scrum states that the development team is responsible for this task. And it’s the right way – who is more knowledgeable of the system than the people who develop it, the software engineers(2). But the ones that state the priority of the tasks to go into the sprint are the product owners.
And actually, they play a significant role in keeping the architecture clean and the technical debt low. They are usually pushed by the business to deliver as fast as possible, and some more inexperienced POs can also be “tricked” to make some commitments about deadlines based on preliminary estimates from the dev team. So they will put stress on the team to achieve this deadline. And this stress will likely result in lower quality and compromises with the architecture with the promise things will get better again later.
That’s why for software projects, a new role emerged in the scrum teams – technical product owner. Although the business PO remains the lead role, the technical PO acts as an advisor and can help bridge the desires of the customer and the capabilities of the technology. This role can be taken from the tech lead in the team, but then it means he will be distracted from achieving the sprint goals.
Another well-adopted approach is having an Architect as part of the team (usually part-time) as an adviser and auditor on the architecture. These architects usually have authority in the company, and they have the power to reject an implementation approach if it poses a risk for the quality of the architecture.
In conclusion, Scrum defines a process, an interaction between the parties involved in creating and iteratively delivering a product with short feedback loops. It relies on everyone understanding his role and responsibilities, executing it most professionally, and not overlapping with the role and responsibilities of the others.
As for the Dev Team, they must understand that Scrum is not only about achieving your sprint tasks. It is about finding (and fine-tuning) what amount of tasks the team can accomplish with quality and professionalism for the sprint length so that for the next sprint you commit to a similar amount of work.
It is also about regularly reviewing the process and the steps, finding bottlenecks or places for automation so that the team can increase their velocity but not for the sake of quality.
Again going back to the second value, the ability to change, it’s important to be able to refactor the code constantly on different levels (classes, modules, system, infrastructure). As the Boy Scout Rule in software development states:
“Always leave the code better than you found it.”
To be able to do so, the developers need a mechanism, a safety net, that will save them from breaking the expected behaviour of the system. This safety net is called automation tests. The different automation tests (unit, component, functional, system, end-to-end) will help you make the refactoring on the different levels mentioned above.
If you refactor a single class, then the unit tests on the API on this class will help you validate you haven’t broken its behaviour. If you are doing a more complex refactoring, then you need tests on upper levels to validate it.
And last but not least, writing automation tests will ensure your code is testable. This is an important feature as it is strongly related to the two most important concepts of clean and extensible code – loose coupling and strong cohesion. You can hardly write tests if your code is not loosely coupled, because it will be hard to test it in isolation. And your tests’ logic will most likely duplicate if you do not have strong cohesion.
That’s why having low or zero test coverage means low or zero ability to change, which results in losing the most important feature of your system – the ability to change.
Finally, I want to stress the importance of everyone from the dev team understanding the architecture of the software they are contributing to. Having a visualised architecture helps us understand it better.
Everyone should be aware of the different modules and their purposes, so when they code a new functionality, they are certain of where it must be added and why. And if it is hard to decide, this probably means the organisation of the modules needs change.
Everyone should be aware of the dependencies in the system and what are the allowed communication channels from one part (or layers) to the other components (or layers). This way you can avoid illegal crossing of boundaries and avoiding the spaghetti code result(3).
Of course, by documented architecture, I don’t mean one that was created at the beginning of the project and never changed afterwards. The architecture must be part of the shippable artefacts with every release.
These three guidelines are essential but not exhaustive, and if done right in a project, I’m sure they will have a significant impact on keeping the system’s ability to change.
- It’s first by my inner feeling-based prioritization of business rules
- It’s a pity technical engineers argue that only they deserve this title, and the architects of buildings also do not agree with sharing their title either. I think it is time to come up with our own titles.
- Actually there are nice tools where you can define the allowed communication channels between components and then have automated tests that will verify if someone has made a “shortcut”.