From time to time I find myself writing longish e-mails to individuals or groups, offering them my point of view on various topics that are currently on their radar. This one, on the topic of Technical Debt, went out to a Management Team.
A few words on technical debt
Technical debt is a common term in our industry. The term is a metaphor to describe the consequences of the trade off between quality and speed. To better understand this, let’s think about what debt is.
We take on debt when we take a loan. The reason we take a loan is so that we can have something right now that we cannot actually afford at the moment. The option is to wait, save our money, and then buy whatever it is we want at a later date when we can afford it with existing funds. Gathering these funds takes time, time we sometimes feel we don’t have.
We pay interest on our loans, and this interest is the price we pay for the value of having that “thing” we cannot afford now, rather than later. This means that the total cost of that “thing” that we want will always be higher when we take out a loan. This is a still a rational decision if we put a high enough value on having the “thing” right now instead of later. In the case of business, one might say that taking out a loan is rational if the cost of delay is higher than the total cost of the “thing” plus interest.
But there are some rather troubling side effects that are sometime lost in the discussion.
Interest is compounded (interest on interest) which means that the size of the debt increases if we don’t pay off our loans. Left unattended, the cost of any loan will over time become greater than what the value was of having that “thing” now, rather than later. As this happens slowly over sometimes long periods of time, we tend to be blind to this fact.
Translating the metaphor into the realm of tech, we can say that we “buy” functionality and get it sooner by taking out a loan from technical sustainability, scalability and quality. More simply put, we make conscious quality trade off’s to get things sooner, rather than later. These trade off’s compound as we add new layers of quick fixes on top of quick fixes, until such time that our system becomes slow and difficult to work with and becomes so entangled that we are eventually prevented from making use of new advances in technology. This is the effect of compound interest.
As everyone with loans on their homes understand, regularly paying of small parts of our loans hurt at lot less than one big payment per year. In the world of tech, it’s established good practice to make these small amortizations called “refactoring” continuously, so that we at least avoid the compound effect on our debt. With this habit, the debt is still there, but at least we don’t add to it. There’s no interest on interest.
There is, however, a level of debt we can reach when this becomes unpractical, where these small amortizations are either A) no longer possible, because of the messy nature of the code (quick fixes depending on quick fixes depending on quick fixes) and/or B) poor use of time, as no small amortization would have a good return on investment, because in order to have an impact on our debt, we would have to dig deep and do a lot of work.
When we have reached that level of technical debt, we have to move away from the preferred method of regular refactoring – the continuous small amortizations, and initiate more focused efforts.
Misuse of the metaphor: Not every larger technical problem is caused by “technical debt”
My view is that the term technical debt should only be used for situations were we are fully aware that we are acquiring debt. If we unknowingly create suboptimal technical circumstances, “debt” is no longer a suitable metaphor, as it would be a very irresponsible person who wakes up one day and realises they have a loan with interest, and a debt to pay.
If we put ourselves in unfavourable circumstances unknowingly and find ourselves surprised that we’re in trouble, it’s time to start talking about suboptimal engineering practices,issues with decision making, visible or invisible power structures, explore our quality issues and backtrack to discover any basic errors of judgement – and work on how we can make sure that we do better in the future.
A more responsible person is of course very aware that they’re taking a loan, and that there’s a debt building up. Some folk, however, go into debt without a pay off plan. This too is bordering on the irresponsible.
The very responsible person goes knowingly into debt, and has a plan for paying of the loan. This is where we need to be.
Misuse of the metaphor: Our system not being able to sustain new requirements is not neccessarily a case of technical debt
When we design and build software, we are constantly optimising our efforts based on our circumstances and constraints. We make choices and trade off’s based on the information at hand. We might look somewhat into the future, but trying to design and build a system that can cater for any future requirements is a bad idea. This is what is known as premature optimisation, which is not only expensive and slow, but also invites over-engineering and useless speculation.