Software development is a process of discovery and exploration. Therefore, to succeed at it, software engineers need to become experts at learning.
We must be experts at learning and experts at managing complexity. There are five techniques that form the root of this focus on learning:
A) Iteration
B) Feedback
C) Incrementalism
D) Experimentation
E) Empiricism.
And to become experts at managing complexity:
A) Modularity
B) Cohesion
C) Separation of concerns
D) Abstractions
E) Loose Coupling
If our software engineering practices don't allow us to build better software faster, then they aren't really engineering, and we should change them! So, Applying this kind of engineering thinking to software does not need to be heavyweight or overly complex. The paradigm shift in thinking differently about what we do and how we do it when we create software should help us see the wood for the trees and make this more straightforward and efficient. This is not about more bureaucracy; it's about enhancing our ability to create more sustainable and reliable high-quality software.
If we re-think what is Engineering? We have learned that our industry was strongly related to "production-style thinking," like the waterfall process. That process is more related to mass production than discovery, learning, and experimentation, which are, or at least should be, at the heart of our profession.
It's essential to understand the empirical learning it's so important, like for instance errors 1201 and 1202, during descent to the moon, was an alarm that was defined by Hamilton and her team, which indicated that the system needed to reboot by overloading, so from experience, they know that the system should be restarted sometimes. It was a "fail-safe" situation that happened.
Let's review the definition of what is Engineer by Farley David:
Engineering is the application of an empirical, scientific approach to finding efficient, economic solutions to practical problems.
So, it said engineering is about applying scientific rationalism to solving problems. It's the "solving of the problems" where the engineer comes to play, most just the solutions themselves. It's the process, tools, and techniques. The ideas, philosophy, and approach together make up an engineering discipline.
Another important aspect of our professions is to understand the difference between crafting our solutions vs. massive production (like gun manufacturing). But what does it mean, Craft? Usually, when we talk about craft, we talk about building something unique, which is quite the opposite of generating massive production of something like a car or gun. To archive that craft but in a scalable way, we should take seriously the scalability and the management of the complexity as critical factors to consider when we are building our software (that is one of the main reasons it is so important to build modular software). Another essential aspect to consider is to use the right tool to scale our solutions, like the need to use automatic tests for the code we produce to ensure that our product behaves as we meant to be. The machine could execute repetitive task like tests millions of times faster than a human can do for a longer time (as long your computer have electricity will keep doing over and over again). We should use the right tools considering human limitations, to achieve our goals. Edsger Dijkstra said: "The art of programming is the art of organizing complexity."
So, back to the idea of software craftsmanship, it's really important actually like creativity and innovation, skills and apprentice schemes, because it represents a step away from the big-ceremony, production-centered approaches to software development that proceeded them. But take into consideration that software craftsmanship alone is not enough. An Engineering discipline in virtually all human endeavors increases quality, reduces cost, and generally provides more robust, resilient, and flexible solutions.
It's a big mistake only to associate ideas like skill, creativity, and innovation with Craft. Engineers, in general, but certainly design engineers, exhibit all these qualities in abundance. These Attributes are central to the process of design engineering.
Another critical aspect of engineering decisions is to take into consideration to make our choices, the rational criteria, the strength of the team, and the product and economic impact. You decide based on the evidence before you and your theory of what that will mean, and then you test your ideas to see if they work. It's not some perfectly predictable process. Following the problems of the take choice, if we make our system more secure, it will be more challenging to use; if we make it more distributed, we will spend more time integrating the information gathered. If we add more people to speed up development, we will increase the communication overhead, coupling, and complexity, all of which will slow us down. At the end of the day, we should have some measures to help us to build better software.
How could we measure our work?
The measures are Stability and Throughput. Teams with high stability and throughput are classified as "high performers."
One of the standard practices and tools used by the high performer teams are:
- Automation test
- Trunk based development
- Deployment automation
Stability's key aspects:
- Change failure Rate: The rate at which a change introduces a defect at a particular point in the process.
- Recovery failure Time: How long it takes to recover from a failure at a particular point in the process.
Throughput key aspect:
- Lead Time: A measure of the efficiency of the development process. How long does a single-line change take from "idea" to "working software" in production?
- Frequency: A measure of speed. How often are changes deployed into production?
Also, it's important to mention that you can have velocity and quality simultaneously, but you will need to measure your work with the metrics mentioned.
The route to speed is high-quality software, and the way to high-quality software is the speed of feedback, and the way of both is great engineers.
We should ask ourselves very often, "If I adopt this new language, does it increase my throughput or stability?", “Does this increase the quality of the software that we create?” (measured by the stability metrics). “Does this increase the efficiency with which we create software of that quality?” (measured by throughput)
Having meaningful measures that allow us to evaluate actions is essential, even vital, to take a more evidence-based approach to decision-making.
Let’s recap again Foundational ideas:
Following, we will talk about some of the essential stuff that we need to pay master:
Become an expert at learning: We should recognize and accept that our discipline is a creative design and has no meaningful relationship to production engineering, and instead, we should focus on mastering skills like exploration, discovery, and learning. So, let's review some essential characteristics and healthy habits that a high-performance team does on a daily day:
- Working iteratively:
Iteration is at the heart of all exploratory learning, is fundamental to any real knowledge acquisition, and is strongly related to work on small units of work -> “Progress in software is hard to measure, but we can measure finished features, so let’s work on smaller, releasable features so in that way we can see when they are finished (it delivers some value to our users).”
- Employing fast, high-quality feedback and Working incrementally
Working iteratively in small, definitive, and production-ready steps provides us with outstanding feedback! This is one of the most crucial aspects of continuous delivery (CD).
TDD and CI are both iterative processes that could help us to work iteratively and get feedback fast, even when we don’t know much about what we're trying to resolve.
This could sound so obvious, but →
Feedback allows us to establish a source of evidence for our decisions!
Once we have such a source, the quality of our decisions is inevitably improved. It allows us to begin to separate myth from reality.
There are two codes of fast feedback; the first is my IDE. We should use tiny refactors the IDE provides to help change your code more safely. The second one is our test that should keep running along the way, which provides immediate feedback after I introduce a new code that the system still works as it should. Organizing our work into tiny steps allows us to reflect on our progress and steer our designs toward better outcomes. Then, after I commit my code, there are the integration tests that start working and help me know if my code, after I pushed it, leaked out and caused some other parts of the system to fail.
The CI way to works says that each small step gives us feedback but does not necessarily add up to a whole feature. This is a very challenging change of perspective for many people, but it is a liberating step when embraced and positively impacts the quality of our design. Not only does this approach mean that our software is always releasable and that we are getting frequent, fine-grained feedback on the quality and applicability of our work, but it also encourages us to design our work in a way that sustains this approach.
- Being experimental and empirical
Because industry data says that for the best software companies in the world, two-thirds of their ideas produce zero or negative value, we are terrible at guessing what our users want. The most effective approach is to iterate, so we should try them out as quickly, cheaply, and efficiently as possible.
- Being empirical
Based on user feedback, we observe and learn and then keep iterating based on that experience. It’s essential to be empirical in our profession; we should learn from the result of what we implemented from our theory and keep learning how to improve our user's needs based on results, not guessing.
Become an expert at managing complexity: we build systems that don't fit in our heads. Also, we make systems that should scale with a larger group of people in an organization. We should be experts in dealing with complexity at the technical and organizational levels.
NOTE: Many of these ideas come from the fantastic book “Modern Software Engineering” by Farley David.
