I will preface the conversation by stating that I am not a practitioner, nor have I attempted to practice TDD in a professional setting. This is a thought experiment vs a critic of TDD. I also assume that the reader is familiar with the basic concepts of TDD. Finally, one last caveat – my professional experience is in the embedded firmware space, e.g. microcontroller development.
My dumbed down perspective of TDD is that it offers the following advantages:
- Forces the developer to write automate units (whenever possible).
- All automated unit tests must pass before merging code.
- “A key benefit of TDD is that it makes the developer focus on requirements before writing code.” [1]
- You can “safely” refactor once all your unit test pass.
I 100% agree with the list above, however in my experience I don’t have to use TDD to accomplish all of the above.
So the open question is: what I am missing about TDD? Assuming my statement is correct that you can obtain the benefits above without employing TDD.
My alternative:
- As part of your SDLC process, make it a requirement to have automated unit test whenever possible. For many years now, for the projects I have led – we have had the requirement that all code require unit tests, or said another the unit tests are the tangible proof that your code works vs. “just trust me”. In the embedded space not all unit tests can be automated when the code under test directly ‘touches’ the hardware registers. However, that does not prevent writing a ‘manual’ unit tests that requires supervision when executing on the target hardware. As an aside – I claim that over 80% of any embedded project can (and should be) tested using host based automated tests. This is not hard (or time consuming) to achieve if it is part of the SW architecture and design from day 1.
- Incorporate CI/CD pipelines into the development process. With GitHub this simply means that Pull Requests are not merged until the CI build passes, where the CI build includes building and running all automated unit tests.
- Separate the “design” activities from “coding” activities. My upbringing goes back to the Shlaer-Mellor methodology in that coding is a translation step from your detailed design. Or said another, solve the problem before you start typing code. And yes, this includes documenting the solution in a detailed design document. I advocate that Software Architecture, Software Detailed Design, and coding + unit tests are three distinct steps in the SDLC process (see my books)
- Once the unit tests exist (see item#1) – then yes, I can have confidence in the results of my refactoring because I have a known, proven existing test suite.
Final Note:
My personal test strategy, methodology, etc. is:
- Every sub-system, component, module requires a unit test. And whenever possible the unit is automated test that can be incorporated into the CI/CD pipelines.
- Build and run your unit test(s) before you fully complete the coding of your sub-system, component, module. The initial tests should focus on creating and initialization of the code under test. This helps identify cyclical dependencies that are easily missed in your design.
- Once your basic unit test is up and running. Iterate between coding, writing unit test code, and executing the unit tests.
- Note: I rarely write the test code first.
Well, you asked what you’re missing about TDD, so, here’s my view. I have practiced TDD for several years and taken Grenning’s workshop twice, though TDD is not my primary mode of development.
There are several aspects that stand out to me about TDD:
1. Developing a practice of working in much smaller steps and shortening your feedback loop (“TDD Microcycle”).
2. Developing a practice of more thoroughly checking your test cases (making sure your tests actually fail, and proving to yourself that the changes you’re making are what cause the test to pass)
3. Writing production code with testability in mind from the start (vs trying to add tests on later, where you may need to rework the code to make it testable or simply skimp on coverage)
4. Improving test coverage (c.f. test-later approaches) by having a practice of adding/tweaking a test before writing production code.
Framed another way, TDD aims to foster “discipline” in the basic development workflow. The fundamental discipline of “work in smaller steps, check your work at each step” has benefits even if you don’t have tests as an output (but they’re a nice bonus).
From my experiences consulting and training developers, this basic discipline is really not all that natural – there’s a lot of resistance to working in smaller steps. It gives me sympathy for my math teachers, who all scolded me that I needed to “show my work” or “go back and check my answers” when I was the first one to finish a test.
FWIW, best way to understand the true value add is to practice it yourself. Grenning’s workshop is a great way to experience TDD for a period while receiving feedback from an instructor in an embedded-focused way.
All good points Phillip.
I have to agree with your bulleted item #2 – “…thoroughly checking your tests…”. The best way to be consistent with respect to ensuring that your tests are valid (i.e. the test case will fail if the code is wrong) is to write the tests first. I might have written a test case or two in the past that would always pass no matter what my code did 😉
Your comment about teams lacking “discipline” is spot on! So if TDD brings more discipline the “art of coding” – then it is good thing. The lack of discipline (or lack of actual “science”) is what motivated me to write my “Patterns in the Machine” book.
Yes, you’ve hit on a number of the advantages of TDD. I use it whenever I can, primarily working on embedded systems. I just finished 2 and half years of contracts with a billion-dollar automotive supplier who used a form of it.
Regarding your alternative, you’re already miles ahead of many organizations by requiring unit tests that way, TDD or otherwise. I wholeheartedly agree that you can separate the final “touching the hardware and RTOS” bits from all the rest of the code in your architecture. The latter can be tested off-target, meaning as you say that 80% or more of the code can be fully exercised this way. The last 20% or less must be tested on target because of target-specific dependencies.
Then the build process must require that all tests pass.
I also have practiced Schlaer-Mellor. There’s nothing in TDD that says you can’t think before starting. Thinking ahead is always better. But typically any system is a mix of things you’ve thought through a lot and have committed to decisions, and things you haven’t. TDD works across all of those. Regardless of how much you’ve pre-designed it, it allows you to verify that the code you write actually does what it’s supposed to.
One thing about TDD is that it encourages you to defer any of that thinking as long as possible so that you can remain flexible to changing needs. Some decisions need to be made early, but some can wait. Then you have a mix of up-front design and just-in-time emergent design, whatever mix is appropriate for your project and industry.
And no one ever said you shouldn’t document; the tests themselves are one form of documentation, executable and changing as needed when code changes. The Agile Manifesto favors working software over comprehensive documentation, but doesn’t say to eliminate documentation. The goal is to avoid investing all your resource in documenting something, that has to be thrown out because things changed. New features, changed features, chip shortages, unexpected timing issues, etc. can cause that.
You should also look at BDD (Behavioral Driven Development). It subtly changes the focus and approach in order to avoid the pitfalls of doing TDD poorly. I think of it as an extra layer of discipline on top of TDD, especially helpful for those less familiar with TDD (that was the motivation for it in the first place).
For a real example of embedded code developed via BDD, see my blog post (with linked repo) “Bit-Banged Async Serial Output And Disciplined Engineering” at https://www.embeddedrelated.com/showarticle/1544.php. It’s fairly simple but illustrates the method, scalable to any size system.
Note that Phillip and I both took Grenning’s TDD class at the same time!