Join us
@jessiehernandez ă» Sep 23,2021 ă» 9 min read ă» 1658 views ă» Originally posted on medium.com
âWhen I master testing, will I get to dodge bugs?â Jessie: âNo, when you master testing, you wonât have toâ
Much has been written on the subject of automated testing, whether it be unit tests, integration, end-to-end, or any esoteric permutation thereof. As with everything these days, there are a multitude of opinions on the subject, from those prescribing a long list of complicated rules for automated testing, to those who say unit-testing is overrated or a waste of time. Then there are those who like to sound smart and politically-correct by answering âIt dependsâ to every question, which helps absolutely no one improve.
You will not find that in this article. This is a highly-opinionated piece on what has helped me improve my testing practices, and more importantly, as will be mentioned later, the quality of my code. Take what you will and apply it your own development. If just one item from this article helps you in the slightest with your code quality, then I have accomplished my goal.
I will be presenting code examples mostly in Go, which is one of the most popular languages used today for cloud and microservice development and my favorite language currently. In some instances, Iâll also present some examples in Java to reflect the OOP model of other common languages.
It is a well-known fact that fixing a bug after a product release is much higher than if the bug were uncovered during the development or quality assurance phase. For example, quoting from Celerity:
The
Systems Sciences Institute
at IBM has reported that âthe cost to fix an error found after product release was
four to five times as much as one uncovered during design
, and
up to 100 times more than one identified in the maintenance phase
.â
The same article mentions that
âA 2003 study commissioned by the Department of Commerceâs National Institute of Standards and Technology found that
software bugs cost the US economy $59.5 billion annually.
â
The message is clear: the earlier in the development process a bug is detected and resolved, the faster and cheaper it is to fix it.
When I started my career in software development in the year 2000, automated testing, of any kind, was not yet a widespread practice. In several of the companies I worked for initially, there was an entirely separate QA team that would test a release before it was pushed to production. I remember writing what seemed to be beautiful code for new applications which worked perfectly locally, only to have my ego shattered when it was deployed to the QA environment and the QA team tried thousands of ways to break my application, AND succeeded.
In the end, though, I was happy the QA team found these issues, as it meant they would not make it to production and affect actual customers. On the other hand, when an application was in its maintenance phase, adding just a few enhancements every few weeks, the QA team spent more and more time testing the application, not just for the new features, but also all the existing functionality to make sure nothing broke as a result of a new feature. This was a very tedious process for the team and, as a result of being human, some of these manual tests were either overlooked or skipped in the interest of time, only to result in a bug in an existing feature making it into production. This would end in lots of finger-pointing between the development and QA teams, with the QA testers asking âwhy did you introduce this bug here?â and developers saying âwhy didnât you catch it during your testing?â.
It was around the year 2008 when I first learned about unit-testing, which marked a turning point for me in my career. As I started writing tests, and eventually doing true Test-Driven Development and BDD, the quality of my code improved, the amount of defects that made it into production was just a small fraction of what it was before, and being on-call was no longer something I dreaded.
I have learned a lot about testing since then, and Iâm still learning about it each day. With this article, I hope to impart my lessons learned so that we can continue to move the needle in this exciting field.
Besides the reasons above, remember this: well-designed code is not necessarily easily testable, but easily testable code is usually well-designed. If you want to improve the quality of your code (which is the REAL goal out of all of this), then make your code easily testable. The best way to guarantee this is by writing your tests first.
When you first learn about unit testing, you will inevitably learn about the Testing Pyramid as first coined by Mike Cohn. In-depth information on this pyramid can be found on the blog of the âman, myth and legendâ himself, Martin Fowler, at https://martinfowler.com/bliki/TestPyramid.html and https://martinfowler.com/articles/practical-test-pyramid.html .
For simplicityâs sake, and because we live in the day-and-age of API-first development and microservices, here are the layers I think of today and their explanation (from bottom-up):
As seen in the pyramid graph, the tests in the lower layers are easier to write and faster to execute. The higher you go up in the pyramid, the more work there is in setting up an environment in order to make sure that tests can be executed in a consistent way, and as a consequence of actually hitting the database and other dependent services, the slower these tests will take to run. In this article, Iâll be focusing mostly on unit tests and lightly on integration tests. If thereâs enough interest, Iâll create a separate article for component and end-to-end testing.
Before we delve into the technical aspect, I need to stress one thing: code quality metrics by themselves are useless at best, and misleading at worst. For example, when starting out in the testing journey, developers and/or management might be tempted to set out a goal of âletâs meet a code coverage goal of 80%â or something similar. This is a recipe for failure. It is possible (and easy) to achieve a high code coverage percentage without improving maintainability/code quality. Spaghetti code with high code coverage is still spaghetti code. As a simple example:
This program simply prints the list of prime numbers from 1â20, and surprise, it has 100% code coverage! Yet:
âŠand so on and so forth. It is obvious that low code coverage means that more test cases must be written, but conversely, a high test code coverage does not mean that you have covered all cases in your program. As a simple example, in languages that have the ternary operator like Java, you can have logic such as the following:
String secretValue = cloud.getName().equals("AWS") ? AWS.getSecretValue(secretName) ? GCP.getSecretValue(secretName);
The above contrived example retrieves the value of a secret which can exist either in AWS or GCP. Depending on the name of the cloud, it calls the getSecretValue
method of the corresponding cloud class. Guess what? If you write a test for just one cloud, such as AWS, this line will show up as covered! If your GCP implementation has a bug in it, you will not know it until you hit production. Worse yet, if the cloud name value is empty or contains another value, GCP will always be called in these cases, which is not the desired behavior. The code coverage report also canât tell you that this line is simply badly-designed. What if we want to add another cloud in the future? Clearly, there needs to be refactoring in this case to have the class accept an interface, and then delegate the secret value-handling to a concrete implementation of the interface.
So clearly, one line of code does not equal one test case. Instead, we should concentrate on testing situations, features and code branches, not simply writing tests to cover lines.
Managers: do not simply set goals of increasing your test code coverage to some percentage. Set out a goal of improving your teamsâ code quality, of which higher test code coverage is a component. Code quality cannot be measured by a tool, no matter how fancy or expensive it is. It requires review by an architect or one or more senior developers in the company. Drill these two things into your head:
Clearly, then: the most important aspect of testing is your mindset going in AND NOT any particular technique you use. We will delve into the proper mindset in the second part of this series.
Thanks for reading!
Join other developers and claim your FAUN account now!
Influence
Total Hits
Posts
Only registered users can post comments. Please, login or signup.