 |
Why Unit Testing Alone Fails 
More unit testing can fix mistakes, yet fail to
prevent real problems
Article appeared in May 2005 issue of Software
Test & Performance magazine
by John Michelsen and Jason English, ITKO
|
|
The Current Quality Situation
Unit testing as a quality strategy? It won’t fly.
Yes, unit testing solves problems. Yes, developers should continue
to practice it. But is unit testing still a relevant frontier
for improving the quality of enterprise applications? Brand
me a heretic, but I think it is only solving the small problems.
Testing during development is a quality discipline you need
to enforce. Therefore, you’ve invested in training and
tools for your programming staff. More unit tests are helping
your development teams find structural bugs more efficiently.
Churning through even more unit tests, and looking at charts
that show how your byte code is 99.5% covered is reassuring.
Then you realize while using the finished project at deployment
time that the developers executed flawless code, but built an
application that fails to meet business requirements in production.
In our experience, only 1 in 100 critical software
failures that enterprises encounter in production could be prevented
by better developer unit testing.
Increasingly in the enterprise application development field,
quality improvement initiatives have become oriented around
either 1) testing the front-end UI of a nearly finished application
to tell you "it's broken," or 2) promoting the practice
of unit testing as a discipline reserved for developers. We
believe these forces open up a serious quality gap.
This article addresses the second assumption: that code coverage
(or "white box") developer unit testing can prevent
most errors in production. Relying on unit testing alone to
improve software quality is extremely dangerous when you are
trying to uncover problems due to resources, business rules,
logic errors, performance hang-ups, data quality, user behavior,
security, long-term system stress and integration.
|
