So Wil Shipley's post a few days ago about the merits (or lack thereof) of unit testing has enraged quite a few people.
Looking at the comments, there's a bunch of people that agree with Wil's reasoning on face value ("Yeah! Unit testing sucks! It takes up so much time!"), but then you have others who disagree (to an extent). The most enlightening post I've seen on the topic is by Bill Bumgarner, who notes that unit testing was used by the team developing Core Data to make sure the code adhered to the specification (or expected behaviour), as well as preventing regressions when bugs were fixed.
As has been pointed out by many, unit testing is most useful for application "infrastructure", and, with my limited experience, I would tend to agree. Unit testing of the classes that directly handle user actions usually doesn't make sense. However, at the foundation classes, it does. In the Confluence team at Atlassian, we write unit tests (although probably not as often as we should!) for many of the base classes to ensure that they behave as expected. The advantage of this is that should the implementation change, we can run the unit tests and make sure the code behaves as we expect it to! Granted, this is an application deployed over the web, but the same principles can be applied to an application with a GUI on the Mac.
When developing Connoisseur, I had no idea what unit tests were, and as such there have been no tests. Recently, while re-writing some of the parsers, I decided to write tests to chuck recipes at the application and make sure they're parsed correctly (or to an acceptable degree – nothing's perfect). The advantage of this is that if I made a change in the code that stopped the parser from interpreting a certain type of recipe, I would know about it. It took a while to do, but I believe it was time well spent, as I can now determine when there are regressions in the quality of the various recipe parsers, automatically.
Something which Wil didn't mention is automated/functional testing. We use this heavily in the Confluence team (and part of my work has been to write more test cases), utilising jWebUnit. While unit testing is handy, I can more readily see the benefits of using automated acceptance testing (at least for a web application). Over the past two days, I've been re-writing one of the default macros to add support for a feature in the next release. Obviously, backwards-compatibility with the old version of the macro is essential, so I wrote a bunch of test cases that would cover most areas, including testing different parameter settings, permissions, errors, etc. Had I done this manually, actually implementing the new version (and making sure it worked) would have taken forever.
Things on the "normal" application side are a bit different. With jWebUnit, you can simply inspect the HTML source and make sure such and such link is present, some other text isn't present, and so on. With a traditional application, how to you determine that the correct entries are in a table, and that the correct sheet with the correct text appears? One solution I've seen is Eggplant, which looks great for doing such testing, but costs a fair amount (let's just say a one-user license is a little bit more than the most expensive version of Confluence…), and neither Mat nor I are prepared to spend that much at the moment. So automated acceptance testing is out of the picture, for now.
However, as Wil said, there are other forms of testing that are essential to product development, such as user/beta testing. Ideally, the development process could benefit from the use of unit, acceptance AND user testing, to effectively eradicate a large proportion of bugs in the program. This is something that has been lacking in the last few releases of iPodRip and Connoisseur, and something we're working on for the future.
Phew. What a rant. Basically, unit testing has its merits, but should be used in conjunction with user/beta testing and (ideally) acceptance/functional testing.
