In the last month, I shared my experiences with Spring Batch—not all positive—which led me to ponder the political correctness of criticizing open source software. Time to move on.
This week I’ve been writing unit tests for Java. In the past I’ve always used JUnit for no other reason than it’s the first such unit testing framework I learned. When doing repeated tests, you have a couple of options.
Firstly, you can create a single test that loops across a list of inputs and expected results. For example:
public class RepeatTest {
private final static List TESTS = Arrays.asList(...);
@Test
public void testData() {
for (TestData data : TESTS) {
assertEquals(doSomething(data), data.getExpectedResult();
}
}
}
You get the idea. This of course is coarse-grained.
Wherefore Art Thou, Documentation?
I had a recollection from years past that you could create a parameterized test set up in JUnit (although I had to google to figure out that’s what it was called) so I went looking for info on it.
Just take one look at http://www.junit.org/. Seriously, take it all in. I have never in my life seen a site so woeful for a framework that is as commonly-used and allegedly mature as JUnit. Considering I’ve seen Apache projects whose documentation borders on the criminally negligent, that’s really saying something.
Take Log4J as a prime example. I will say that the introduction to Log4J is quite reasonable. The problem? That’s all there is. Sorry, let me retract that: there is documentation. You just have to pay for it. I’m not sure how much influence Ceki Gülcü—the author of that book and the guy who started Log4J—still has (he has since moved on) but if anyone who was running a project (and thus deciding to a large degree what the priorities are and who does what) then if they were also selling documentation to that same project it in the very least has the appearance of a conflict of interest.
But back to JUnit. Consider the documentation. The "cookbook" is only a cookbook in the sense that “tear lid and microwave on HIGH for 8 minutes” on the back of a frozen dinner can be considered a recipe. There is an example of what I was looking for in the Javadoc for @Parameterized but the real answer came from the blogosphere, notably Writing JUnit 4 parameterized tests with varargs data and a few others.
I’ve previously evaluated TestNG as a unit testing framework. Although there didn’t seem to be anything wrong about it there wasn’t anything really compelling either. Test dependencies and the like just seemed complicated to me and not how I tend to use unit testing. Plus you generally find people know JUnit works and people know how to use it. Well, the basics anyway: @Before, @Test and @After, which covers 95%+ of cases.
It’s worth nothing that TestNG has extensive support for parameterized tests.
It’s the Documentation, Stupid
That’s not a compelling reason to switch though but I’ve now found one: documentation. I am gushing like a schoolgirl over TestNG’s documentation. There are examples of how to integrate into different IDEs—not just Eclipse (my pet peeve as a diehard IntelliJ IDEA user). The user guide is detailed and extensive. There is even integration documentation for Ant and Maven as well as a migration guide for JUnit (here is a comparison).
Documentation alone is reason enough I will in future choose TestNG over JUnit without exception where I have a choice.
Some open source projects have excellent documentation. I consider Spring documentation to be the gold standard of OSS documentation. Even Spring Batch, which I criticized, has good documentation. It’s not perfect. It’s coverage in some areas is a little light and it could use some more examples but overall it’s pretty good. TestNG documentation is excellent.
This will inevitably invite several counterarguments.
RTFS
This one is predictable. Some will argue that the source code is sufficient documentation. Bollocks to that. While at some point it is inevitable you will end up reading or stepping through the source code of any framework or library you use for any non-trivial purpose, the fact that you have the source code is no substitute for high-level documentation that describes the overall architecture and design principles as well as how to get started and how to do common tasks.
DIY
Another popular defence of poor OSS practices is you can get involved and do it yourself. While theoretically true it is typically completely impractical. For one thing, before you can document something you have to know it. How do you learn it without reading the (non-existent) documentation?
Depending on the size and complexity of the project in question, it might takes anything from days to months or even years to get up to speed (eg the Linux kernel). Even spending days getting up to speed on something that isn’t documented is typically time you don’t have because you have a job you’re trying to get done.
It’s also unlikely to happen. Once you figure out how to do something, what are the chances that someone will then turn around and spend a few more days writing up some half-decent documentation? Chances are they’ll be so frustrated by their efforts to date it’ll simply be time to move along to the next problem.
It’s Free
This is the one that bothers me the most. It comes up in any criticism of open source and is little more than a justification for laziness. The idea seems to be that you can’t complain if it didn’t cost you anything.
Oh but you can.
You see, many people approach open source projects as a means of self-promotion. They simply want to make a name for themselves. That’s fine. I have no problem with that (I am after all writing a blog). The problem is it’s not about you.
Depending on your job, your company and your jurisdiction, if you are a consultant of some kind and thus dealing with people who don’t know a lot about software development you will typically have a duty of care to that party. You are making representations regarding your professionalism, capabilities, skills and/or experience. That other party is making has legal recourse for negligence (or even fraud) should you misrepresent yourself or fail to fulfil your duties.
When you create a library or framework or just some significant piece of code that you decide to share you are making representations—implicit and/or explicit—regarding it’s efficacy. What’s more your users will typically know a lot less than you.
What you should remember when you hand out “free” software is that if that software doesn’t work as advertised, it’s unreliable or the documentation is non-existent or misleading then you are doing it’s users harm. While you may not be legally liable for such deficiencies you have an ethical responsibility to ensure what you deliver does what it says it does.
Conclusion
Open source is no excuse. It costing nothing is no excuse. Like doctors, we programmers should first do no harm. We don’t have an Hippocratic Oath but that doesn’t mean we shouldn’t do what’s right.
I for one am tired of accepting mediocre libraries with documentation that is coming Real Soon Now [tm] (“Under Construction”). We should not tolerate second-rate offerings. Reward projects like TestNG who have done the right thing by us.
I don’t care if writing documentation is not fun or sexy. I care that you can explain how it works and provide me with some confidence that it does what you claim it does. Otherwise I’m just not interested anymore.
Posts
