Tuesday, July 31, 2012

A Test Story for Those Who Don't Like Writing Tests

Without tests, eventually you'll fall down.
My first experience with unit tests sucked. It involved a Java ecommerce bohemoth with good "test coverage" along with more bugs than should be ethically allowed in a commercial product. The company admitted it was one of the worst versions they had ever released (and supposedly some people lost their jobs over it), but the net result was me thinking unit tests were useless. If the same team who wrote the crappy code also wrote the tests, how helpful could they be?

So I decided to just write production code and leave the tests to people with too much time on their hands and no customers to serve.

Yes, that was me being arrogant, ignorant, and stupid. The tests weren't the problem. They just didn't have tests to cover the issues we were finding.
With tests, you have a chance to stay standing.

Years went by, and I got along fairly well. But then I started building something complex. Really complex. A fully RESTful API. It quickly got out of hand. I needed the ability to refactor a major controller for the whole system and know with confidence a change in one place wasn't breaking something else.

Going with Silex and their Symphony WebTestCase pushed me over the edge. Their createClient method helped me quickly test my whole API from end to end. I had no excuses not to write tests. 803 tests and 2,289 assertions later, I'm completely sold on testing.

The story I'm about to tell would have helped the earlier version of me get on board faster and may help you or a colleague finally write those neglected tests.

During what had to be a "Gee, I wonder if this quick hack will work" coding session, I added an attributes table to our system for transactions, customers and subscriptions. I was lazy, so I made a dumb decision and created a name/value table with transaction_id, subscription_id and customer_id fields. Why would we ever need more attributes, right? ;-)

I usually make long-lasting database design decisions, but this was admittedly pretty lame.

So the time came to add more attributes to products, shipments, users, clients, and stores. I knew to do things "right" would involve a database schema change, and it could impact live stores if I screwed up. Most everything we do is versioned in FoxyCart, so what you get today is what you'll have tomorrow. The catch is, all of the versions we support use the same data model.

Testing to the rescue!

The updated table was going to use foreign_key_id and type fields to keep track of the various attributes. The existing versions expect attribute data to be organized in a very specific way. Here's how I went about making the changes without screwing anyone up:

Step 1: Write tests for how it works today.

Ideally you're all hard-core with test driven development, and you wrote your tests before you even started building anything. But if that's you, this isn't your story. You're the guy who is "too busy" to write tests and would rather write code that "meets real needs" in the marketplace.

Am I right?

So I wrote 27 tests with 35 assertions on everything the current class object handled. This included outputting to XML for our API, updating and saving objects, and the like. Keep in mind, this was a real time investment.

Step 2: Break it all.

Once I had everything working, I went ahead and made my database changes in development and refactored all the code. Now almost all of my wonderful new tests failed. Some methods had been renamed or completely removed, others were slightly re-purposed or adjusted. I made a copy of the test file and went to work on it.

Step 3: Mindless coding for speed and profit.

It wasn't exactly test driven development, but this approach made my job so easy. The tests literally told me which line to fix next. Going through and tweaking the code was almost mindless. I can't tell you how many times I said, "Oh, yeah. That doesn't work that way anymore. That's a quick fix."

Soon I had things green across the board again with my new tests against the new class with a new table structure. I was stoked.

Things clicked for me. I got a greater understanding for the power of proper test coverage. I could now confidently roll out a significant change (with a required data migration step) and know our users wouldn't be impacted because the inputs and outputs of the class were verified to be exactly the same as they were prior to the change.

So here's my challenge to you, like the me-of-old, who isn't writing test code:

Start today.

Start with the very next thing you have to code. At first it will be painful, and you'll think you're "wasting time." Trust me, you aren't. Ask anyone who's worked on a lot of real code for more than a few years, and they will most likely back me up on this. You may have gotten by this long because you haven't been faced with a real challenge or you haven't had to fully realize the extent of your current technical debt.

Once you get a taste of the confidence you can have in your system and future changes to it, you'll wonder (as I have) how you got by for so long without writing tests.

Do you write tests (unit, functional, PHPUnit, Selenium, etc)? Why or why not?

P. S.: I know the pictures in the post really have nothing to do with it, but I needed excuse to post them somewhere. :) I got a bunch of cool shots from surfing last week. Check 'em out.


Brian Reese said...

I have found testing to be a handy way to learn about code. I've written tests to see if what I thought the code did was correct. It saves a lot of time this way vs stubbing out a complete web app with various dump statements littered throughout.

Adam Culp (Crazy Floridian) said...

Though I have known for a very long time I should be testing, it was only this past 6 months that I finally started. I started by creating Selenium tests for a new project (http://reqharbor.com) as I went, and quickly saw how valuable it was. After returning from Tek12 I decided to finally take the leap and also write unit tests for the app as well. Wow did it help me, coding-wise. Although the Selenium tests worked I found many places in the app that really needed refactored to help make unit testing possible. (which also meant I should have done a better job to begin with) Ya know what? Once again reading and commenting on your blog has given me a new topic for a blog post of my own, but this time on my geekyboy.com site. :)

Jon Wolski said...

In a way, we always test one way or another. In my reluctant days, I would write code, and then write a quick-and-dirty page to see if the code did what I thought it would. At some point, I got enough layers deep that it was actually less work to use a testing framework than to write a quick-and-dirty script to test what I wrote.

Since then, it's been red-green-refactor. It's like getting positive feedback all day long.

Fred Alger said...

I will consider this a belated apology for giving me the brush-off when I suggested unit tests ever so long ago. Glad you've seen the light! :)

The best software I've ever written has 2x more test code than production code and 90% coverage on all modules. I love using tools that automatically run the unit tests on every single save — like you wrote, it makes my job incredibly easy. Also the act of designing software so that it CAN be tested is a worthy exercise in itself, I've learned a lot by doing so.