Unit Testing Thoughts

by msimpson 11/13/2007 6:03:00 PM

I had an interesting discussion with a former colleague, Clay Lenhart, where he said he's currently disenchanted with test-driven development because of the need to constantly update the unit tests as the application code changes.  This struck me as a valid point; TDD, viewed in that light, would seem to better suit a traditional waterfall development process than an agile one.  Agile processes supposedly embrace the change that will inevitably occur in requirements, design, etc., and try to mitigate its impact by (among other tactics) reducing the effort involved with refactoring.  Writing the test first seems to assume that you can get your requirements right up front - the hallmark of your father's development process, right?

The more I think about this, the more I think there's an appropriate time to write unit tests, and it can be discovered by looking at code churn.  When the churn in a module falls to a certain value, the code is stable enough to test, and only then do you write the tests and automate them.  This allows you to operate at maximum speed, and helps keep you from avoiding refactoring, early on when the code is being ripped apart and put back together wholesale; it also preserves the benefits of unit testing later on in the project. 

Broadly speaking, I would relate this approach to my general approach to decision-making, which is to defer decisions until I feel that either 1) I have to make a decision to proceed, regardless of the information available, or 2) little or no more useful information is forthcoming.  In an even broader sense, the point is that there's a tradeoff between doing things sooner vs. later, and you have to find the point on the timeline that yields the most benefit.  Deep Thoughts By Mike ;-)

Speaking of unit testing, I'm thinking of developing an API and tool that will generate test data based on policies specified by the developer, and in general facilitate unit test development.  Here are some ideas:

Test Data Generation

  • Provide scheme for specifying generated data formats
    • Minimum length (character data)
    • Maximum length (character data)
    • Minimum value (numeric data)
    • Maximum value (numeric data)
    • Hardcoded portions
    • Random portions
      • Within limits
    • Data types (string, numeric, strings of only certain characters, etc
    • Values from another table
      • Generate a dependency in the test data generation, so that if we’re generating data for the remote table, we do that first
    • Manual specification of test data by the developer
  • Specify test data generation policies for each desired table and column in the database
    • Number of rows desired
    • Whether to violate data generation policy
    • Provide reasonable defaults so not everything needs to be specified
  • Allow scheduled, programmatic or ad-hoc test data generation
  • Allow generation of test data without stepping on existing (presumably user-entered) data

Unit Test Support API

  • Clear table
  • Generate test data in accordance with specified policies
  • Assert existence of specified rows
  • Assert existence of a specified number of specified rows
  • Assert non-existence of specified rows
  • Delete specified rows
  • Delete test data generated as part of this test
  • Support notification upon success and/or failure (does NUnit do this already?)
  • Support generation of a report with test results (does NUnit do this already?
  • Clean up

Unit Test Generation?

  • Template-driven
  • Mocking (probably skip this in favor of third-party framework or built-in VS support)

Anyway, the first step is to research what's already out there.

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Tags: , , , , , , ,

Software

Related posts

Comments

3/19/2008 9:32:23 AM

Josh Gough

Mike,
Good thoughts,

One thing that is interesting in this area is MS Research's PEX:

It is a Unit Test generator:
http://research.microsoft.com/Pex/

ScottHa interviewed the author about it here:
www.hanselminutes.com/default.aspx?showID=111


Take care,
Josh

Josh Gough us

Add comment


(Will show your Gravatar icon)  

  Country flag





Live preview

8/27/2008 4:47:25 PM

Powered by BlogEngine.NET 1.2.0.0
Theme by Mads Kristensen


Calendar

<<  August 2008  >>
MoTuWeThFrSaSu
28293031123
45678910
11121314151617
18192021222324
25262728293031
1234567

View posts in large calendar

Pages

    Recent posts

    Recent comments

    Disclaimer

    The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

    © Copyright 2008

    Sign in