Test Driven Development

Borland Conference 2005

by Charlie Calvert

Index

• Part One: Introduction to Unit Testing and TDD
• Part Two: Basic Syntax: DUnit and NUnit.
• Part Three: Mock Objects and Stubs
• Part Four: Refactoring and Patterns.
• Part Five: NAnt and NCover

Initializing class to with interface for six step Noel example.

Overview

• Unit testing and Test Driven Development (TDD) got its start with JUnit, a tool built by Kent Beck and others from the Smalltalk community.
• Smalltalk was a groundbreaking computer language with an illustrious community that thrived in the early 80s.
• Test Driven Development vs XP vs Agile Development
• You do not have to use any particular methodology with unit testing. However, methodologies such as Agile or XP are closely associated with unit testing. Think of them as competitors to methodologies that recommend writing out a long specification before you write any code.
• Types of Testing: Regression, Functional,
• Refactoring
• Mock Objects
• Patterns
• UML and Agile modeling.
• Claim to be tested: Unit tests promote good architectural habits. Good designs falls out naturally from the principles of Test Driven Development
• Demos are explained here.
• And go to http://www.elvenware.com to get updates or more info.

Some Links to More Articles I've Written on this Subject

• Test Your DotNetGui with Mock Objects
• NUnit Code Coverage with NUnit
• Using NUnit in Other Tools
• Building C# Projects with NAnt
• Using NAnt to Update Config Files Part 01
• Using NAnt to Update Config Files Part 02
• Creating Custom NAnt Tasks

Part I

Introduction to Unit Testing and TDD

The Different Kinds of Tests

There are several different kinds of tests, and the boundaries between them can be quite fuzzy at times. Nevertheless, it is possible to make at least a few generalizations which can help you navigate the lingo found in most texts and web sites.

Unit Tests

• This is the lowest level of tests.
• It is very fine grained, and works on a method per method basis.
• You usually test one class at a time, with mocks standing in for other classes.
• We ware usually talking about only public methods
• More about this subject later.

Integration Tests

• The classic example is testing if two classes can work together.
• But really any two abstractions, such as a class, package, assembly, service, or a subsystem, would do.
• The point is that two distinct entities are being tested to see if they play well together.
• A lot of people, including myself, use unit test technology to to run many integration tests.

Functional Tests

• Tests to see whether a subsystem or class meets a requirement.
• A functional test on a compiler would test whether the compiler can compile Hello World
• A functional test on an FTP client would test whether the client could connect, or download, or list a directory.
• Functional tests are sometimes run on whole applications, while unit tests and integration tests run on subsystems or methods.
• A lot of people, including myself, use unit test technology to run at least some Functional tests.
• Once you start testing a whole application, you are not really in the realm of unit testing any longer.

Stress/Load/Performance Tests

• These tests verify that an application can handle a load or perform in standard set of time.
• Can a server handle 50 clients? How about 250?
• How long does it take to compile Hello World? How about an application with 500,000 lines of code?
• You generally can't use unit tests to do this kind of thing, though there may be a few simple tests you could run

Acceptance Tests

• Final tests, usually by the customer, asserting whether or not the application meets the specification.
• The question is whether or not the customer is satisfied.
• Usually includes both functional and performance tests.
• Issues like appearance, ease of use, etc, are part of this level of tests.
• I think of acceptance tests as being run by the client, or a by a very disengaged manager.
• Unit tests help the developer get the application ready to pass an acceptance test.

Regression Tests

• Fundamentally a different kind of tests than those described above
• Change a feature, then come back and run tests to confirm that everything still works.
• Unit, integration, functional, stress and acceptance tests can all be regression tests.
• All the matters is that you are going back to confirm that nothing is broken.
• We automate unit tests to run at least daily to confirm that nothing has been broken.

Unit Testing

• A non-intrusive technique for creating tests to verify that a program behaves as expected.
• Many unit testing tools are free, open source projects
  • This makes them easy to extend.
  • There is always the possibility that someday they won't be maintained anymore, but that is unlikely in the case of big projects like JUnit or NU nit..
• Emerged from the Extreme Programming/Agile field of development.
• You can be a proponent of Unit Tests and/or Test Driven Development and still dislike XP or Agile technologies.
• Learning the syntax for writing unit tests is simple, learning to design them correctly is hard.
• It is a very labor intensive activity. You write tests for each important method that you create.
• The goal is to write tests covering all your code. Then run the tests, and if they pass, then in theory, your code works.
• Just because a unit test passes, that doesn't mean the client is satisfied. It's not an acceptance test. But it is a step in the right direction.
• TDD: Test Driven Development. It's not just an acronym, its a development methodology.
• Unit testing is now a mature technology.

Why Use Unit Tests?

• They provide the confidence you need to stand behind your code.
• They can, in theory, eliminate bug hunts. You know where your bugs are, because the unit tests reveal them as soon as they are introduced.
• Unit testing makes refactoring practical. We refactor to help us find the best possible design for our code.
• Specify requirements. Write tests that describe what software should do. When you the tests pass, the requirements are met.
• Create documentation. This kind of documentation is very practical, and never out of sync.
• Rapid feedback.
• They encourage communication and change.
• Ultimately, unit testing is really about designing applications. It is a technique for helping you find the right architecture.

Reasons Not to Use Unit Tests

• These aren't necessarily good reasons, but they are reasons why people don't adopt this technology.
• It requires work. A lot of work. Perhaps 50% of your programming time will be spent writing tests, if you go out at full bore.
• In fact, it radically changes the way you right code because you need to make time to write a lot of tests.
• Managers might not understand what you are doing.
• To do it right, you need to understand a philosophy of programming. If you don't understand that philosophy, you will still get benefits, but not as many as you expect.
• It can take time to learn how to write good tests. At first, you might bungle things, and find that you need to rewrite tests.
• It would be a bummer if you put a lot of time into tests, and then didn't do it right.
• Writing tests should be easy, but it is not easy until you learn how.
• Unit tests won't save a really, really bad project. At least not overnight.
• In fact, it is best if you can start using unit tests right from the beginning.

Part II

Unit Testing Technology and Syntax: NUnit and DUnit

Install and Setup

• In the shipping versions of BDS 2006, everything should come pre installed and completely set up.
• Lets go over the details, just in case.
• Also, these products will be updated from time to time, so you might want to understand how they are set up, so you can upgrade.
• I'm going to work with NUnit and DUnit, but any unit testing project out there will work in more or less the same way.

NUnit

• This is the DotNet unit testing tool for DotNet languages Delphi and C#.
• Download the free, open source NUnit project from http://www.nunit.org/
• Install NUnit where you place other executables.
• NUnit should be registered with Delphi and Visual Studio automatically after the install.
• If that fails, bring up references and search for NUnit.Framework.dll from the bin directory of the install.
• Three assemblies: Your test code, Your code to test, and NUnit.Framework.dll.
• This must be in your references for all NUnit libraries. Delphi should take care of this for your automatically.
• You create libraries, which are loaded into an executable called NUnit.

DUnit

• This is the Delphi for Win32 testing library.
• Download the free, open source DUnit from http://dunit.sourceforge.net/
• Install DUnit into your source directory tree. I don't like to install it into a bin or My Programs directory, but into the directory tree where I store my source.
• In Delphi 7, I set up a Delphi environment variable called DUNIT that points that the DUnit src directory. In may case: c:\src\srcpas\dunit\src.
• You create a program that contains the DUnit GUI. You build and run the test program using the DUnit libraries that come with the source.

Getting Started

• Test a particular method in a class and see if it returns the expected values
• This is different from trying to run a GUI app, enter data, and see if the app behaves correctly.
• The setup and teardown methods.

First Demo: Getting Started with NUnit

• Let's see it in practice.
• First in Delphi, then in CSharp.

NUnit Basics

• NUnit Quick Start
• In NUnit, put the attribute [TestFixture] before your test class, put the attribute [Test] before the public methods you want to use as tests in NUnit.
• Make sure you have added NUnit.Framework.dll to your references section.
• NUnit
• You don't have to construct Suites or register your tests. Just load the DLL in NUnit.

Second Demo: Getting Started with DUnit

• Let's see it in practice.
• First in Delphi, then in CSharp.

DUnit Basics

• Add TestFrameWork to your uses clause
• In DUnit, descend your test class from TTestCase.
• Append the word Test before each public method you want to run as a test,
• Here is an example of the syntax.
• In DUnit, you write code to construct Suites and register your tests.

DUnit Links to Articles

• DUnit Quick Start
• Charlie's description of the DUnit Structure example
• DUnit related tools.

GUI vs Text Tests

• You can run tests either inside a GUI front end, or from the command line.
• The GUI front end is colorful and "fun to use." This is the from red to green theory.
• The command line tools are easy to automate.

Color Coding

• Green is success, red is an exception, pink is a failure.
• Only occurs in GUI mode
• Part of the XP culture.
• Look at the JUnit site to see the color culture in action

The Config File Issue

• You will often need a config file for your tests to define certain variables in your code.
• The config file is just like an app.config or web.config, but you need to put in place yourself.
• For instance, if your tests are in MyTests.Dll then create the following file:
• \bin\debug\MyTests.dll.config.
• Usually you want it in \bin\debug, but there may be reasons to put it someplace else.

NCover

• Web Site: http://www.ncover.org/
• Downloads: http://www.ncover.org/Downloads.html

NAnt

• Follow this link to view the outline for the NAnt part of the talk.
• If you are not already on the Elvenware site, the most recent copy can be found here.

Part III

Test Driven Development: Refactoring

Agile and XP

• Much of the best work on Test Driven Development has been done by advocates of Agile and XP programming methodologies.
• You can read more about these subjects in the attached document called Agile and XP Development.
• It is important, however, not to get lost in theory. Ultimately, what one needs to do is write tests.
• Ironically, this is what Kent Beck would also argue. Stop with all the planning and theorizing and start writing code. When it comes to programming, code is concrete, plans and theory are just so much paper.

Third Demo

• Simple Refactoring

Write Tests First

• Write tests before you write your code
• Unit testing is usually top down, not bottom up.
• Write tests to help you define your requirements
• If your tests pass, then your requirements are met.
• Sometimes you just have a list of tests to write
  • Your write one test, then get the code to pass
  • Then write the next test on your list, then get it to pass. Etc.
• If your code doesn't work, you should write a test documenting the failure
• Then fix the bug. The point is, you write tests first!

When to Write Tests

• Don't write tests that you don't need. Test when there is uncertainty. If you have no doubts, then you probably don't need the test.
• Don't try to anticipate what tests you'll need. Instead, write tests when you need them.
• Don't do anything uncertain without first writing tests. Before you refactor, write tests. Before you add a new module, write tests.
• If you find a bug in your program, write a test that reveals it.
• If you can think of a exceptional situation that might cause problems, write a test to document it. For instance, divide by zero, or passing in null as a parameter. Remember, a test can pass if it raises the exception you expect it to raise.

Communication and Change

• If your program is covered with unit tests, then you can change it.
• If you can change your program, then you can feel relaxed when talking to clients
• If you talk more often to clients, then you know the problem domain better.
• You know how to write code, the client knows the problem domain. The more you communicate, the better chance you have of creating a successful program that addresses the right problems.

Caveats

• Beginners always use it half way, but you won't see the real benefits till you embrace it fully.
• Not great at testing the interface.
• It's not nearly as easy as it seems at first. It takes both discipline and creativity to maintain a test suite.

XUnit Family

• DUnit Delphi
• NUnit .NET
• JUnit Java
• CPPUnit C++
• PyUnit Python
• Test::Unit Ruby or (RubyUnit)

Red Green Refactor

• This is the philosophy in a nut shell.
• Write the test first. Stub out just enough of the actual objects to be able to compile. When you run the tests, they fail, because the code is not written or not complete yet. Failed tests show as red in GUI mode.
• Get the tests to pass. With the GUI front end, the tests show up in green when they pass.
• Now refactor your code to make sure it is as simple as possible. Then run again to make sure nothing is broken.

Checks

Write tests that check for expected failures.

• Check
• CheckEquals
• CheckNotEquals
• CheckNotNull
• CheckNull
• CheckSame
• CheckException
• CheckInherits
• CheckIs

Problems

You should cover all, or nearly all the classes in your program with tests. How are you going to find the time or skill to write all that code?

Many people suggest that you write the test first, and the code second.

Some classes are hard to test.

From the point of view of a Delphi programmer, some forms are going to seem impossible to test.

Finding the answer to these kinds of questions turns out to be a complicated quest.

Architecture Philosophy

• http://www.clrstream.com/Downloads.htm
• http://c2.com/cgi/wiki?ExtremeProgrammingRoadmap
• http://c2.com/cgi/wiki?UnitTest

Refactoring

• A key purpose of Unit Testing is to support refactoring.
• You should feel comfortably changing your code. If you make changes, then you can run tests to confirm that all is okay.
• Write tests before you refactor.
• It also encourages refactoring. You must refactor almost every existing project to make it compatible with Unit Testing. This is not just a good thing, in some ways it is the good thing.
• I've included a separate document about refactoring. Martin Fowler is one of the key authors to read when you discuss refactoring.

Design

• Follow this link to Design section of the Refactoring article

Simplicity

• Follow this link to the Simplicity section in the Refactoring article.

Patterns

Patterns are a way of preserving programming lore.

Patterns are lists of proven techniques for solving common problems.

You can read more about patterns in this attached article.

UML and Agile Modeling

Modeling techniques can help you discover the best architecture for your program.

For more information on this subject, see the attached article called Introduction to UML with Together and JBuilder.

Getting Stuck

There is going to come a time during development when all of a sudden you are going to say: "Hey, this doesn't work. You can't write tests for the kind of code I want to create." That is natural, but you have to understand that there is still further for you to go. There is more exploration necessary. In fact, there probably is a solution, only it is requires that you start thinking in new ways about how to write your code.

Simple Tests

Nothing is more important than keeping tests simple.

If your tests are hard to run or hard to interpret, then you might as well not use TDD.

UI Testing

You don't test the UI, instead, you design your code so that it can be tested in a non-interactive manner. After all, you don't need to test an edit control to know if it works. That's someone else's job. You just need to test the data that goes into and out of the edit control. Put that in a separate, non-visual class, and then you are okay. This is very much at the heart of Unit Testing philosophy.

Build Process

• Using tests for development and their role. (mention build process and want, make, ...)

Exception Handling

• Exceptions play many roles in unit testing.
• Sometimes a method should raise an exception under some circumstances.
• Unit tests can test for this circumstance. The test will pass only if the exception occurs.
• You will never see the exception, it will be suppressed. But your test won't pass unless the exception occurs.

Setup TearDown

• There are two methods that you can place at the top of your code to help setup your tests and to help finalize your tests.
• The setup method is a bit like a constructor. It runs before each of your tests. Use it to create global variables or data that you will use in your tests.
• The teardown method is a bit like a destructor. It runs after each of your tests. Use it to clean up any global variables or data that you will use in your tests.
• If you are testing a class called MyFileParser, then typically you will create an instance of it in your set up method, and destroy it in your teardown method.

Fixtures

• The word fixture is used in Unit Testing in many different ways.
• It sometimes seems there are as many different interpretations of this word as there are programmers who use unit tests.
• I use the word fixture to describe the setup and teardown methods of a test.
• In particular, if you have tests that inherit from a base class that define your setup and teardown methods, then I regard that base class as a test fixture.
• Be aware, however, that there are many, many, other interpretations of this word. Many of them are very authoritative sources. My interpretation is just one of many.

Suites

Test Suites

MySuite := TTestSuite.Create('This is my suite');
MySuite.AddTests(TMyTest01);
MySuite.AddTests(TMyTest02);

But mostly you just do this:

TestFramework.RegisterTest(TMyTest01.Suite);
TestFramework.RegisterTest(TMyTest02.Suite);
GUITestRunner.RunRegisteredTests;

Single Unit?

• Should testing code go in the same unit as the class you are testing?
• Placing code in conditional compilation in one unit. Then just define testing.
• My opinion is no, this is not a good idea.
• Because C# specifies namespaces, it is clear what to do there
• But in Delphi, a unit is a namespace, and then the decision becomes more difficult.

Mock Objects

This section is in its own set of files, found here.

Summary

• Unit testing is an art, not a science.
• Understanding the syntax gets you up and running.
• Writing tests first gets you 50% of the way there.
• Understanding and using Mock Objects helps seal the deal.
• When we learned how to write objects, we had a long learning curve
• The same is true with mock objects.
• It takes a long time to learn to do these things the right way.
• But we never stop learning. There is always ways to improve our unit testing code.