June Agile Groupies Meeting - Random Discussion thoughts

[Helene Gidley]

20 people attended this evenings meeting! We had folks from Owosso, Southfield, Farmington Hills, Bowling Green Ohio, and of course Ann Arbor.

People are interested in writing tests for legacy code. Ideas shared among the group were:
1. Working with legacy code book by Michael Feathers
2. begin with some tests, minimize total costs and work to maximize the amount of code to be tested
3. Setting up the testing environment is very time consuming. The testing environment should have a set up and tear down structure. Sometimes this is hard to sell to management.
4. Getting real data for testing - think of CRUD - create, read, update, delete - create the data you need for testing, start with that.
5. Test boundary conditions in range situations. Often most errors occur within -1 and +1 of the ranges.
6. Only test what you write
7. Only deal with what you have to do to get it done (do the simplest thing that will work). Refactoring should be an integral component of your overall coding.
8. Separate your test code from the development code discussed - because the run time test code need to be deployed along with the code (do you expect your users to be changing prod code? Why else would they need the unit tests?)
9. A unit test never touches the production database
10. A test case is the precursor to writing the code. The test defines what code to write.
11. TDD forces you to focus on writing tests to validate the requirements with the minimal amount of code written to deliver value.


Joe Hershey and Jay Wren paired up to code. The benefits of pairing discussed:
constant design feedback, keeping you focused, ensuring you're completing what you expected.
Visual Studio used, coding in .net

Why TDD?
Example: Code built, last minute change needed, team that didn't have TDD spent all night checking the code - team that had TDD ran their tests and shipped in hours.
TDD is more about design not tests. TDD forces better design in order to create the tests.
Comment made that even TDD code can be spaghetti code. Focus is to improve design of the code.
Refactoring an important component.
Using TDD, you catch requirement gaps as you code, rather than beginning to code and guessing on what is needed.
Resharper, Eclipse, Code Rush, all provide usability tools for testing.

Pattern for test creation: range, act, assert

What specifically should I test - what values?
Suggestion to involve customer/product owner involved all the way through on the development to help define the requirements and functional specifications. Goal is to have them own the code as much as the development team does rather than just throwing it over the wall.

Identifying a user story requires defining done. This is the criteria for the test.

Mentioned: Cucumber = acceptance test framework for Ruby

[Don Wells]

Unit tests for legacy code.

Write the tests before you make a change. Write tests only for the section you are going to change. The rest of the system will get a turn later. If you follow this plan the hot spots, code that changes the most, will be covered first.

The first test is the hardest to write. Most people give up. I have a proverb: Where there is a will there is a way to test. Your problem is that the code you have was not designed to be easy to test. So you need to test great big hunks of code all at the same time. Possibly even run the entire system end to end in extreme cases. That means you will need large amounts of test data to run against. That in turn means you need to write some code to create that data or put it into a data base of some kind. Creating a tangled mess of objects out of thin air can be very complex, create a tool to help you. Get the source code to your unit testing framework and modify it. (Assuming you failed to follow my advice of writing your own.) You are going to be spending lots of time using that unit test framework get to know it well enough to make any extensions you need.

Now that you have one test the second test is easy. Cut and paste it together from the first test. Same for the next few tests you need to verify the code you are about change. You have started and can’t turn back now. How much to test is an art form. You don’t want to test every combination of data because that will actually cost more than if you tested nothing. You heard me, don’t test everything. Test “most” of everything. What “most” is can change, your goal is to balance the cost of tests versus the cost of finding a bug in production. Find the sweet spot.

It will take you between 9 months and a year to reach about one third of the code covered. The good news is that the code that is covered are the hot spots. That means that even with only one third of the code covered you will reduce your production bugs by a huge amount. That in turn means fewer resources dedicated to fixing bugs and more resources on creating new functionality and refactoring crappy designs. During this time you will have created several tools to help you manage test data.

You might find that you need to break some of the unit test mantra to get coverage. Large tests that look like system or integration tests are common. Tests that depend on each other and must be run in a strict order are also common. You might need to make your testing framework aware of the database so that it can manage connections when tests fail.

During the next iteration of 9 months to a year you will complete your coverage. You have tools to help, you have example tests to cut and paste from, and you have extra people to look for uncovered areas and add a couple tests. So 1.5 to 2 years after you start you will notice something very strange, something you may never have experienced before. That’s right, no new production bugs. First a week goes by before another bug comes in, and then a month goes by. Before you know it a year has gone by and production doesn’t have any bugs other than honest misunderstandings and disguised requests for new functionality.

So you are now 18 months to two years into unit testing, what next? Refactor like crazy. You can now make huge sweeping changes to the basic design without having to fear breaking something in production. On one project we started over from scratch because the code was just that bad. We had the rewrite done in a couple months while at the same time maintaining the old system. (We had way too many people on that project fixing bugs when I first showed up.) One feature had to be incorporated into both the old and the new system. The old system took us 30 days to implement it. The new system took us 3 days to implement it. On another project we just refactored…everything. We redesigned the entire system from top to bottom step by step without introducing a production bug. At the end we had about half the code we started with. Oh joy of joys we finally get to implement the design we always wanted.

[Don Wells]

And another useful technique is the testing subclass. Subclass an untestable class and add methods and assessors to help you test it. Don’t deploy the testing subclass or allow it to be used as part of production code.

For databases there are 3 basic ideas. First is the production backup. Take a backup of production and restore it into a test database. This guarantees that stored procedures are identical to production. Eventually you will have tests for stored procedures, right? But out of the starting gate just make sure you use the production versions, even if they are the wrong versions.

If you have sensitive data in your database you can’t just use a backup. If you have stored procedures that take forever to run with full real data you might not want to use a backup. So the next choice is an empty database. It isn’t much different to use than a backup except that you may not know if the stored procedure versions are identical to production.

The third choice is the mock database. I can create a database connection that looks at the query sent and sends back a pre-specified result set. I will have a list of query and results in my mock database. This approach can require huge amounts of maintenance since your database has just become agile and open to changes of all sorts.

On the last project we used a combination of the first and third methods for testing. You don’t have to stick to just one way to test you know. One of the changes we made to our unit testing framework was to have the framework open the database connection and start a named transaction. After the tests run it rolls back the transaction. This speed up our testing considerably since we were not opening, rolling back, and closing our database connections for each and every test. With hundreds of unit tests it can take a while.

We also had the test framework insert some basic test data. A couple of pretend clients and the data that goes with them. So long as each test rolls back its own changes the test data remains viable for the next test. Don't get too fused about failure cascades. (One test failure causes another test to fail and so on.) This is how things go when you have legacy code to test. Just always start fixing first test that failed.

That leads us to some more advice. Unit tests should take about 10 minutes to run total. No more than 15 minutes. When you get to 15 minutes it is time to refactor and optimize your tests. Possibly you may need to optimize your code too.

[Don Wells]

Here is a presentation I used to give on unit testing with Java. It is a little of date. Cancel that, it is a lot out of date! You can tell because it was written when we called TDD test first design. I offer it up because going through it you might discover some things you want to ask questions about.

[Don Wells]

P.S. be careful, the color scheme was designed to keep people awake. I am a powerpoint professional, don't try this at home.

[Don Wells]

P.S.S. Any references to corporate entities that might actually exist are purely coincidental.

[Don Wells]

“Identifying a user story requires defining done. This is the criteria for the test.” A word of caution here: User stories are usually written at a much courser granularity than you want your unit tests to be in. Traditionally user stories turn into acceptance tests, not unit tests. Unit tests are written to support the higher level acceptance tests. Having said that, it is true that having a well defined user story will help you decide what to test next.

[Helene Gidley]

Don, these are all great tips! And the slides are a wonderful sharing with the group - many thanks!

I can't agree enough on defining done for a user story. I've found this to be the hardest task for most of the teams I work with. It's too easy to write a task, or a feature, but leave out the critical piece of how do you know when you're done?

Writing user stories is often the first place I start with when working with new clients.