Erlang Central

Difference between revisions of "Test Driven Development"

From ErlangCentral Wiki

m
m (Tagged as Article)
Line 1: Line 1:
 
[[Category:Testing]]
 
[[Category:Testing]]
 +
[[Category:Articles]]
 
==Authors==
 
==Authors==
 
[[User:Rvg|Rudolph van Graan]]
 
[[User:Rvg|Rudolph van Graan]]

Revision as of 16:38, 2 July 2007

Contents

Authors

Rudolph van Graan

Erlang is great for any project that uses Test Driven Development. Why?

First of all, you can compile code that references other code that doesn't exist yet. Call this Runtime Binding. The Erlang Virtual Machine treat each module as separate entity and any other modules and functions that it references are only resolved as needed. This means that you can call code that you will write in the future and it will compile.

Let us see how runtime binding can help us develop a module that is useful for encoding and decoding Hexadecimal strings with the test driven model.

We want our module to do two things, translate a hex string into plaintext and vice versa. Let us name our future module hex. Let us think of two simple use cases for our module.

Case 1 - Encode a string into hex format

Let us pick any string. How about Hello World? We know that the ASCII Hex value for H is 0x48. And an e is 0x65. You can verify this by type the following into the [[Shell|Erlang Shell]:

(gdb_rdbms@wyemac)34> [16#65].
"e"

Erlang agrees with us. Encoding the whole string by hand, should give us:

48656C6C6F21576F726C64

Ok, we've worked this out by hand. Now we need a module for it. In essence we want this:

ExpectedHexString = "48656C6C6F21576F726C64",
HexString = hex:encode("Hello World"),
ExpectedHexString = HexString.

This is our first test. Let's write it as a Test Server test suite and call this test encode_test_1:

encode_test_1(doc) -> 
  ["Tests that hex:encode(...) works"];

encode_test_1(suite) -> 
  [];

encode_test_1(Config) when is_list(Config) -> 
  ?line ExpectedHexString = "48656C6C6F21576F726C64",
  ?line HexString = hex:encode("Hello World"),
  ?line ExpectedHexString = HexString.

and run it with testhelper:

(gdb_rdbms@wyemac)37> testhelper:run(hex_SUITE,all).
====================Start of Test encode_test_1======================
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Testhelper catch error undef
[{hex,encode,["Hello World"]},
 {hex_SUITE,encode_test_1,1},
 {testhelper,execute_test,3},
 {timer,tc,3},
 {testhelper,run_test2,4},
 {testhelper,'-run_test/4-fun-0-',6},
 {proc_lib,init_p,3}]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================End of Test encode_test_1========================
Running hex_SUITE:end_per_suite (runtime version)
****************************************************
hex_SUITE:encode_test_1                                                                             failed (61ms) Line 43 []
0 tests passed
1 tests failed

This is the first rule:

Rule 1 
Write a test that states the assumptions that you make for your code

Our test doesn't work. Not surprising as we've not written the code yet. Let's make this test pass. We create hex.erl as follows:

-module(hex).

-export([encode/1]).

encode(String) ->
  "48656C6C6F21576F726C64".

Testhelper is happy now:

(gdb_rdbms@wyemac)39> testhelper:run(hex_SUITE,all).
****************************************************
====================Start of Test encode_test_1======================
====================End of Test encode_test_1========================
Running hex_SUITE:end_per_suite (runtime version)
****************************************************
1 tests passed
0 tests failed
ok

This is strange. We haven't done anything in hex:encode(...). In fact I've hardcode the method to always return our string. Why? I've followed the second rule of test driven development:

Rule 2 
Write the simplest code possible to make the test pass

What is wrong here? Yes, the code doesn't do anything yet. But according to Rule 1, we cannot modify it without a test. Let's add another test, this time encoding "Hello" and see the results:

encode_test_2(doc) -> 
  ["Tests that hex:encode(...) works"];

encode_test_2(suite) -> 
  [];

encode_test_2(Config) when is_list(Config) -> 
  ?line ExpectedHexString = "48656C6C6F",
  ?line HexString = hex:encode("Hello"),
  ?line ExpectedHexString = HexString.
(gdb_rdbms@wyemac)41> testhelper:run(hex_SUITE,all).
****************************************************
====================Start of Test encode_test_1======================
====================End of Test encode_test_1========================
====================Start of Test encode_test_2======================
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Testhelper catch error {badmatch,"48656C6C6F21576F726C64"}
[{hex_SUITE,encode_test_2,1},
 {testhelper,execute_test,3},
 {timer,tc,3},
 {testhelper,run_test2,4},
 {testhelper,'-run_test/4-fun-0-',6},
 {proc_lib,init_p,3}]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================End of Test encode_test_2========================
Running hex_SUITE:end_per_suite (runtime version)
****************************************************
hex_SUITE:encode_test_2                                                                             failed (476us) Line 57 []
1 tests passed
1 tests failed
ok

As expected, test 2 fails. Lets fix it and write some more code into hex.erl:

encode(String) ->
  to_hex(String,[]).

to_hex([],Result) ->
  lists:reverse(Result);

to_hex([Char|Rest],Result) ->
  <<HB:4,LB:4>> = <<Char>>,
  HBHex = if
	    HB =< 9 ->
	      HB + 16#30;
	    HB >= 10 ->
	      HB + 55
	  end,
  LBHex = if
	    LB =< 16#09 ->
	      LB + 16#30;
	    LB >= 16#0A ->
	      LB + 55
	  end,
  to_hex(Rest,[LBHex|[HBHex|Result]]).

Run it through testhelper again:

(gdb_rdbms@wyemac)45> testhelper:run(hex_SUITE,all).
****************************************************
====================Start of Test encode_test_1======================
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Testhelper catch error {badmatch,"48656C6C6F20576F726C64"}
[{hex_SUITE,encode_test_1,1},
 {testhelper,execute_test,3},
 {timer,tc,3},
 {testhelper,run_test2,4},
 {testhelper,'-run_test/4-fun-0-',6},
 {proc_lib,init_p,3}]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================End of Test encode_test_1========================
====================Start of Test encode_test_2======================
====================End of Test encode_test_2========================
Running hex_SUITE:end_per_suite (runtime version)
****************************************************
hex_SUITE:encode_test_1                                                                             failed (528us) Line 45 []
1 tests passed
1 tests failed
ok

We still have a test that fails. Test 2 passes, but not test 1. There was a bad match error on 45. For some reason hex:encode(...) doesn't agree with our own encoding for "Hello World".

Our's:     48656C6C6F21576F726C64
hex.erl's: 48656C6C6F20576F726C64

Ah... We mistranslated the space as 0x21 instead of 0x20.

This is a very important feature of test driven development. We can catch bugs in other lines of code that we didn't expect. This brings us to rule 3:

Rule 3 
You are not finished until all your tests pass

Case 2 - decode a hex encoded string

In essence this use case is the opposite of our Case 1.

The abbreviated tests could be:

Test Number Description
Test 2a
 ?line "Hello World" = hex:decode("48656C6C6F20576F726C64") 
Test 2b
 ?line "Hello" = hex:decode("48656C6C6F") 

So when are we done with hex.erl? This brings us to rule 4:

Rule 4 
You are done when there are no more cases that you can think of

I can immediately think of a couple of other tests:

Test Number Description
Empty string encode
 ?line "" = hex:encode("") 
Empty string decode
 ?line "" = hex:decode("") 
String not modulo 2
 ?line hex:decode("486") 
Non-hex characters in string
 ?line hex:decode("#$%2") 

The last two tests are strange. Here we actually want to check that code will NOT work for a specific use case. Because we follow the Erlang rules, the last two cases must crash. Here is such a test case:

invalid_data_1(doc) -> 
  ["Tests that hex handles invalid data correctly"];

invalid_data_1(suite) -> 
  [];

invalid_data_1(Config) when is_list(Config) -> 
  try
    ?line hex:decode("#$%2"),
    testhelper:fail("This test should have failed!")
    catch 
      error:badarg -> ok
    end.

Note how the test fails if it runs over the test line without a crash, but with a crash everything is ok.

The last rule is a simple one:

Rule 5 
Bugs don't exist unless there is a test for them

or alternatively

Rule 5 
You may not fix a bug without writing a test for it first


The five rules

  1. Write a test that states the assumptions that you make for your code
  2. Write the simplest code possible to make the test pass
  3. You are not finished until all your tests pass
  4. You are done when there are no more cases that you can think of
  5. a Bugs don't exist unless there is a test for them
  6. b You may not fix a bug without writing a test for it first