Erlang Central

Difference between revisions of "Timing Properties"

From ErlangCentral Wiki

(essentially a copy of QuickCheckTimingProperties)
 
Line 3: Line 3:
 
==Authors==
 
==Authors==
 
[[User:thomas|Thomas Arts]]
 
[[User:thomas|Thomas Arts]]
 +
 +
http://www.quviq.com/
  
 
==Minimum time to wait==
 
==Minimum time to wait==

Revision as of 17:49, 18 November 2008


Authors

Thomas Arts

http://www.quviq.com/

Minimum time to wait

Assume you have a function startup that takes a certain time to do its job, but is not signaling when it is ready. When testing something depending on this startup function, you may need to wait a pre-defined time, since there is no way to observe that startup has finished.

The following test code illustrates this idea. One has a startup function that returns a process identifier. One can send a message to that process in order to see if the process is ready, but if you send it too early, the message is ignored by the process and you will wait for a reply forever.

test_mything() ->
    Pid = startup(),
    timer:sleep(2245),
    true = ready(Pid).


ready(Pid) ->
	Pid ! {ready,self()},
	receive {done,Pid} -> true
          after 20 -> false
	end.

The code in the function ready ensures that if startup was not completed and therewith the reply never send, the test fails.

Instead of a sleep, one could probably poll with the function ready until a value *true* was returned. After that, the proper test can be executed. However, we have seen situation in which sending a message too early, *causes the startup process to crash*.

We want the value in which we wait for startup to complete as little as possible. That is, it should always succeed, but should wait no longer than necessary, in particular not if we run hundreds of tests. In order to find this minimal value, we can manually run a lot of tests, but we could also use QuickCheck's shrinking to give us that value!

Assume we look for a value between 1 and 3 seconds. The following property fails if we do not wait long enough for the startup function to finish.

prop_minimum_wait_for_startup() ->
    ?FORALL(T,minimum(1000,3000),
	    begin
	       Pid = startup(),
	       timer:sleep(T),
	       ready(Pid) 
            end).

%% Try Tmin, if that fails, shrink to Tmax
minimum(Tmin,Tmax) ->
	?LET(T,shrink_int(0,Tmax-Tmin,Tmax-Tmin),Tmax-T).

The generator minimum takes two values, a minimum and a maximum. It will first try the minimum value. Make sure the tests fails on that value, otherwise nothing will happen! After that the minimum value failed, the value is shrunk. It will start with the maximum value and slowly shrink down to the minimum. The idea is that the maximum should succeed, but somewhere in between there is a local maximum for which the test fails.


After having found out what the minimum time is, one should record this in a test case, such that future tests fail if the timing property changes.

-define(wait_for_startup,2245).

tc_minimum_wait_for_startup() ->
   [MinimumWait] = 
	eqc:counterexample(prop_minimum_wait_for_startup()),
   MinimumWait =< ?wait_for_startup.

Now one can safely use the macro ?wait_for_startup in other test cases that depend on the startup function to be ready.