September 1, 2014
23:01 -0400
Hubert Chathi: is glad that the rain mostly held off today until we finished our arbour
August 24, 2014
20:48 -0400
Hubert Chathi: this weekend, we got all the posts in for the # arbour, and half the posts in for our own
August 5, 2014
14:22 -0400
Hubert Chathi: Tor relay up and running to help defend online anonymity
10:11 -0400
Hubert Chathi: finished the grape trellis at the # last night
June 29, 2014
18:14 -0400
Hubert Chathi: finally got the posts for the grape trellis in for the # yesterday
June 4, 2014
22:53 -0400
Hubert Chathi: got a 419 scam via snail mail yesterday...
May 22, 2014
15:59 -0400
Hubert Chathi: Dear, if I wanted it animated, I would have asked for it to be animated.
May 15, 2014
11:15 -0400
Hubert Chathi: Goodbye to my father in law, who passed away on Tuesday. You will be missed.
May 13, 2014
11:01 -0400
Hubert Chathi: Everyone else posts pictures of their food on Instagram. What's wrong with posting a picture of a baby's food?
May 9, 2014
22:35 -0400
Hubert Chathi: What is a housing co-op? Video featuring some members.
April 29, 2014
15:21 -0400
Hubert Chathi: I hate it when my tools lie to me
April 24, 2014
17:23 -0400
Hubert Chathi: Beware of phone calls claiming to be from the RCMP #
April 3, 2014
11:12 -0400
Hubert Chathi: In defense of "New Math" # #
Via: reddit
April 2, 2014
15:42 -0400
Hubert Chathi: The Chronicles of Narnia is now Public Domain in Canada, and available at Project Gutenberg Canada
Via: reddit
April 1, 2014
11:45 -0400
Hubert Chathi: Study finds that when no financial interests are involved programmers choose DECENT languages
March 13, 2014
11:57 -0400
Hubert Chathi: loves #
February 28, 2014
16:53 -0500
Hubert Chathi: now has a functional install
February 5, 2014
19:43 -0500
Hubert Chathi: Today while I was working, # decided to lie down on my lap and fall asleep.
January 31, 2014
17:31 -0500
Hubert Chathi: If I wanted to deal with memory leaks, I'd be programming in C/C++.
December 30, 2013

Random testing

17:55 -0500

My current project at work requires implementing non-trivial data structures and algorithms, and despite my best efforts (unit testing consisting of over 600 assertions), I don't have everything unit tested. In order to find bugs in my code, I've created a randomized tester.

First of all, the code is structured so that all operations are decoupled from the interface, which means that it can be scripted; anything that a user can do from the interface can also be done programmatically. Of course, this is a requirement for any testable code.

I want to make sure that the code is tested in a variety of scenarios, but without having to create the tests manually. So I let the computer generate it (pseudo)randomly. Basically, my test starts with a document (which, for now, is hard-coded). The program then creates a series of random operations to apply to the document: it randomly selects a type of operation, and then randomly generates an operation of that type. It then runs some tests on the resulting document, and checks for errors.

Most of the time, when doing random things, you don't want things to be repeatable; if you write a program to generate coin flips, you don't want the same results every time you run the program. In this case, however, I need to be able to re-run the exact same tests over and over; if the tests find a bug, I need to be able to recreate the conditions leading to the bug, so that I can find the cause. Unfortunately, JavaScript's default random number generator (unlike many other programming languages) is automatically seeded, and provides no way of setting the seed. That isn't a major problem, though — we just need to use an alternate random number generator. In this case, I used an implementation of the Mersenne Twister. Now, I just hard-code the seed, and every time I run the tester, I get the same results. And if I want a different test, I just need to change the seed.

It seems to be working well so far. I've managed to squish some bugs, uncover some logic errors, and, of course, some silly errors too. Of course, the main downside is that I can't be sure that the random tests cover all possible scenarios, but the sheer number of tests that are generated far exceeds what I would be able to reasonably do by hand, and my hand-written tests weren't covering all possible scenarios anyways.

Addendum: I should add that when the randomized tester finds a bug, I try to distill it to a minimal test case and create a new unit test based on the randomized tester result.