Testing Software Engineers

After accidentally hiring a dyslectic kangaroo as a software tester, I have decided to improve my interview process for hiring new software engineers. The first thing I did was to learn from the best. I found the Guerrilla Guide for Interviewing from Joel Spolsky, the Five Essential Phone-Screen Questions from Steve Yegge, How to Interview a Programmer from Bill Venners, Programming Interview Questions from Maxim Noy, and The Phobia List from Fredd Culbertson. — I admit that the last one did not help me in any way, but I found it nevertheless. So I thought it was worth mentioning.

There were a couple of things I learned from these pages:

• Ask your candidates some generic but vital software engineering questions, and don't bother with specific technologies. Because technologies come and go, but it takes good software engineering skills to cope with ever-changing technological environments.
• Ask open questions, so that you can determine a person's professional views and talent from the discussion that might follow.
• The experts don't agree whether or not you should let candidates write code for some exercises on-the-fly. Joel Spolsky and Steve Yegge think this is essential. Scott Meyers and Kevlin Henney think it is a waste of time.
• Apeirophobia is "the fear of infinity" and medorthophobia is "the fear of an erect penis". Though I'm still struggling to put this factoid into good use.

Armed with these bits of knowledge (except for the last one) I created a big list of questions. I made liberal use of the Guide to the Software Engineering Body of Knowledge. I copied its classification of knowledge areas, and changed it a bit so that it would better fit my purposes. I then used the SWEBOK guide and the abovementioned articles to come up with 100 questions in total.

Let me give you some random examples:

Requirements
Can you name a number of non-functional (or quality) requirements?
Functional Design
Can you name different ways and techniques of prototyping an application?
Technical Design
How can you manage concurrency of modifications by different people to the same data?
Construction
When would you use a class with static members and when would you use a Singleton class?
Algorithms
How do you find the middle item in a linked list?
Data Structures
How would you store a vector of N dimensions in a datatable?
Testing
Which tools are essential to you for testing the quality of your code?
Maintenance
How can you debug a system in a production environment, while it is being used?
Configuration Management
Are there differences in managing versions and releases?
Project Management
What is the difference between an iteration and an increment?

I am not going to reveal all one hundred questions, as this could possibly lead to me hiring more kangaroos with perfect scores, dyslectic or not. This test will take about an hour when I make some random selection of between fifty and seventy questions. The end result will be a nice diagram showing the scores over the entire range of ten knowledge areas. Yesterday I tried the test on my good friend Niels. His overall scores were quite high. And the nice thing is that I now know I'm never going to let him contribute to any testing activities on my projects.

I realize that people who've seen these ten example questions will enjoy an advantage over other candidates. But, any candidate who can prove being a regular reader of my blog will get ten bonus points anyways. So this will surely not hurt the statistics. All in all, I'm quite happy with the results, as I think the new test is quite comprehensive. The only thing I haven't been able to test yet is cyberphobia and logizomechanophobia: "the fear of computers or working on a computer".

Note: Greetings to Johanna Rothman and Mike Griffiths, who will bringing the total number of readers of this blog to the staggering amount of 19.