Back in May of 2020 I opened a job posting for the first of three Software Test Engineer positions. I collected data on my hiring experience and published the results in 2021 in an article called Hiring a Software Tester, an Analysis.

You can read the article or review the infographic Ranorex created:

It was March of 2021 when I published the first article. I had hired again. In the conclusion of that article I promised to write more about my process. Today I'll one-up that promise by analyzing the next two Software Test Engineer hiring rounds from 2021. This time I have a theory to test: testing education is a positive signal in my hiring process.

About the data

Some of this data came through our recruiting system and some I entered. Some of the data wasn’t captured at all and I made guesses such as for location and gender. I don’t hire based on gender, but I was curious to see how this might have changed throughout. All the data was from 2021.

Median vs Averages

When digging through the data I wanted to understand both the median and average values. Why? Median values deal with the extremes in a better way than averages do. Both can useful.

• Median is the halfway point: half the numbers are higher and half are lower.
• Averages add everything up and divides by the total number.

Findings

1. Many Experienced applicants

This data came across two different hiring cycles in 2021. Despite having 190 submissions, I ended up pulling data on 176 resumes. There were a lot of experienced applicants:

Only 18% of applicants were at or below 3 years of experience (the minimum in the job description). The first time 43% of applicants were below the minimum because I forgot to include it in the job description. Turns out applicants do read job requirements!

The two green lines in the image above show the average and median experience for all applicants. The median experience across both positions was 6 years while the average came in at almost 8 years.

When it came to interviewing the median experience was 10 years and the average was 10.5 years. I hired 2 people: one with 11 years experience and another with 22 years experience!

2. Location mattered less

In 2021 our company came to realize remote work wasn’t going away. I had the green light to hire outside of the Los Angeles (LA) area. I made sure to emphasis this was a remote position but that we tended to favor meeting times in PST.

As a result, I saw an increase in the number of applicants outside LA. When it came to interviewing 55% of applicants were not from LA. When it came to the people I hired they all came from the west coast: one from Los Angeles and one from Portland.

3. No more long resumes

Your resume should tell a story about who you are and what you’ve done within the context of the job you are applying for. It’s not always easy to get this narrative in place but when it works, it works well.

60% of applicants had resumes 2 pages or less. 80% had 3 pages or less. The first time, 75% of applicants had resumes 2 pages or less. The less pages, the easier it is for the hiring manager to understand your story and move you along in the process.

I didn't interview anyone with a resume over 6 pages. The median resume was 2 pages and the average was 2.7 pages. When it came to the people I hired: one had 2 pages and another had 3 pages.

4. More women applicants

Tech demographics are heavily skewed towards men and software testing is no exception. While I don’t hire based on gender, I am curious about how many women apply and how (if at all) it changes over the hiring process.

Almost 32% of applicants were women while men made up 40%.

When it came to interviewing almost 43% of the applicants were women while 50% were men. Compared to the previous hiring cycle data, these numbers are a small improvement.

Both applicants I hired were men.

5. Testing Education Mattered

There were no education requirements in the job descriptions. I'm more interested in the person being a capable tester then having gone to a specific school. Instead of analyzing the data for education in general, I wanted to look at my theory: testing education is a positive signal in my hiring process.

If this theory is true, there should be a greater percentage of applicants interviewed with a testing education compared to the overall population. Let's take a look at the population of applicants:

73% of applicants had no listed testing education on their resumes. In the previous hiring round, it was 84%. (This was likely skewed due to the number inexperienced applicants.)

When it came to interviewing, 50% of applicants had some form of testing education. Of those I hired, 100% had some form of testing education on their resume. One had a testing certification and another was taking courses on Udemy.

Theory proved! Turns out testing education IS a positive signal in my hiring process!

A gold mine of data

Each time I've done this I've learned something about my process and about the applicants. But it's a lot of work to go through the data, analyze it, understand it and publish something useful. There's a lot of data I collected or could have collected but don't have time to do anything with. It's like a gold mine of data waiting to be mined!

Having data to back-up experiences and test assumptions helps us make better decisions in the future. As I said in Effective Technical Investigators, testers are investigators and researchers. We gather data and test assumptions.

At the beginning of this article I made a claim, based on a theory. As we saw by going through the data, this claim turned out to be true: testing education was a positive signal in my hiring process!