NRG Global Test Competition Retrospective
Roughly two and a half weeks ago I competed in the first NRG Global Test Competition. The idea behind the competition was simple: get a bunch of people/ teams together to test a few products, split the competition into two days, one with functional testing and another with performance testing, and based on the reports submitted judges would award points and announce winners. The full details are available here and here.This was the first online testing competition I’d tried but thanks to my experiences with testing challenges and rapid testing online I knew I’d have fun once I got past the quirks. By quirks I mean it can take time to get comfortable with the discussion format, figure out how to ask questions, how best to communicate with my fellow team members, etc. The competition took place at 10 am Eastern which sucks if you live on the west coast and have to wake up before 7 am like I did. It was all for fun anyways.
For as early in the morning and as new as the competition was I think I did reasonable. Not great, not even good, but reasonable. I think the best way to phrase it is: I’m not happy with my work. (I might be overly critical here but still.) Now we only had 3 hours from introduction of the products under test, to learn the product, ask questions of the “owner”, test it, ask more questions, file bugs and write a report. Yet when I think back at what we turned in I’m not happy with it. Let me explain.
My team member and I barely communicated with one another. We were using Skype but we didn’t do much planning ahead of time (not like we could have because nothing was public) so when it came time for the competition it was a simple “hi”, “what are you working on” and “I’ll look at x”. That was it. We each went to different applications. Thinking back on it now I think we would have done much better if we were on the same application, talking to one other about what we were seeing. My experience has been any collaboration no matter how small results in finding and learning amazing things.
At the time of competition I considered using Bach’s HTSM to map out the application but didn’t. I wish I had. Even though its a bit detailed and were we on a short deadline I think heuristics would have lead me to think about and discover even more potential problems. At the very least I’d feel more confident in what I had tested.
It took me an hour or so to really get started looking at the products, deciding which one to test and with what equipment (iPad), browsers, etc. I’m still blaming the time of day. I started with some simple touring of this “home built” application that must have been built specifically for the competition because it was really simple and full of problems. Even though I saw lots of problems initially I took note and kept searching until I felt I had covered the entire application as best as I could. Then I circled back, asked a few questions from the “product owner” Matt Heusser and began testing the problems I saw. By that time I had just enough time to get my bugs written up in the tracking system (I had maybe 5) and started working on our Test Report. I think we got our report in right at the deadline.
I wasn’t able to commit to the second part of the competition, for performance testing and I’m not sure if my team member was able to either. I knew going in I couldn’t commit time for it however I’m still holding on to hope that I’ll get a chance to play with the AppLoader tool. Despite my displeasure with my performance I’m glad I joined, in fact I’m looking forward to getting the feedback on how other’s think we did. =)