If there’s one certainty in the Silicon Valley, it’s that the success of a startup highly correlates to its people. I didn’t realize this when I started my first company, Shazam, in late 1999. Our initial team of twenty was mostly hired from inside our network, and I would generally describe them as top-performers (and many of them have indeed gone out to start their own companies). The next set of hires at Shazam came through traditional recruiting and interviewing. This process, however, did not allow us to separate the A+ players from the A players, and the difference was meaningful. For example, I have firsthand seen how an A+ software engineer will easily do the work of three A-caliber engineers. Only when a person was six months or so in the job, we would be able to tell whether he or she was “the best” or “great” (“good” would not be hired), and the outcome felt like a mere draw of luck.
After stepping down from my operational role at Shazam, I applied for a job at Google in early 2005. The hiring process involved fifteen interviews (I vividly remember getting grilled by Susan Wojcicki, Sukhinder Singh, and Joan Braddi on their meteoric rise) and, above all, a monstrous homework assignment. I was asked to put a pitch deck together for Google Video, without knowing anything about the product or strategy (i.e. Google Video had yet to launch). It took me 30+ hours to piece it together, but it was worth every bit: I got the job, and, ironically, I had my pitch fine-tuned before I even met with a first potential partner. Joining Google was beyond refreshing: every single one of my colleagues was smarter than me. I felt like a 9-min/mile marathon runner who had just joined an 8-min/mile team, and I will admit to severe moments of self-doubt in the first three months. Google hires incredibly well, and this has been well-published over the last decade (e.g. everyone in the hiring team must be supportive, resumes get screened at the highest level, etc.). To me, the true differentiator in Google’s hiring process lay in the homework assignment, and the presentation of it to the hiring team. It is hands down the best way to separate the best from the great. Github’s journey into recruiting is therefore not surprising: the ability to read someone’s code is an indirect way to score his or her (home)work.
Here’s what makes a homework assignment superior to other interviewing techniques (including a few recent Tatari examples):
It requires the candidate to answer in detail (or, a “fluffy” vague response won’t work). For example, for engineering hires, you can see the elegance in the code (or absence of it, for that matter).
Homework is a strong test of interest. For example, recently a friend felt that she was above a Tatari homework assignment; I immediately knew she lacked motivation, and we did not proceed. In contrast, when Brad joined our team, he did three homework assignments: I knew he wanted to join us.
It gauges the candidate’s grit. Hard-workers will not flinch at the thought of preparing a homework assignment for two hours. An A+ player will surprise you with “Yeah, I worked on it for 20 hours”. For example, Denis’ homework assignment was to write a press article for Tatari. He delivered it in the style of The Economist, with the fonts, design, magazine cover, and all. Wow!
The presentation of the assignment is a great way to
assess the candidate’s communication skills (and for those in sales, presentation prowess). For example, Param was asked to do a campaign review of a fictitious client: her presentation and PowerPoint delivery was better than anything I had ever done.
Homework assignments also reveal personality and attitude, especially during the Q&A time. We often push a person to his or her limits by exhausting the question (e.g. each answer is followed by “Why?”). Do they keep their cool, or (as we once had), do they get frustrated and angry?
Ever since I left Google, I have religiously instilled homework in the interviewing process. First at TrueCar (I joined as the third employee, and everyone since then has enjoyed the homework), then at Rockmelt (granted, the team was mostly already in place), and now at Tatari. And it works, it’s that simple! Tatari has grown from zero to more than twenty clients in just one year, has been profitable throughout, and is laying the foundation for how TV will be measured and purchased for the next decades. Our new hires may lack in TV advertising experience, but they make largely up with raw intelligence and an insatiable appetite for learning. Best of all, they increasingly take over parts of my job, and do it with an element of surprise and execution beyond what I could have ever pulled off myself.
That said, I will admit that there is a problem with homework assignments: they don’t scale well with hiring. At Tatari, for example, we need to fill twenty roles at the company this year. That’s a lot of homework. To that extent, we recently started using a platform called interviewing.io for engineering hires. Candidates on interviewing.io are not screened based on resumes, but through practice, assignments, and interviews, all of it anonymous. It is a scalable, objective and reliable approach to identifying the top performers. I first got to know Aline, founder and CEO of interviewing.io, when I was trying to hire the best recruiter in the Silicon Valley. Since then, she has truly productized my preferred hiring process and the use of homework assignment. She runs a fantastic blog and if you want to join the cream of the cream at Tatari, you can always apply directly with us or find Tatari after completing interviewing.io (and get some interviewing practice in at the same time).