We launched Triplebyte last year with the goal of building a hiring process focused on evaluating skills and not credentials. Since then we've evaluated over 10,000 engineers without using their resumes, and helped them join companies ranging from three person startups up to Dropbox. Doing thousands of blind technical interviews has forced us to get really good at identifying programming skills directly, and allowed us to work with engineers from a great diversity of backgrounds.

One of the most surprising things we've learned through this process is just how much companies differ in which programming skills they value most. A deliberate, academic programmer, for example, may do extremely well at one company, which thinks that she will be able to tackle tough problems. Other companies instead want fast, intuitive thinkers, and will reject this same engineer on the grounds that she may not be productive enough. Some companies want all their engineers to understand deeply how a computer works. Candidates have no way to know what specific companies prefer, and this results in a large amount of wasted time. Rejection is also demoralizing and we've seen many engineers, especially those working outside Silicon Valley, start questioning their own abilities after a few failed technical interviews.

To make the process of finding the right company better for engineers, we're announcing the Triplebyte Engineer Genome project. Using the data we've gathered through our technical interviews, we've mapped out the engineering attributes that technology companies care most, and measured how the companies we work with weigh these attributes differently. We've used this data to build software to intelligently match engineers with the companies where they're the best technical fit, and we're using this software with engineers who go through our process.

Intelligent matching with software is how hiring should work. Failed technical interviews are a big loss for both sides. They cost companies their most valuable resource, engineering time. Applicants lose time they could have spent interviewing with another company that would have been a better fit.

Moving skills assessment and company matching into software also has another important consequence. It increases diversity in the hiring pool. If companies can trust that an applicant has the technical skills they're looking for, it gives them confidence to speak with candidates who lack the usual credentials of attending a top school or working at a prestigious company.

We've built up the list of engineering attributes in the Triplebyte Engineer Genome by collecting a large amount of data from the thousands of blind technical interviews we've completed ourselves. We've then tracked how the data we've collected about each engineer matches with their interview performance at top technology companies we work with. This has been a huge schlep but it's the only way to build a matching system that actually works.

By evaluating this many engineers and working with over a hundred companies, we've seen how little consensus there is on what a great engineer means to any single company. We’ve calculated statistically the extent to which interviewers at different companies agree about which candidates are good and which are bad (for the statistics nerds, we calculated the inter-rater reliability), and found it to be about the same as the extent to which people agree on which movies on Netflix are best.

Recruiting services today avoid tacking this problem altogether. Mapping what companies actually want is a much harder problem than scaling the traditional recruiting agency model of spamming companies and candidates. There are so many different engineering attributes you could conceivably look for it's hard to narrow it down to a concise list. Even if you could cleanly identify these attributes, assigning the right weight to each one adds another layer of complexity. It's too much to expect a single person or team making hiring decisions to do this well.

Zooming out, it's too much to expect a single company to be capable of solving this problem either. Google arguably does the best job and their data is still limited to (1) engineers who applied to Google (2) finding which attributes are most important for success at Google. Most companies are bad at identifying what a great engineer looks like. Even the famous ones get it badly wrong, like Facebook rejecting WhatsApp founder Jan Koum (they did eventually hire him but the price went up a bit).

Triplebyte is uniquely positioned to fix this. Over the past year, we've collected both quantitative (e.g. time to complete milestones within programming problems) and qualitative (e.g. problem solving approaches or code quality) data from several thousand technical interviews and have use it to create a list of the engineering skills most important to technology companies - the Triplebyte Engineer Genome. These are:

  1. Applied problem solving
  2. Algorithms knowledge
  3. Professional code
  4. Communication skill
  5. Architecture Skill
  6. Low-level systems understanding
  7. Back-end web understanding

By scoring engineers on these criteria and then assigning weights to each companies based on empirical observations of their hiring decisions, we can use software to better identify engineering skill than humans. We're excited about this because it moves us towards removing human biases from the hiring process altogether. Humans making judgement calls about objectively measurable skills introduces bias and hurts diversity. If we want more diversity in tech, this needs to be done with machines crunching objective data.

We expect the list of attributes in our Engineer Genome to continue evolving over time as we gather more data on what companies are looking for. We'd welcome your thoughts or feedback on the project and thanks to everyone who has completed the Triplebyte technical evaluation.


Share Article

Continue Reading