Recruiting in China

Care to dramatically enhance your chance of finding great employees? Trade in your gut instincts for a systematic approach to interviewing, testing, and evaluating job candidates.

What was her company missing? Susan Bowman asked herself that as soon as she plopped into her chair at Tri-anim, a medical-supplies distributor in Sylmar, California. It was two and a half years ago. Bowman had just joined the company as head of human resources, and her highest priority was improving the company's hiring. When she arrived, the HR department was basically shut out of the hiring of salespeople. Bowman wanted to make it more useful, especially after she noticed some hires were fantastic and others were disappointments.

What Tri-anim was missing--and Bowman fortunately recognized this--was something most employers in America have been missing: Conventional job interviews don't work.

A typical interview--unstructured, rambling, unfocused--tells the interviewer almost nothing about job candidates, other than how they seem during a couple of meetings in a conference room. But what are these people like late at night and under pressure? What motivates them? How smart are they? Have they handled tough projects? Do they prefer working alone or are they better with a team? Regular interviews assess barely any of this, and in fact are miserable predictors of job success. In technical terms, they have a .2 correlation with predicting success.

Discouraging, isn't it? It would be--except that industrial and organizational psychologists are on the job, seeking the best ways to evaluate job candidates. A focused three-part approach can make the hiring process as standardized and objective as possible--and can help predict the best performers. The system starts with what is called behavioral interviewing, in which candidates are barraged with tough questions about how they've handled specific assignments and problems. Bluffing becomes close to impossible, and the process is based on facts, not feelings. Interviewing is followed by two kinds of tests: cognitive tests, which measure intellectual ability, and personality tests, which are now sophisticated enough that companies can directly compare candidates with their top performers. The third step is asking candidates to do tasks like the ones they'd do on the job.

Most employers will recite over and over that people are the secret to their success--and given that turnover costs about 1.5 times the salary of the employee who moves on, according to PricewaterhouseCoopers, they'd better mean it. But it's astounding how few companies bother with more than improvised, all-but-meaningless interviews to hire their people. "This is a topic that's been researched to death by the field of industrial and organizational psychology," says Peter Cappelli, management professor and director of the center for human resources at the Wharton School of the University of Pennsylvania. "The amazing thing is how few companies take this seriously. It's kind of mind-boggling that they would undertake such huge investments and not pay attention to what we know about how to pick out the people who are going to be best."

Susan Bowman had been studying some of this research. She was pleased to see that Tri-anim had been using the testing company PSI to assess candidates for some positions. She was less pleased that the test criteria hadn't been updated in six years and that some of the company's hiring managers didn't use the tests. Bowman immediately had PSI reassess the best and worst performers in a number of areas and develop profiles of the top performers. The goal is to compare candidates with the ideal. Tri-anim salespeople, for example, need to be not just energetic and detail-oriented (pretty common in salespeople) but also unusually independent: They spend a lot of time alone.

Bowman began requiring the PSI assessments as a last step in the managerial, IT, and sales hiring processes. They've already turned up surprising results. Recently, a recruiter and a manager were disagreeing over two candidates for a position--until the PSI reports came back. "The results were really staggeringly different. It was a combination of not only skill sets, but that one individual's people skills were so much lower than the manager had anticipated and the other candidate scored much higher," Bowman says.

She has now trained all of Tri-anim's hiring managers in behavioral interviews. "Structured interviews with behaviorally based questions really allow us to drill down," she says. In a daylong session, the managers learned the tenets of behavioral interviews and practiced asking open-ended questions. Though she doesn't use work assessments--and that could increase the company's hiring success even further--these two steps paint rich, objective portraits of candidates. Even the sales hiring managers, who didn't want to abandon their random interviewing tactics, have become believers as turnover has dropped. "We all want to hire the best," Bowman says. "This gives really good, objective information that allows the manager to take the halo off the applicant."

Step 1

In which the bored interviewer turns intrepid interrogator
Other than people's wan complexions beneath fluorescent office lights, there's not much that's consistent in typical job interviews. Topics discussed completely depend on the interviewer, who might spend an hour discussing a candidate's alma mater, the recent weather, or even himself. He could dismiss the candidate before she's even started speaking because she's overweight or overdressed, or he could lose focus because he's having a rotten day. Afterward, the interviewer is left with a resume and a vague sense of...how the candidate acts during an interview. Is she qualified? Dunno, but her resume looks nice. Would she be good at the job? Well, she likes to sail, which is fun.

.2 Correlation between conventional interviewing and successful hiringAs psychologists have pointed out, traditional interviews produce a subjective, acutely narrow view of a job candidate. That view is likely biased--studies have shown interviewers tend to prefer candidates similar to them, judge candidates on fewer criteria than they think they're judging them on, and tend to let biases about matters like race and gender get in the way. "Everybody thinks they're much better interviewers than they are," says Ben Dattner, a New York City industrial and organizational psychologist.

Still, the interview is a brilliant tool if you make certain changes to it. Behavioral interviews have almost triple the correlation of conventional interviews with job success. (To gauge if a hire is successful, academics use measures like the dollar value of an employee's contribution to the company, his or her relative share in overall output, and later performance reviews, promotions, and raises.) Behavioral interviewing involves, by definition, a group of interviewers defining qualities needed for a job, asking candidates to give past examples of how they've demonstrated those qualities, asking the same questions of each candidate, and taking notes throughout. The premise is that what someone has done in past jobs is a superior indicator of what he or she will do in future jobs. It's the same idea behind checking references.

To see how structured interviews work, take a look at Hope Lumber & Supply, where HR chief Bill Vogt credits much of his company's growth to behavioral interviewing. Hope, which is based in Tulsa, brings in $1.2 billion a year selling building supplies to contractors. Eight years ago, when the company was making a fifth of that, Vogt and the owners predicted, correctly, that the housing market was about to surge. If they hired the right managers, they could ride that wave.

Following behavioral-interviewing maxims, Vogt starts by talking to people intimate with the job and deciding what qualities are necessary for it. He has a standard template for what he wants in managers: leadership, a drive to make money for the company and for themselves, ambition, and past operational responsibility. Depending on the challenges of the specific business unit, he'll alter the template.

Then he comes up with open-ended questions that get at the desired qualities. Behavioral interviews use questions that are rooted in the past--"Tell me about a time when"--rather than hypotheticals--"What would you do if?" Vogt digs deep into his candidates' work experience. "I get into the current operation," he says. "What did you inherit? What were the sales margins, accounts payable, percent current status, inventory like? What did you do with that, what did you achieve? Clearly, we're looking for achievers and winners and people very knowledgeable of their operation." Specific questions like these, in addition to assessing candidates' skills, combat resume fraud--it's pretty difficult to lie about sales margins and inventory turns.

Ideally, a team of people will meet with the candidate. That minimizes the importance of any one person's reaction, good or bad. Vogt arranges a panel interview for general questions, and then sets up one-on-one interviews focused on specific areas. Vogt asks about EEOC compliance and OSHA incidents; the CFO asks about accounting details; the COO asks logistics questions. In any behavioral interview, questions should be job-related, to keep the interview relevant and to avoid discrimination complaints. To the extent possible, every candidate should be asked the same questions. Interviewers should take notes, and should get together to discuss their views just after the candidate leaves.

Step 2

In which the candidate relives college-entrance tests
As helpful as behavioral interviews are, they're even more effective when combined with employment tests, many of which are now administered online. These are given to candidates to assess either cognitive abilities (cognitive tests are filled with SAT-like verbal and math questions) or personality traits (personality tests include preferential questions like "Would you rather spend a night at home alone than go to a crowded party?" or biographical questions like "Were you a class officer in high school?"). While cognitive tests have a slightly closer correlation with job success, personality tests are useful both as a basis for interview questions and for subsequent development. For the best results, companies should use both sorts of tests or a single test that combines the two elements. (For a roster of tests, see "Choose Your Weapon".)

Many testing companies today can do impressive comparisons of candidates against existing employees--the goal being to essentially clone top performers. "The assessments allow you to really identify what is different between our stars and our slugs," says James Hazen, an organizational psychologist and the owner of Applied Behavioral Insights, a consulting firm based in Wexford, Pennsylvania. Hazen uses several tests with his clients.

2,500 Number of cognitive and personality tests on the marketAssessments can turn up some fascinating findings. Dayton Freight Lines, a trucking company based in Dayton, Ohio, had been having trouble with drivers. Customers reported that some drivers were rude. Some drivers were complaining over their CB radios. Some workers' productivity was falling, or they were late on their deliveries. Denise Noel, the director of quality at Dayton Freight, was stumped. These drivers all had good qualifications and had interviewed well, yet she saw no way to predict who would be an outstanding performer on the road. Finally she brought in a company called Hogan Assessment Systems and had the company present its extensive research on truck drivers.

Noel had assumed all truck drivers were similar. But Hogan had found two distinct truck-driver profiles. The top city performers are social and gregarious, great with customers--which makes sense, because they pick up and drop off multiple times a day. The best line-haul drivers are quiet and introspective--which is good for people who never see a customer. Noel has adjusted her hiring now, having candidates take the Hogan assessment to find the best job for them. Turnover for drivers has fallen to 22 percent (the industry average is 116 percent). "You just think a driver is a driver, and that's not true," Noel says. "We just didn't look at that part of the hiring process enough."

Discussing the results of assessment tests with candidates--or even giving them the full report--is increasingly popular. "The trend has really been to lay it all on the table between the second and third interviews," says James Hazen. This gives candidates the chance to explain themselves, gives the interviewer a chance to address weak spots, and, if someone is hired, points out ways he or she might best be managed.

There are, by some estimates, 2,500 employment tests on the market. One of the biggest mistakes companies make is using the wrong test. A classic example is the Myers-Briggs Type Indicator, that ubiquitous test that sorts people into 16 personality categories. Myers-Briggs, a test created by a Pennsylvania woman who was fascinated by how her merry personality differed from that of her straightforward husband, has a weak record of predicting job success. Indeed, its publisher warns that "It is unethical and in many cases illegal to require job applicants to take the Indicator if the results will be used to screen out applicants."

With so many tests available, it's not a surprise that employers use tests meant for other purposes, like Myers-Briggs (which is fine, by the way, for employee development), or even design their own tests. But choosing the wrong one can mean dismissing qualified candidates and even getting sued for discrimination. Employers need to know whether a test is appropriate for hiring, what it measures, and how it's designed, along with making sure it's legal. Psychologists evaluate a psychological test by two measures, called reliability and validity. Reliability examines whether items that supposedly measure the same thing (agreeableness, say, or conscientiousness) correlate highly with one another. Validity asks, in this case, for proof that scores on tests are related to success in specific jobs. "If you go out on the Net and look at the hundreds of tests out there, a very small percentage have validity data," says Seymour Adler, a senior vice president at Aon Consulting and a teacher of organizational psychology at New York University.

Recent psychological research supports going beyond validity and reliability data. First, both for legal purposes and to ensure usefulness, make certain the test is designed for selecting--as distinct from developing or training--employees. It should be created or adapted for the workplace, not for clinical or medical diagnosis. Pre-employment tests are more predictive when they compare an individual's score against a group (they use "normative" scales, in the lexicon) instead of just presenting it on its own ("ipsative" scales). For the best results, too, employers should continue to evaluate and revalidate the tests within their companies to make sure they are still predicting top performers.

A note about testing for hourly employees. There, employers might care most about who's punctual and honest. Rock Bottom Restaurants, a 29-store chain based in Louisville, Colorado, switched three years ago from a pencil-and-paper application for its hourly employees to a test from Unicru. (Kenexa and PreVisor are two other assessment companies focusing on entry-level and hourly applicants.) For waiters, it tests for sociability and team orientation; for the back of the house, it asks applicants whether they've worked in on-their-feet jobs before; for all job candidates, it looks at integrity. Applicants in each pool--cooks, bartenders, and so on--are ranked according to their assessment scores, which gives the Rock Bottom management a good starting point. "It's not 100 percent predictive, and that's why we interview people, but it's at least an indicator," says Ted Williams, senior vice president of the brewery division at Rock Bottom. Rock Bottom's turnover for its 6,000 hourly employees has dropped by 20 percent, which Williams thinks is largely because of the system.

Step 3

In which the process starts to imitate finding World War II spies
In 1943, a pretty countryside residence in Fairfax, Virginia, was renamed Station S and repurposed as a testing site for Office of Strategic Services recruits. In an atmosphere of intense secrecy--candidates were stripped of their clothes and given military fatigues, then driven in a windowless van to Fairfax, where they would invent a cover story and fake name--the OSS studied their performance during job simulations. One test had "couriers" giving candidates a map, which they'd need to memorize in eight minutes. Other exercises included interrogating ersatz prisoners of war, devising propaganda plans, and recovering papers from an agent's room (and, aggravatingly, getting interrupted by a rifle-wielding "German" midway). The tests went on for three and a half days.

Inspired by that work-based approach, corporations such as AT&T starting using assessment centers to select executives. By the late 1950s, the candidate in the gray flannel suit was performing in-basket assessments in which he'd be graded on how he handled a set of letters, papers, tasks, and telephone calls that mimicked what he'd get on the job.

Today's work samples are essentially updates of those AT&T tests. Work samples are a proven predictor of success and can be simple to arrange. A company can design its own by laying out the criteria for a job and asking a candidate to perform a task based on those criteria. For example: "Explain how you would sell this product to Target, step by step," or "Tell me how you'd improve these lines of C++ code."

4 Number of weeks capital H Group dedicates to hiring a single consultantAt Sterling Communications, a technology PR firm in Los Gatos, California, CEO Marianne O'Connor knows her account reps have to be good at understanding technical information, at figuring out how to pitch to a media outlet, and at writing. Logical enough. So she's started giving job candidates a two-hour test before she even meets with them. It describes a client's technology, identifies a target publication and its readership, and asks a candidate to distill the salient technical points and write a pitch to the magazine. Three staffers review the pitch, and that decides whether the candidate will get an interview. "If they can't write in my business, it's not going to work," O'Connor says.

On the complicated end of the work-sample spectrum, Seymour Adler, the Aon Consulting psychologist, has created a four-hour online exercise called Leader, which Motorola and other companies use to test would-be executives. Candidates see an in box with e-mails that came in the night before, answer phone calls and listen to voice mails, and have access to reports and research. They're asked to tackle tasks like ones they would see on the job, such as solving a conflict between two underlings or leading a team of workers in creating a presentation for the CEO. At the end, Adler's team assesses the candidates on whatever areas the company is curious about--decisiveness, leadership, and so forth--and issues a report to the company. A company called Development Dimensions International offers similar exercises; these take place at one of its 75 assessment centers rather than online. Half-day and full-day job simulations cost from $4,000 to $12,000.

And finally...

Put it all together-- without riling your candidates
Dan Weinfurter runs Capital H Group, a human resources consulting firm in Chicago, though he's not an HR guy but an entrepreneur at heart. He founded the accounting and consulting firm Parson Group, which hit No. 1 on the Inc. 500 in 2000 with a four-year growth rate of 27,992 percent, and sold it four years ago for $55 million. Before that, he was second in command at Alternative Resources, an IT staffing company that was a two-time Inc. 500 honoree. For all he knew about running a company, however, Weinfurter came to the conclusion that he didn't know much about hiring. "I thought I was pretty good at interviewing," he says, "but I was no better, and maybe was worse, than other people. If you're just going through it and trying to guess, you'll guess right some of the time. But you won't be able to guess right often enough to grow a business from scratch."

So at Capital H, he unleashed his on-staff psychologists, who created a hiring system that's a textbook example of the latest hiring research. Let's say Capital H has an opening for a consultant. A group of candidates are interviewed by telephone by the HR manager (or by Weinfurter himself, if the position is very senior), and candidates with appropriate skills and backgrounds are then passed to a local office to meet with local executives. He or she takes the Watson-Glaser Critical Thinking Appraisal, a popular and well-validated cognitive-ability test, and the Devine Inventory, which measures the applicant's traits and tendencies against those of existing Capital H consultants. (See "Let's Turn the Tables" for a sample of questions from Watson-Glaser.) About one in four candidates are then flown to Chicago headquarters, where they spend a full day in behavioral interviews with multiple executives. Finally, applicants are asked to choose a presentation they've done in the past and give that to a group of Capital H execs back at the local office in a work-sample exercise. The executives discuss the candidates until they reach consensus.

Weinfurter figures he spends up to four weeks, and tons of his workers' billable hours, per interview. But he estimates the cost of hiring a bad consultant can be in the millions, considering not just salary but also missed sales and lost clients. "I think the hiring process is the most important process in business, but it's probably the least disciplined in terms of how it's executed across American business," he says.

People who study hiring, and business owners who are passionate about the subject, love to see systems like Capital H's. Candidates may not feel the same way. Certainly you'll have to make concessions in some cases--say you're trying to recruit a CFO from a rival company. "If they've already done a job like this, what's the point of the test? It's not obvious you want to give this to everyone and for every job," Peter Cappelli at Wharton notes. In every case, candidates will have a better attitude toward the process, and the company, if they believe that the hiring methods are respectful, fair, and smart. So use appropriate cognitive tests--don't ask accountants basic math questions. Use only tests designed for the workplace, so that the questions clearly deal with business situations and seem relevant. And explain why you're adopting an approach that to some candidates will seem overwrought: to be fair and quantitative.

There will always be skeptics about this approach to hiring, people who believe their gut tells them more than any structured interview or test could. And while Bill Vogt or Denise Noel or Dan Weinfurter could offer testimonials about the new science of hiring, the point is not that this system has worked in a handful of cases. It's that hundreds of studies have confirmed that testing and structured interviews do a much better job at finding good workers than do regular interviews. Given that, the gut-feel proponents start to seem like people who eschew antibiotics in favor of good old-fashioned bloodletting.

Maybe people don't like to believe that something as crucial to a business as hiring can be reduced to a series of processes. After all, we rely on feeling and judgment to get through our lives, whether to fall in love, keep safe on dark streets, or assess business partners. This science-based approach isn't perfect. It won't anoint every superstar, and it won't bar the door to all of the mediocre players. What it will do is give employers a fuller, more balanced, and fairer view of candidates, and give them a much better shot at hiring the best people. It's still up to employers to make the call on whether to hire or to pass, and that's where feeling and judgment still play a part. But that part now comes after employers have gathered all of the facts.

Stephanie Clifford is a staff writer.