Author Topic: I'm seeing personality tests and cognitive tests in tech now  (Read 6014 times)

mizzourah2006

  • Handlebar Stache
  • *****
  • Posts: 1063
  • Location: NWA
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #50 on: November 01, 2019, 01:04:53 PM »
This whole discussion reminds me of something one of my psych profs said in 3rd year, when everyone was stressing out about getting into grad school.  Apparently, someone made an error one year and offered admission to the master's program to the wrong person, who then turned out to be one of the best students they'd ever had.  Just goes to show .....

oh for sure. There's clearly measurement error. That's why the best use case is in large volume roles. The probability of an assessment identifying the best person out of 5 is pretty slim, even with a correlation coefficient of 0.5. A correlation of 0.5 really only demonstrates that 25% of the variance in job performance can be explained by the variation in the predictor.

The way I've always explained it to a laymen is like this; You have two bowls of red and green marbles. If you select a green marble you have a good hire, if you select a red marble you have a bad hire. Would you rather pick from a bowl that was 50/50 or one that was 70/30 in favor of green marbles?

There's certainly the assumption that they do, but I'm not entirely convinced that personality and IQ tests give you a 70/30 bowl.

Well depending on the correlation between the overall battery and job performance that's fairly simple to mathematically derive. 70/30 was an example, it could be 63/37 or even 60/40, but either way, if you were hiring 50k people/yr I bet you'd be willing to pay a decent chunk of change for the 60/40 bowl as compared to a 50/50 bowl. This of course only focuses on the probability of hiring success and ignores the efficiencies gained by basically making the candidate sort themselves into green or red instead of a person manually doing it based off of their own heuristics.

GuitarStv

  • Senior Mustachian
  • ********
  • Posts: 23128
  • Age: 42
  • Location: Toronto, Ontario, Canada
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #51 on: November 01, 2019, 01:15:44 PM »
This whole discussion reminds me of something one of my psych profs said in 3rd year, when everyone was stressing out about getting into grad school.  Apparently, someone made an error one year and offered admission to the master's program to the wrong person, who then turned out to be one of the best students they'd ever had.  Just goes to show .....

oh for sure. There's clearly measurement error. That's why the best use case is in large volume roles. The probability of an assessment identifying the best person out of 5 is pretty slim, even with a correlation coefficient of 0.5. A correlation of 0.5 really only demonstrates that 25% of the variance in job performance can be explained by the variation in the predictor.

The way I've always explained it to a laymen is like this; You have two bowls of red and green marbles. If you select a green marble you have a good hire, if you select a red marble you have a bad hire. Would you rather pick from a bowl that was 50/50 or one that was 70/30 in favor of green marbles?

There's certainly the assumption that they do, but I'm not entirely convinced that personality and IQ tests give you a 70/30 bowl.

Well depending on the correlation between the overall battery and job performance that's fairly simple to mathematically derive. 70/30 was an example, it could be 63/37 or even 60/40, but either way, if you were hiring 50k people/yr I bet you'd be willing to pay a decent chunk of change for the 60/40 bowl as compared to a 50/50 bowl. This of course only focuses on the probability of hiring success and ignores the efficiencies gained by basically making the candidate sort themselves into green or red instead of a person manually doing it based off of their own heuristics.

Not the numbers I'm quibbling with.  I don't really believe that a personality test is necessarily going to increase the number of good hires that you get at all.

In that case, you're choosing between two bowls of 50:50.  One bowl costs a little bit more in time and effort on the part of the person doing hiring.

Psychstache

  • Handlebar Stache
  • *****
  • Posts: 1594
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #52 on: November 01, 2019, 01:36:48 PM »
See? So much work involved in arbitrarily choosing from the 20 out of 100 applicants who could do the job. Keeps people busy for entire careers! "Fruitful, productive work," as Sir Humphrey would say.

So your answer to the problem hiring managers have is to just....pick out of a hat. I’m a Data Scientist who just interviewed with a professional baseball team for a senior data scientist role. I’ll go ahead and tell them that’s how they should handle the draft....well if they played college ball they can all “do the job”...so just pick one. No need to use data and projections. They can all “do the job”.

As an aside...I hope you’re at least pulling 2x BW in that pic or you probably shouldn’t own a strength training gym. But...I guess anyone that can perform the main powerlifts is capable of training people...isn’t that right?

Maybe instead of focussing on the HR lackeys who create a career out of it, focus on the feel-good stories of the employees who are good at taking such tests, who use that fluid intelligence to get scholarships, promotions and eventually get paid handsomely and end up achieving FIRE like all of the rest of us on these forums.

If that's not a feel-good story I don't know what is.

I laugh when people consider me an "HR lackey"....I guess I've fooled all these companies...because I'm an in-demand data scientist that commands $200k+/yr and I work from home in a low cost of living area.  Sure sucks to me :)

Interesting you bring up professional sports leagues, as they are a unique case for a couple of reasons:

1. There are clear metrics for success (wins, championships)
2. There is a massive screening effect going on for years before candidates are even considered for pro level competition (youth sports, high school, minor leagues/college athletics)
3. The impact that each individual player can have is highly detailed and scrutinized in statistical breakdowns of what their performance is on stats that have direct ties to increasing the number of wins, which improves the chance to win championships.

Basically, professional sports are are the most controlled environments outside of actual laboratory studies. Even with all of that, we still see a tremendous amount of failure both systemically (only about 30% or so of 1st round picks in the NFL draft go on to have meaningfully impactful careers) and in individual cases (Tom Brady being drafted in the 6th round).

So in the best case scenario where there is near infinite money to spend to try and get this right using what we think are the most valid measures for measuring what we know are the outcomes we want, we still do a pretty mediocre job. Why then do you think a middle of the road software company with limited resources trying to accomplish much fuzzier goals (most white collar jobs do not have anything close the level of specificity of outcomes that individual pro athletes do) is going to do a better job?

Kyle Schuant

  • Handlebar Stache
  • *****
  • Posts: 1314
  • Location: Melbourne, Australia
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #53 on: November 01, 2019, 04:48:01 PM »

So your answer to the problem hiring managers have is to just....pick out of a hat.

Going on the actual results achieved, as the guy talking about sports noted: yes.


Remember too that most people in their workplaces are not required to perform at the level of professional sportspeople. The stakes are not as high. Because the stakes are not as high, the assessment of their professional skills need not be as strict. If you're hiring someone for $5 million, okay I get it that you'd fuss over details, even if it doesn't work well, as that guy noted; but if you're hiring someone for $50,000, don't stuff them around. That's just busywork.
Quote
As an aside...I hope you’re at least pulling 2x BW in that pic or you probably shouldn’t own a strength training gym. But...I guess anyone that can perform the main powerlifts is capable of training people...isn’t that right?
I welcome this aside as a chance to talk about my wonderfully fulfilling job, and shoot you down further, thankyou.

The picture was 180kg, at the time my bodyweight was 85kg. So yes. My best is 222.5kg for a double at a couple of kg bodyweight higher. That's not the awesome part, that's mediocre.

Performance of a task and teaching a task are different skills. This is why the best athletes rarely go on to become great coaches. Everyone who's been to university, especially in the sciences, has had the experience of someone who's brilliant in their field but who can't teach even first years well.
Typically, good teachers of a skill are those who struggled to be mediocre; they figured out ways to improve despite obstacles, and the people who want a teacher are those who want to improve despite obstacles.
Certainly, the teacher must have gone through the process themselves, but they don't have to be particularly accomplished at that thing at the end of it.

You don't assess my ability to teach the lifts by looking at my lifts, but by looking at my clients' lifts. Nobody pays me to lift, they pay me to teach them to lift. That you don't understand this distinction makes me doubt your ability to assess someone's potential workplace performance.

In any case I don't train people for powerlifting competitions, but for health and everyday life. I took a guy to competition and he pulled 250kg, but he is not my typical client, who is a timid and broken beginner. A 25yo with a herniated disc or a 70yo with arthritis are not ever going to pull 250, nor do they need to; they need to be able to perform the movements pain-free. They assess my ability to get them moving moderate weights painfree not by looking at my lifts, or looking at what I had someone do in competition, but by looking at the people I trained to do the movements painfree.

You are correct that it does not take a high level of skill to teach the lifts. I can teach an idiot to do each lift in about 10 minutes, and teach them to teach others in 6 months or so of thrice-weekly sessions - after that, they'll be as good as any globogym trainer. That's 100-200 hours of experience.

The difficulty in my job comes with the many and various injuries and health conditions people present with. A guy presents with his leg not working because of lymphoedema, how do you get him moving again? Someone else presents with a previous herniated L3/4, they've had to leave their job because of it, and want to do that work again, how do you get them there? Someone else presents with bradycardia, should I train him or refer? there's a fifty-something woman with osteoarthritis and Van Willebrand's Disease, what are appropriate regressions for her, and how will the two conditions interact to affect her progress? The answers don't exist in any textbook, simply because few people lift, few people have these various conditions, and the overlap is small - the answers must be discovered by experience.

And the ability to provide this service must be assessed by potential customers by looking at the results of previous and current customers. The ability to perform a workplace task is assessed by looking at the ability to perform a workplace task. Not by looking at general capabilities, or by looking at other tasks.
My job is very fulfilling. With weights, you can change lives. And in talking to you, as well as being reminded why I prefer training women (men know everything already), I am reminded of when I was a child and had model WWII aircraft, I'd fly them around and go "brrrt, brrrt" and have them shoot each-other down, it was lots of fun. Thankyou. Please keep posting!

Brrrt!
« Last Edit: November 01, 2019, 04:51:08 PM by Kyle Schuant »

mizzourah2006

  • Handlebar Stache
  • *****
  • Posts: 1063
  • Location: NWA
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #54 on: November 01, 2019, 05:02:21 PM »
This whole discussion reminds me of something one of my psych profs said in 3rd year, when everyone was stressing out about getting into grad school.  Apparently, someone made an error one year and offered admission to the master's program to the wrong person, who then turned out to be one of the best students they'd ever had.  Just goes to show .....

oh for sure. There's clearly measurement error. That's why the best use case is in large volume roles. The probability of an assessment identifying the best person out of 5 is pretty slim, even with a correlation coefficient of 0.5. A correlation of 0.5 really only demonstrates that 25% of the variance in job performance can be explained by the variation in the predictor.

The way I've always explained it to a laymen is like this; You have two bowls of red and green marbles. If you select a green marble you have a good hire, if you select a red marble you have a bad hire. Would you rather pick from a bowl that was 50/50 or one that was 70/30 in favor of green marbles?

There's certainly the assumption that they do, but I'm not entirely convinced that personality and IQ tests give you a 70/30 bowl.

Well depending on the correlation between the overall battery and job performance that's fairly simple to mathematically derive. 70/30 was an example, it could be 63/37 or even 60/40, but either way, if you were hiring 50k people/yr I bet you'd be willing to pay a decent chunk of change for the 60/40 bowl as compared to a 50/50 bowl. This of course only focuses on the probability of hiring success and ignores the efficiencies gained by basically making the candidate sort themselves into green or red instead of a person manually doing it based off of their own heuristics.

Not the numbers I'm quibbling with.  I don't really believe that a personality test is necessarily going to increase the number of good hires that you get at all.

In that case, you're choosing between two bowls of 50:50.  One bowl costs a little bit more in time and effort on the part of the person doing hiring.

I’d agree with you. I’ve never built and validated an assessment that only measured personality.

Kyle Schuant

  • Handlebar Stache
  • *****
  • Posts: 1314
  • Location: Melbourne, Australia
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #55 on: November 01, 2019, 05:28:37 PM »
I feel I should clarify the "pick out of a hat" comment; I did previously but I'm aware nobody reads a whole thread before commenting on it.

If you get 100 applications, you should not pick one of them from a hat. But the normal procedures of sorting for academic qualifications, phone and in-person interviews, etc, should be able to whittle it down to 20 or so applicants, any of whom would do a decent job. I'm prepared to believe the number might be as high as 50, but not lower than 10. Let's call it 20.

Among them may be 3-4 who would do a great job. However, the lengthier and more involved recruitment process required to find them is likely to make those 3-4 awesome people give up and move on to a potential employer who doesn't screw them around as much - after all, awesome people get to pick and choose.

I recall at a gym the manager getting a letter starting, "Dear Genesis gym manager" - to a YMCA email address. In 100 applications, at least 20 can be immediately binned for that sort of stupidity. Another 30 can go because of insufficient qualifications. And so on. These normal processes reduce the 100 to 20 people who will do a decent job. It's that whittled-down number you could pick randomly from and do alright.

The personality tests are an interesting one. Agreeableness and conscientiousness as measured by them are strong predictors of career success. This makes sense: if you have to spend 40 hours a week with someone, it's better if they're pleasant to hang out with, or at least not an annoying prick, and having someone who does what they said they'd do when they said they'd do it makes life easier, too.

As the company gets larger, agreeableness and conscientiousness become less important, since however annoying you are people can forget you in the crowd, and someone will pick up the slack you dropped in those harried 12.5 hours a week of actual work most people report doing.

Bloop Bloop

  • Handlebar Stache
  • *****
  • Posts: 2139
  • Location: Melbourne, Australia
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #56 on: November 01, 2019, 06:14:42 PM »
Kyle, I think what you're missing is that IQ and personality tests are only used for things like entry-level and graduate jobs, at large companies. "Awesome people" will not give up, because at that stage of your life you are not awesome, as you have no experience or CV to speak of. Star graduates are expected to jump through numerous hurdles just to work at a prestigious firm which then treats them like shit and pays them a humiliating salary (less so in the US where 6-figure starting salaries are common).

Once you have 3-5 years of experience behind you, the job finding process consists of you calling up recruiters or friends and doing a deal with barely an interview. However, at the early stages when the applicant numbers are high and career history is low, you need to whittle down applicants.

mizzourah2006

  • Handlebar Stache
  • *****
  • Posts: 1063
  • Location: NWA
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #57 on: November 01, 2019, 07:10:53 PM »
I feel I should clarify the "pick out of a hat" comment; I did previously but I'm aware nobody reads a whole thread before commenting on it.

If you get 100 applications, you should not pick one of them from a hat. But the normal procedures of sorting for academic qualifications, phone and in-person interviews, etc, should be able to whittle it down to 20 or so applicants, any of whom would do a decent job. I'm prepared to believe the number might be as high as 50, but not lower than 10. Let's call it 20.

Among them may be 3-4 who would do a great job. However, the lengthier and more involved recruitment process required to find them is likely to make those 3-4 awesome people give up and move on to a potential employer who doesn't screw them around as much - after all, awesome people get to pick and choose.

I recall at a gym the manager getting a letter starting, "Dear Genesis gym manager" - to a YMCA email address. In 100 applications, at least 20 can be immediately binned for that sort of stupidity. Another 30 can go because of insufficient qualifications. And so on. These normal processes reduce the 100 to 20 people who will do a decent job. It's that whittled-down number you could pick randomly from and do alright.

The personality tests are an interesting one. Agreeableness and conscientiousness as measured by them are strong predictors of career success. This makes sense: if you have to spend 40 hours a week with someone, it's better if they're pleasant to hang out with, or at least not an annoying prick, and having someone who does what they said they'd do when they said they'd do it makes life easier, too.

As the company gets larger, agreeableness and conscientiousness become less important, since however annoying you are people can forget you in the crowd, and someone will pick up the slack you dropped in those harried 12.5 hours a week of actual work most people report doing.

You’re thinking about everything on to small of a scale. I work with companies that have 150k+ hires per year for a specific set of roles, hell I worked for a company with 5 million + applicants per year for entry level jobs. You expect these places to hand sort through all of them and read all those resumes? I’ve already agreed that these assessments are less valuable and pre-quals like specific degrees become more important as roles become more specific. If I used these to hire software engineers or data scientists it would be a data point and given to people that on paper have the skill set. I’d place way more value on job simulations like working through a common DS problem.

BTW...impressed by your lift. My best big 3...all raw are: bench: 450@235, squat 535x3@225 and dead 465x3@225 all lbs of course. I was more of a bodybuilder than a power lifter in my prime, don’t go higher than 315 on anything anymore. I like functional strength for mountain biking now.

GuitarStv

  • Senior Mustachian
  • ********
  • Posts: 23128
  • Age: 42
  • Location: Toronto, Ontario, Canada
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #58 on: November 01, 2019, 07:34:10 PM »
Kyle, I think what you're missing is that IQ and personality tests are only used for things like entry-level and graduate jobs, at large companies. "Awesome people" will not give up, because at that stage of your life you are not awesome, as you have no experience or CV to speak of. Star graduates are expected to jump through numerous hurdles just to work at a prestigious firm which then treats them like shit and pays them a humiliating salary (less so in the US where 6-figure starting salaries are common).

Once you have 3-5 years of experience behind you, the job finding process consists of you calling up recruiters or friends and doing a deal with barely an interview. However, at the early stages when the applicant numbers are high and career history is low, you need to whittle down applicants.

So, I think that I mentioned this earlier in the thread . . . but I was given both personality and IQ tests multiple times when searching for a tech job a few years ago.  This was after 10 years of experience as a software engineer.  I was not applying for entry level jobs.

Kyle Schuant

  • Handlebar Stache
  • *****
  • Posts: 1314
  • Location: Melbourne, Australia
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #59 on: November 01, 2019, 08:01:40 PM »
Kyle, I think what you're missing is that IQ and personality tests are only used for things like entry-level and graduate jobs, at large companies.
As has been mentioned, they're used for more than entry-level jobs. And even then they're not necessarily useful. Again: just an apparently fair way of choosing randomly.

Quote
Star graduates are expected to jump through numerous hurdles just to work at a prestigious firm which then treats them like shit and pays them a humiliating salary (less so in the US where 6-figure starting salaries are common).
Much as in medicine.

I would suggest that in law, you are perhaps driving out some people who would have been very good, while keeping the more desperate sort with no other options. Certainly we can see that in medicine, people have successfully defrauded others about their credentials - people with no medical degree successfully posing as a qualified doctor, working alongside other qualified doctors, for 10 years or more. This suggests that the rigorous testing is not actually able to test job-related competence, and/or that actually qualified people aren't much more competent than unqualified ones. Which concerns me as a potential patient, but there you go.

Quote from: mizz
You’re thinking about everything on to small of a scale. I work with companies that have 150k+ hires per year for a specific set of roles, hell I worked for a company with 5 million + applicants per year for entry level jobs.
I would suggest that after a certain level, economies of scale have diminishing returns, which eventually become negative returns. Most of us are astounded that people who win the lottery can end up bankrupt, how much more astounding is a multi-billion dollar company collapsing?

Quote
BTW...impressed by your lift.
Don't be. My gym's a small sponsor of a powerlifting meet going on right now - women will be going in squatting 250kg. Not drug-tested, of course, but it's not like drugs double your lifts, they're all remarkable women either way.

From training people for health, I think if you can regularly and reliably deadlift your bodyweight for work sets, you've probably reduced your chances of (further) back injury as much as you ever will, about the same for squat, and you'll be fine for recreational sports - nobody wants to be the guy who pops his ACL playing lunchtime soccer in the park next to work. Essentially everyone under 50yo who is not on a disability pension can do that in 3-6 months of not trying very hard.

Of course, we can and should do more. But DL and SQ about bodyweight is going to be pretty protective of you, keep it up and you'll never need a walking frame. Regularly doing some sort of overhead press will cut down on all those shoulder issues you see in desk workers.

Obviously we were just joking about the deadlift assessment. But it'd go a long way to reducing workplace injury claims. It's sure as shit needed in physical jobs, which includes things like nursing. It doesn't matter how awesome they are if they're off work with a popped disc.

HBFIRE

  • Handlebar Stache
  • *****
  • Posts: 1311
  • Age: 45
  • Location: Huntington Beach, CA
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #60 on: November 01, 2019, 09:18:37 PM »
It can generally be assumed that anyone applying for a professional job has a substantial IQ, so testing just to confirm this is a waste of everyone's time.

Wow really?  Not in my experience.  I'm amazed at how many people get through college with low IQ's.  I think cognitive testing is a great start for hiring to narrow down the pool (as long as the testing is accurate as you mentioned).




I'd also disagree with you on the thought that everyone applying for a professional job has a relatively high IQ. I've seen executives at fortune 50 companies that only get a 10 on the Raven's. Plus when you throw in the fact that so many people today inflate their resumes and all of the information out there about the coding challenges that each company does, etc. It's possible that someone with a relatively low IQ (for a tech position) could get through without encountering a novel question or scenario. IQ is extremely important to solving novel tasks/problems, which I'm certain any tech person will tell you is basically their job.
Now if you are talking about testing doctors then I'd agree with you, it's a waste of time because they all had to get high MCAT scores and graduate from medical school.





Thanks for weighing in, yes this is was my initial thought as well.

« Last Edit: November 01, 2019, 09:34:00 PM by HBFIRE »

gaja

  • Handlebar Stache
  • *****
  • Posts: 1681
Re: I'm seeing personality tests and cognitive tests in tech now
« Reply #61 on: November 02, 2019, 05:10:36 AM »
Kyle, I think what you're missing is that IQ and personality tests are only used for things like entry-level and graduate jobs, at large companies. "Awesome people" will not give up, because at that stage of your life you are not awesome, as you have no experience or CV to speak of. Star graduates are expected to jump through numerous hurdles just to work at a prestigious firm which then treats them like shit and pays them a humiliating salary (less so in the US where 6-figure starting salaries are common).

Once you have 3-5 years of experience behind you, the job finding process consists of you calling up recruiters or friends and doing a deal with barely an interview. However, at the early stages when the applicant numbers are high and career history is low, you need to whittle down applicants.

So, I think that I mentioned this earlier in the thread . . . but I was given both personality and IQ tests multiple times when searching for a tech job a few years ago.  This was after 10 years of experience as a software engineer.  I was not applying for entry level jobs.

I went through the personality testing and whatnot testing hurdle to get my current job. This is in public management, and I came directly from a similar level position and had quite a few years of experience. The IQ-part was ok enough, but the personality thing was bullshit and nearly caused me to turn down the job. Not because of the scores, but because I got into an argument with the HR representative about how to interpret it. I'm usually very diplomatic, and am known for being able to cooperate even with the most quarrelsome individuals. But that interview went completely wrong.

The thing is; I have a touch of photographic memory, and the personality test they used was quite small, with less than 100 questions. Also, I had recently read up on personality tests just for fun, and knew what the different types of questions were used for. So in the meeting with HR about the test results I explained that this might cause errors they should be aware of when considering the results, and I pointed out several places where the summary of the results came out wrong due to the way I had answered specific questions (citing those questions and how I had answered them). The cause for conflict? The HR lady accused me of lying. "This is a foolproof scientific test, and nobody is able to remember all the questions and answers while answering it. There are built-in failsafes to ensure that noone cheats". I admit I have a weak spot when it comes to responding politly to people calling me a liar and cheater.

The outcome: HR lady strongly adviced my boss not to hire me (advice she luckily ignored), and I nearly declined the offer since I wasn't sure I could work with an organization with that type of HR people (but was convinced by a friend that worked there that this was not typical of the organization). After working with me for a few months, the boss got the HR lady to admit she was wrong about me.