The real reasons investment firms can’t find good data scientists

eFC logo

Over two days at this week's AI and Data Science and Trading Conference in New York, one general theme became very clear: banks, hedge funds and asset managers are desperate for good data scientists. The prevailing theory has always been that the supply just isn't enough to meet the demand. But hiring managers across several different panel discussions suggested that wasn’t necessarily the case. The bigger issue is that the majority of master’s and PhD candidates simply aren’t skilled enough in all the areas necessary to demand big pay packages.

When asked about the biggest challenge facing the data science and machine learning community over the next five years, Anthony Ledford, Man AHL’s chief scientist, pointed to the lack of “suitable” candidates, noting that too many young people think they can just plug-and-play data mining tools and be successful. “Finding those people with the curiosity and ability to really scrutinize data and come back with something different,” is a challenge, Ledford said, noting his firm has become extremely “fussy” about who they hire.

Sameer Gupta, head of data sourcing at Point72, said that much of what he considers “talent” in data science is passion and a curiosity to learn. Yet he fails to recognize those traits in many candidates with perfect resumes, 4.0 GPAs and Ivy League educations. “They know all the buzzwords, but they are rarely the people for the job," he said. Some new recruits are also surprised or turned off when they are asked to do so-called “janitor work,” an industry term for data preparation and cleaning, according to Two Sigma’s machine learning guru Claudia Perlich.

The apparent conflicts between resume and execution is why Gupta said he’s not a fan of interviews, something echoed earlier by his colleague Matthew Granade, Point72’s chief market intelligence officer. Instead, they prefer projects and tests.

“Give them a problem to solve, throw them in the water, and if they don’t like it or can’t do it, they often don’t even come back to you,” he said. “It’s a great way to figure out who to bring on.”

The general takeaway was that firms need to look beyond target schools and GPAs to find niche talent. Roughly 80% of PhD-level machine learning talent works at Google and Facebook, said Sarah Hoffman, vice president of AI and machine learning research at Fidelity Investments, citing an unknown study (likely touched on in this paper). That leaves just 20% for everyone else, meaning they need to be more creative.

Investment firms have to get involved in more university projects and hackathons, she said. “We need to get to the point where we can say: ‘I don’t care about if you went to college, I’m going to give you data and see what you can do with it.’”


The final struggle for hiring managers is finding the right personality match. An AI and machine learning headhunter had previously noted that a lack of communication skills and an inability to interface with key stakeholders at investment firms has held back many potential data scientists.

One such example was given to us anonymously by a speaker at a different forum who recalled interviewing a candidate who had a strong resume for an engineer at an asset manager but who didn’t appear to have much in the way of communication skills or a willingness to engage. The interviewer tried to get the candidate to open up a bit, asking him to say something about himself that wasn’t on his resume. After pausing for a few seconds, the candidate questioned why there would be anything that he wouldn't put on his resume.

In an attempt to clarify, the interviewer said he just wanted to get to know him beyond his list of achievements and skills. The candidate paused again, this time for upwards of 45 seconds. He finally responded: “I don’t own an air conditioner.”

In his report, the interviewer noted that the prospect “could well be a genius, but you can never put him in front of anyone.” They decided to pass.

Have a confidential story, tip, or comment you’d like to share? Contact: btuttle@efinancialcareers.comBear with us if you leave a comment at the bottom of this article: all our comments are moderated by actual human beings. Sometimes these humans might be asleep, or away from their desks, so it may take a while for your comment to appear. Eventually it will – unless it’s offensive or libelous (in which case it won’t).  

Related articles