The grim reality of big data jobs when you work in a bank

So you work in Big Data in a bank? Lucky you! You probably turned down a sweet little fintech because you wanted to do a lucrative algorithmic trading job instead. You’re sure the in-house recruiter said that you would be working with the trading team, but somehow the work you're actually doing is all about market surveillance. Your big data job in a bank is about making sure the real traders aren’t being naughty.

There are several problems with this. Either you don’t find anything - which makes you look incompetent, or  else you do find something - which makes the real traders look bad and reduces the likelihood that you'll ever get a sniff of their jobs. And yes, you’re in the compliance bonus pool. - Yes, that’s a real thing, not something I made up to scare you.

There's also the job itself. You may well find that the data you need data is in Oracle. Yes, Oracle. Do you recall your dad telling you about Oracle back in the 1990s (presuming you were born then)? So you’ve got to grind though abstractly documented schema and then get stalled whilst Oracle conduct a licence audit.

When you joined a bank, you thought you'd be around the top of the food chain. After all you worked for this. You got your PhD while your academic peers got model validation jobs with just a masters, or even less. But now you've arrived and you realize that the freelance Kdb/Q consultant on the desk next to you - who is very far from having your PhD in stochastics, is paid twice what you're getting and going home before 8pm.

Then you discover that the Oracle data was the good news, because the trade capture data uses a different time base to the market data and when you look closely it appears that cash equities took a long position in IBM 4 years before Columbus went to America. That’s not your fault, but it is now your problem. Same for the positions in Gilts that the DMO says don’t exist.

As time goes on, you are starting to realize that when they said you’d be working “close to the business”, they were speaking to the truth, This is because close is not the same as in. At best, you're going to be part of the IT bonus pool, which is set based upon the performance of the legacy Visual Basic project.

Ah yes, VB, how you sneered at this in school. You are now a functional programmer, you need to know that F# is good for your CV.

But you’re learning, right? What the nice HR people didn't mention is that the training they're going to give you is actually a crash course in VBA so you can bludgeon your outputs into Excel, because in a bank if it doesn’t appear in a spreadsheet it doesn’t exist. Oh yes, and you will soon know too that C++ will never die.

Of course you build in Python, yes you do. We too are modern people, but you’re in an IT Department now. You can’t just use the latest versions or the package you want, because security and because two years ago a developer who was part of an OSS development made it structural to the inhouse systems then quit and the project has no support. You are now a legacy Python programmer at 26. Welcome to the team.

Dominic Connor is a quant and a headhunter and a journalist