The new number crunchers; One of the world’s first data scientists turned his geeky love of maths into a lucrative career

May 30, 2013 5:12 pm

The new number crunchers

By Emma Jacobs

When Jeff Hammerbacher lies in bed at night trying to sleep, he meditates on a maths problem. “I find it peaceful,” he says. Reflecting on “timeless entities in your brain allows you to get to the objective truth. It makes me calm inside my head”. Relaxation can be a problem for Mr Hammerbacher, who was diagnosed with general anxiety disorder. “I do have a lot of thoughts in my head, metaphysical dilemmas.”

Meditating on numbers is more than a cure for insomnia. It drives the 30 year old professionally. He is a “data scientist”, the hottest job title in Silicon Valley. Harvard Business Review ran an article co-authored by data scientist D.J. Patil, calling it “the sexiest job of the 21st century”. Mr Hammerbacher actually coined the term in 2008 with Mr Patil, who was the head of data and analytics at LinkedIn when he was leading Facebook’s team.

Mr Hammerbacher calls good data scientists “data rats”. Athletes are often considered “gym rats” if they spend a lot of time in the gym, so Mr Hammerbacher believes “data rats” need to spend a lot of time with data. So, when he has downtime he tries to get in some data science.His wife recently found a list he had written when he was seven of things he liked best. It said: “My favourite things to do in life are eat and do math.”

What makes a good data scientist? Mr Hammerbacher has too much to say on the topic. He has tried to refine his thoughts for an audience of Berkeley undergraduates and is writing a textbook on the topic, with the aim of creating a new generation of data scientists. “They should have a mastery of software carpentry, basic probability and statistics, and project management,” he says. “They are relentlessly pragmatic, effective communicators, and have a firm command of the potential work products of data science.”

In 2008 Mr Hammerbacher co-founded Cloudera, where he is now the chief scientist, deliberately eschewing the managerial demands required of a chief executive position. The Silicon Valley company, which employs more than 350 people, builds software to enable the storage and analysis of data. Its ultimate aim, he says, is to allow “organisations to profit from all of their data”.

The long-term vision is “to drive the cost of data storage and analysis to zero so that we can remove data storage and analysis as a bottleneck to scientific discovery”. Most of the software produced at Cloudera is open-source.

The maths of big data

● In 2010 Google’s Eric Schmidt said that about five exabytes of data, or the equivalent of 250,000 years of DVD-quality video, were created in the world every two days.

● According to a McKinsey Global Institute report in 2011, 30bn pieces of content on Facebook were being shared every month.

● About 1.5m more data-savvy managers are required to take full advantage of big data in the US, the report said.

● For less than $600, an individual could buy a disc drive big enough to store all the world’s recorded music.

A data scientist combines lots of old roles. “They assimilate work processes. Previously the roles were atomised: statisticians, business analysts and software engineer. Now we can combine these roles.” These are not geeks in darkened rooms – they need to understand business in order to ask the right questions of the figures.

His father, who worked on the assembly line at a General Motors factory in Michigan, recently asked why he decided to choose to work in data, which has proved to be a lucrative career option. “I didn’t choose it. I set out to do something interesting and it happened to be important.” Curiosity, he says, is key to the profession.

Thousands of data scientists are employed by start-ups and large companies located in industries such as retail and banking as well as public sector organisations. The emergence of the role reflects the fact that companies are now wrestling with huge swaths of information from online interactions to the use of automated teller machines. Data are also coming in new forms such as Facebook likes or YouTube posts.

Companies use sophisticated software to analyse these data, looking for hidden patterns and insights that they can use to tailor their products and services to customers, anticipate demand or improve performance. A 2011 study by McKinsey, the consultants, suggests that retailers analysing large data sets to their fullest could increase operating margins by 60 per cent. The global healthcare industry could reduce annual costs by 8 per cent, or $200bn. The study also warns that there is a shortage of workers with the skills to analyse the data. It says there will be a shortfall by 2018 of about 140,000 to 190,000 individuals with the right analytical expertise.

A mix of geeky and sporty, at school the two poles of Mr Hammerbacher’s existence were baseball and maths. His parents enrolled him in school early to get him out of the house. “I was rambunctious.”

His friends were jocks, though while on baseball tours he read physics textbooks. Playing sports helped him to develop a mental toughness, he says, a good preparation for Silicon Valley. “Baseball is impeccable training for start-up life. There’s lots of failure and you play a long game. You have to be focused on persevering through short-term failure.”

Due to a sports injury he turned down a baseball scholarship at the University of Michigan, instead studying maths at Harvard, which is where he met Mark Zuckerberg.

He had so much fun socially at university that he lost his way with his studies, dropped out for a year and took on a variety of jobs including working in a bookstore and on the assembly line at GM with his father. “I realised if I was going to be a mathematician I would need a degree,” he says. So he returned to Harvard.

After working for 10 months at Bear Stearns, the investment bank, as a quantitative analyst, he was recruited by Facebook and given the job title “research scientist”. His remit was to analyse how people used the social network.

Over the next two years he assembled a team to build new analytical technology tools, which gathered insights into people’s relationships and likes, the foundation of Facebook’s business.

He is evasive on the subject of the social network’s use of data and concerns about privacy, though he said in an interview after leaving the company that “the best minds of my generation are thinking about how to make people click ads … That sucks.”

Today he will not be drawn. “I haven’t worked at the company in nearly five years and don’t think about it much. I was interested in data management, not so much in the product.”

. . .

The experience of working at Facebook persuaded him to design a role at Cloudera, which means he does not have anyone reporting directly to him. “At Facebook I moved into management at 23. I hadn’t really had enough work experience to be a manager,” he says.

It also meant that he was too far removed from the work that he loved, and when it comes to overseeing others he describes himself as “an eclectic iconoclast”.

He hates having a boss. “Ask anyone who has managed me, they will tell you I’m a terrible employee. I question everything.” The opposite of a peacemaker, he says he exacerbates conflict on purpose as a way of bringing problems to the surface early and before they get too big.

Calling himself a “contrarian”, he says his favourite time is when his business is being challenged. “My happiest moments are when our backs are against the wall. My unhappiest is when people are feeling in the lead, they are coasting.”

His wife has also started a company, Rock+Health, an incubator for healthcare technology start-ups.

Is the atmosphere at home relaxing?

“She’s two to three years along, whereas I’m six or seven. We’re complementary rather than competitive. We could work 24 hours a day so we have to regulate each other.”