It Takes Teams to Solve the Data Scientist Shortage

February 14, 2014, 3:27 PM ET

It Takes Teams to Solve the Data Scientist Shortage

By Jeanne G. Harris, Nathan Shetterley, Allan E. Alter and Krista Schnell

If you are looking for data scientists to take your company to Big Data nirvana, we have news for you. Unless you run a hotshot Silicon Valley company, chances are enough of them aren’t going to walk through your door. There’s a more realistic approach for the rest: divide up the job and conquer.

Data scientists are the experts who design the intricate models, algorithms and visualizations that help companies distill insights from huge volumes of chaotic data. But being a data scientist is not only about data crunching. “It’s about understanding the business challenge, creating some valuable actionable insights to the data, and communicating their findings to the business,” says Jean-Paul Isson, the global vice president of predictive analytics and business intelligence at Monster WorldwideMWW -0.40%, Inc.

Data scientists are in high demand, but our research has found there is simply not enough PhD talent to fill the jobs. The shortage is especially severe in the U.S. where 80% of new data scientist jobs created between 2010 and 2011 have not been filled, according to our analysis.

What’s more, the number of graduates with the requisite technical skills isn’t keeping up with rising demand. Training quants to become data scientists can help, but training can take years.

But there is another solution which holds real promise: Create a team of people who individually lack the full skillset of a data scientist, but as a group possesses them all. When physicists take on a big project, they bring together a team to design the equipment, run experiments and analyze the data. Likewise, it makes sense to divide the labor of a data scientist rather than search for one person who can do it all.

For example, Monster Worldwide has built up a team of data crunchers, statisticians, business analysts, computer scientists and “navigators” who can explain findings to managers. And American International Group, Inc. is assembling a “Science” team of statisticians, business analysts, project managers, systems architects and engineers for its Property and Casualty business.



The 7 Skills of Data Scientists: Executives are struggling to find individuals who possess all eight data scientist skills and abilities. A data scientist team can serve as an alternative.

The size of these teams can vary: from a handful of people for pilots and short tactical projects, to 20 or more for longer projects and ongoing analytical work. A small team could include one or more software engineers and quantitative analysts who know Hadoop–an open source computing environment often used for Big Data processing–and can write scripts in the languages used to prepare, integrate, clean, run and analyze Big Data. It could also include a systems architect to maintain the systems that host the data, and ensure these systems can communicate with one another.

Larger teams add specialists to supplement these roles: programmers to write the commands which prepare data for use; quantitative analysts who can dig deeper into the findings to find the insights; data visualization specialists who can turn findings into easy-to-understand graphics; and project managers/liaisons to oversee their efforts.

On the business skill side, data scientist teams can also include quantitatively oriented business analysts and visualization designers. Business analysts act as business function experts who understand what information is most valuable to the business, and can communicate findings back to functional managers in language they understand. They are the most suitable member of the team to serve as the liaison with different departments. Visualization designers are specialists who can effectively use data to tell stories through graphics. Another option is to create what Mr. Isson calls a “navigator,” a professional communicator who shares the data and the team’s conclusions through straightforward language and graphics. At Monster, each business function is assigned a navigator who serves as its liaison with the team.

Between them, these data scientist teams will have the necessary knowledge of the company’s business needs, and the ability to design, perform and deliver data insights in easy-to-grasp ways.

Most organizations already have a great deal of experience managing teams and projects. But executives should keep the following points in mind for creating effective data engineer-scientist teams:

Widen the recruiting pool. Don’t just look for people who already have these roles and skills in competing companies. Search outside your industry, and even outside the business world. Restless academics with strong analytical skills may also be able to find a new home on a data scientist team. Artists, in particular graphic designers, can bring creativity and imagination to data visualization.

Communicate, collaborate, but don’t necessarily co-locate. In an ideal world, all members work together in the same location and even room. But companies should not give up on teams if they cannot co-locate them. Videoconferencing is just the start. Remote workers can be paired up and set up to share the same screen on their computers. Monster’s dispersed data scientists follow a common framework when they produce their findings, says Mr. Isson. “We make sure our people around the world have common goals, methods and processes, and a common view of our market and customer base.”

Boost effectiveness and retention through team learning: On a data scientist team, it’s helpful to encourage members to pick up skills from other members. Over time, this creates flexibility; when one member is unavailable, others can pick up the slack. It can also create a unit that is more resistant to attrition. When everyone is learning new skills from their teammates and thus furthering their careers, team members have more reasons to stay put.

In addition, the time-proven wisdom about managing teams bears repeating: Data scientist teams, like others, flourish best when there is effective leadership, a strong mandate from above and clear goals. We also recommend starting with short, low-risk projects to learn the ropes before tackling longer, more complex ones.

Businesses are long on experience with managing teams. They will remain short on data scientists. Why shouldn’t businesses use what already know to compensate for what they lack?

Jeanne G. Harris is the managing director for IT and analytics research at the Accenture Institute for High Performance; Nathan Shetterley is an R&D manager with the Accenture Technology Labs; Allan E. Alter is a research fellow with the Accenture Institute for High Performance; Krista Schnell is an R&D developer with Accenture Technology Labs.


About bambooinnovator
Kee Koon Boon (“KB”) is the co-founder and director of HERO Investment Management which provides specialized fund management and investment advisory services to the ARCHEA Asia HERO Innovators Fund (, the only Asian SMID-cap tech-focused fund in the industry. KB is an internationally featured investor rooted in the principles of value investing for over a decade as a fund manager and analyst in the Asian capital markets who started his career at a boutique hedge fund in Singapore where he was with the firm since 2002 and was also part of the core investment committee in significantly outperforming the index in the 10-year-plus-old flagship Asian fund. He was also the portfolio manager for Asia-Pacific equities at Korea’s largest mutual fund company. Prior to setting up the H.E.R.O. Innovators Fund, KB was the Chief Investment Officer & CEO of a Singapore Registered Fund Management Company (RFMC) where he is responsible for listed Asian equity investments. KB had taught accounting at the Singapore Management University (SMU) as a faculty member and also pioneered the 15-week course on Accounting Fraud in Asia as an official module at SMU. KB remains grateful and honored to be invited by Singapore’s financial regulator Monetary Authority of Singapore (MAS) to present to their top management team about implementing a world’s first fact-based forward-looking fraud detection framework to bring about benefits for the capital markets in Singapore and for the public and investment community. KB also served the community in sharing his insights in writing articles about value investing and corporate governance in the media that include Business Times, Straits Times, Jakarta Post, Manual of Ideas, Investopedia, TedXWallStreet. He had also presented in top investment, banking and finance conferences in America, Italy, Sydney, Cape Town, HK, China. He has trained CEOs, entrepreneurs, CFOs, management executives in business strategy & business model innovation in Singapore, HK and China.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: