Getting Up to Speed as a New Data Science Manager – an Optimization Problem Getting Up to Speed as a New Data Science Manager – an Optimization Problem
Pursuing a data science leadership position as an outside hire is reserved for the crazy, the optimistic; the brave, or perhaps... Getting Up to Speed as a New Data Science Manager – an Optimization Problem

Pursuing a data science leadership position as an outside hire is reserved for the crazy, the optimistic; the brave, or perhaps just the brashly confident. Because once hired, new managers must learn along numerous axes very quickly. The business domain, the data, the individual contributors, the leadership team, existing politics, and the organizational culture all compound simultaneously. After two days of onboarding, the reality of the challenge sets in.

In-Person and Virtual Conference

April 23rd to 25th, 2024

Join us for a deep dive into the latest data science and AI trends, tools, and techniques, from LLMs to data analytics and from machine learning to responsible AI.

Orienting, not onboarding

New computer, new retirement account, approximately 107 team help tickets, and one headshot later, and onboarding is complete. Being brought aboard, means to provide new hires the bare necessities of survival at a company. But we don’t hire anyone — especially leaders — to merely survive. Managers are expected to quickly and meaningfully contribute to their new organizations. Despite the well-known fact that external hiring poses significant challenges compared to internal promotions, new managers aim to make an impact within short windows, commonly referred to as 30–60–90 day plans.

Given the high organizational — and often self-set — expectations, new leaders find themselves immersed in a massive optimization problem, trying to select the course of action most likely to deliver a desired outcome (which is often unclear or shifty). The leader attempts to determine both the measures of success and the optimal magnitude and combination of those measures to achieve success. This, collectively known as “the game,” in the corporate world, rapidly evolves and is different at each company in each of its seasons. Whereas an internally hired data leader will understand many of the dimensions and appropriate dosages a priori, an externally-hired leader will not. The external hire must play in rapid iteration, adapting with the feedback collected with each action. One searches the feature space — if you will — looking for an optimal orientation.

Onboarding puts you “in the game.” Orientation helps you “play the game.”

First, determine which dimensions matter

Every company is different, but a few foundational dimensions are constant. To be a successful leader, one must craft a well-balanced elixer, pillared by:

  1. People
  2. Project management
  3. Strategy

Once the dimensions are established, the “game” is to reach a combined optimal position, where the leader strikes the right balance of each of the three dimensions. In the three-dimensional plot below, each dot represents one possible position a new manager could take.

For each axis (and others not shown), a new data science leader seeks the optimal position for the team, themselves, and the broader company.


People are, by definition, the reason a leadership position exists. I hate to say people need managers. It’s not really true, since workplaces are full of adults. Rather, leaders are needed in order for people to individually and collectively reach their full potential. There are many takes on leadership styles — too many to cover here — but the constant throughline is the effort to improve those who are led, and ultimately, improve outcomes for the organization.

For a new manager, establishing rapport with each team member is crucially important. Given the bestowed authority of the position, latent emotional intensity simmers during the fluid time in which a leader is attempting to assimilate into — but also positively impact — their new team. For everyone, reorienting around a new set of expectations and processes, in addition to navigating new interpersonal relationships, makes for intense (even if they’re not negatively charged) interactions.

There are so many ways in which one can perform poorly on the People dimension. And more often than not, new leaders fall short of giving proper attention to their people. Whether it’s poor listening, lacking transparency, or just general disinterest in individual team members — it’s common for leaders to be assessed as “not doing enough,” when it comes to people managing. But it’s not malicious — or even deliberately neglectful (this person applied to be a manager, after all.). Rather, it’s likely because they’ve miscalculated the degree of need on that dimension, often opting for focus on other areas.

Too much emphasis on the People dimension can also be detrimental if it comes at the cost of neglecting the strategy and project management axes. Represented in three- and two-dimensional space, a manager too focused on people would look like:

A near singular focus on the people can be perceived as “too nice,” if the other job duties are not fulfilled. There’s no question that being liked is important, but when kindness supersedes other attributes, it can portray a lack of competency and/or confidence.

Over-orientation around being liked:

❌ Risks appearing incompetent on technical, business, or project coordination elements of the job.

❌ Promotes an outgrowth of ad hoc & individualized processes, which can result in a lack of continuity, uniformity, and clear expectations.

❌ Causes team members to question the leader’s ability to develop and execute on long-term vision for the department, and how each team member fits into it.

Striking the correct balance on the People dimension might look like:

✅ Team members are empowered to progress the work. They are proactive, but collaborative, seeking counsel from teammates and the new manager.

✅ Each team member feels “seen” for their unique skills and contributions to the team, while still demonstrating the skill and will to fill in when gaps arise.

✅ Quantity and quality of total department outputs are perceived as even across each team member (i.e. if performance management needs to occur, it is).


Sometimes, a new data science leader focuses on implementing and executing strategy.

Compared to hiring internally, sourcing an outside manager is riskier, more expensive, and lengthens the overall time-to-value-delivered for the position (i.e. long ramp up). According to SHRM, external hiring should be considered when:

  • Tough corporate turnarounds or strategy shifts are underway
  • Specific skills are needed that are not readily available within the organization
  • Succession planning and performance information is inconsistent, absent, or hard to access

In all three cases, the newly hired manager is stepping into the company during a difficult season: strategy shifts can be very painful, skill gaps can be uncomfortable to recognize, and absent performance management solidifies the inertia of the status quo.

For example, consider a new data science leader is hired with three objectives. First, upskill the team. Second, use the newly-skilled team to build data products that enable the company to go public in two years. And third, implement proper technical evaluation measures — everything from technical candidate assessments, to key competencies, and rewriting job descriptions. With all three of SHRM’s key considerations at play, it’s easy for a manager to focus exclusively on creating an exhaustive strategy for how to surmount the situation.

Most likely, this will result in an underemphasis on people and project managing, giving a “know it all” vibe.

Represented in three- and two-dimensional space, a manager too focused on strategy would look like:

Being a “know it all” can result in:

❌ Problem (and solution) misalignment due to incomplete or misunderstanding of the domain. This creates low value-added work.

❌ A feeling that the leader lacks respect for the team’s knowledge, experience, and current projects; disempowerment.

❌ Lack of assimilation of the leader into the team; staunch hierarchy.

A more measured degree of strategy is likely characterized by:

✅ The manager demonstrates a near-tireless pursuit of understanding the core business problems so the team can apply data.

✅ The work environment values all ideas and inputs; it’s a psychologically safe space to consider hypotheses, conduct experiments, and fail but learn.

✅ The team collaboratively determines a strategic direction wherein short, medium, and long-term value can be added to the company.

In-Person Data Engineering Conference

April 23rd to 24th, 2024 – Boston, MA

At our second annual Data Engineering Summit, Ai+ and ODSC are partnering to bring together the leading experts in data engineering and thousands of practitioners to explore different strategies for making data actionable.

Project Management

Taking project management too seriously from the beginning commits the error of creating a “command and control” environment.

Like strategy, a new leader’s approach to this axis is likely influenced by conversations during interviews, as well as the focus of the manager’s hiring manager. If the business “doesn’t know what data science is doing,” then the most urgent dimension for a new hire might be managing existing projects.

The graph displays this mis-orientation, as a combination of low people focus, low-medium strategic emphasis, and high attention paid to project management. Of the many orientation errors a data science manager can make, command-and-control can be particularly egregious on account of its absolute obliteration of creativity among the team. Suboptimal outcomes include:

❌ Increased emphasis on “tracking work” creates a ticket-based, transactional system. Rather than projects coming forth from the data (and domain knowledge), rewarded projects are those with clear outcomes, delivered “exactly as the stakeholder wanted,” and on timelines unnaturally fast for true data science.

❌ Added bureaucracy. Read: meetings, meetings, and meetings about meetings. Not only is this incredibly expensive (i.e. the summed hourly rates of the attendees), but it crowds out development time. The work is at risk of being misguided, uninspired, and superficial. There is a lot of low-value-add busyness that comes from command and control.

❌ Inability to develop, retain, or attract talent, because data scientists do not thrive in this environment.

Better project management likely includes:

✅ The manager and team come to a consensus on optimal project management processes. Everything from project formation, publication of incremental deliverables, and check-in procedures are decisions driven by the team.

✅ Only productive meetings stay on the calendar. Meetings are stacked on designated days, protecting undisturbed development time for team members to get into deep work. The team feels the manager’s trust in them to utilize their time appropriately.

✅ Processes are clearly understood and baseline expectations create some level of standardization and uniformity — at least in the process — around the work (e.g. code review).


Taking this all into account leaves a new data leader like:

And how do you orient when the dots are constantly moving? What may be optimal in the first 90 days is almost certainly suboptimal between days 91 and 180.

While some equation for each new manager at each company likely exists, trialing-and-recalculating different orientations feels like a fitting strategy for someone who manages scientists.

Hypothesize, test, assess feedback, and re-calibrate.

All models are wrong. And those superimposed over people dynamics are usually really wrong in at least one way. But, they can still be useful tools. And for those audacious enough to accept the challenge of being an externally hired data science manager, well, they probably need all the help they can get.

Article originally posted here by Paige Dilmore. Reposted with permission.

ODSC Community

The Open Data Science community is passionate and diverse, and we always welcome contributions from data science professionals! All of the articles under this profile are from our community, with individual authors mentioned in the text itself.