Data Scientists (DS) often have a broad range of responsibilities and specializations, but in a word, their main value is insights. These data insights can come from added or improved data logging, data engineering AKA extract, transform, and load (ETL) pipeline work, success metrics definitions & tracking via dashboards, experimentation design and analysis or other causal inference techniques, and often machine learning (ML) optimizations or prediction (see analytics hierarchy of needs). Some data scientists develop stronger technical skill sets which can lead to more engineering type roles such as machine learning or data engineer. Others focus on a specific area like product, risk, or marketing/GTM and might have traits in common with product managers or ux researchers via their domain expertise. Regardless of primary focus, data scientists should progress in their strategic thinking, influence and collaboration, and business/product acumen over time as well as developing strong working relationships. One of the ways to measure career progression over time as a data scientist in terms of impact and expertise is through individual contributor (IC) leveling (example Square article on engineering level principles). Below is an overview of a general path from entry level to mid-level to senior to staff.
Before we dive into specific levels and what a typical career progression might look like, one thing to note is that typically a data scientist is expected to perform at the next level for 6 to 12 months consistently before being promoted in most cases. This is one area that people often miss in their expectations: it’s not just that you CAN perform at a given level, it’s that you can consistently be successful in the key areas of that level – reliability matters. Meanwhile, external candidates are often expected to show they are performing at the level they are hired for within 3 to 6 months of onboarding. Let’s dive in!
Junior DS (Entry Level)
Junior DS are new team members with potential. They have entry level expectations (with some academic background such as an undergraduate degree in a technical field like math, computer science, or economics), and sometimes start as interns. They are expected to collaborate well with the more senior members of the team (often being assigned a project, working on it, then explaining what they did and why) and focus on learning and impact in a non-academic setting. Generally their projects have lower ambiguity and include more supervision/review as they learn best practices. It is very important they are open to feedback and communicate/collaborate well (see my soft skill debugging blog post). Typically this level is for the first few years of work after an undergraduate degree.
Junior to Mid-Level DS Transition
The progression from junior to mid-level typically involves a deeper understanding of the basic toolset and learning how to make an impact. A big misstep can be pretending to know something you don’t know, or trying to hide your mistakes. Everyone makes mistakes, but unforced errors (like spinning your wheels for hours when you could just ask for help) or technical gaps (e.g. not knowing how to efficiently query a table in SQL) are common barriers. The emphasis to progress here is on fixing weaknesses and learning to apply best practices.
Example project (highlighting the shift in scope): Completing a full customer analysis as part of a product team and communicating takeaways, delivering impact – with some directional guidance and review from a more senior DS.
Mid-level DS
Mid-level DS are solid individual contributors. They are able to take on more well defined projects and scope with some direction from more senior DS. The goal here is to develop the ability to handle more complex problems and get more meaningful work done (often starting ‘on-call’ responsibilities as well). Typically this requires a few years of working experience in data (with a 4 year undergraduate degree) or a masters degree in a relevant field (DS, statistics, computer science, etc). They are reliable, can collaborate well with peers/functional stakeholders, and follow best practices (clean code, etc) with growing domain knowledge.
Mid-level to Senior DS Transition
The progression from mid-level to senior typically involves the ability to function more independently, owning complex or more ambiguous projects from end to end. A common mistake I’ve seen is not knowing how to work well with and learn from others. Imagine being a manager and having a direct report go off into a cave to work, and you don’t really know what they are actually doing. It’s about building trust that you will surface important details and ask for help when stuck, but also not ‘reinvent the wheel’ every time something comes up. This allows you to account for your weaknesses, and start having a stronger impact from your strengths. A senior DS is also typically a domain expert in an area of the business, and starts to really master the work they are doing. This transition is also where it’s expected that a DS deepens their technical expertise, such as understanding alternative causal inference techniques to AB testing (e.g. variance reduction techniques), exploring more ML algorithms, learning how to build out an ETL data pipeline end to end from scratch, or collaborate with broader stakeholders on a multi-team product launch.
Example project: Working independently on a product launch with the product team (including logging, experiment design, impact analysis, etc), while maybe leveraging a technique you researched to handle a difficult technical hurdle.
Senior DS
Senior DS are strong independent contributors. They effectively operate with autonomy and have built trust within their team as the leading expert in some areas of the business. They know the ins & outs of the role, can cover for peers fairly easily, scope out their own projects (a prerequisite of which is to know when a project is actually finished), and are able to adapt as needed with solid impact even when facing ambiguity. They are seen as a domain expert to their peers, a data expert to their functional stakeholders, and are more often called to lead projects and mentor some junior folks. Senior DS typically have at least 5 years of experience with a relevant undergraduate degree or 2-3 years with a masters (or a PhD).
Note: I’m using years of experience/formal education here as a proxy for trust and skill. A question to ask: what would you need to trust someone to operate independently as a data scientist? Or more specifically, trust them to work for a week on a project without any help / direction? It depends on the specific role and responsibilities. Illustrative of this trust in communication, a senior DS will typically gloss over some background details (because stakeholders trust it was done correctly and purposefully) and focus more on telling a story of why what they found matters and next step recommendations. Sometimes at this level a DS will begin to think about transitioning to a leadership or tech lead type role – depending on their goals.
Senior to Staff DS Transition
The progression from senior to staff usually involves proactively identifying and solving critical problems with impact beyond your immediate team (including mentoring others, sharing work more broadly, etc). It involves stepping forward and leading the team in areas of need. Being the go-to person in some domain can help progression here as well, it helps to be an expert in some area, whether it’s experimentation/causal inference, ML algorithms, product strategy, or just effective storytelling in key business areas. The major mistake folks make here is not being proactive about their career, and not taking principled risks. You have to develop a track record of sticking your neck out and having it pay off. Execution at this level often requires cross functional collaboration and projects that reach a size where they can no longer be completed alone, regardless of individual skills. It often involves deeper research around the latest and best techniques to solve complex technical problems. Scoping and leading ambiguous projects is not for everyone, some folks might not want to take on this level of responsibility (or might pivot at this point to management, etc).
Staff is also often where your broader reputation and cross-functional collaboration starts to surface more as part of the role. An organizational leader cannot nominate you to lead an important project if they don’t know who you are. It’s also important to find opportunities for impact for yourself, which often requires an awareness of what’s important to the business, identifying gaps in understanding or foundational work (a product of decisions and organizations), developing an intuition of what might pay off, and leveraging trust to influence things and drive things forward (article on progressive influence).
Example project: Driving a new analytics tool adoption of a key data platform initiative, like getting Amplitude set up for your team and leading training or helping to drive a data platform forward with one’s specific domain (e.g. creating valuable new data structures for downstream stakeholders to leverage in models, analysis, etc). Keep in mind impact and adoption matters here. An interesting tool or dataset no one uses doesn’t add value to the team. Another project could be successfully driving a multi-quarter cross-functional product change with broad stakeholders in a niche business space that had a significant positive impact or leveraging a new causal inference technique and training your team on it.
Staff DS
Staff DS are team and discipline leaders. They are able to tackle large, ambiguous and complex projects proactively and work well with multiple functional stakeholders with strong communication and storytelling. They typically have 8 years of experience with an undergraduate degree, 6 with a masters, or 3 with a PhD. They handle the tasks that senior DS (i.e. strong ICs) cannot handle, often because they involve a much broader scope and overlap with areas where they are not the experts. At staff levels and beyond you often have to start finding your own opportunities and know where you add unique value. This is also the level at which roles start to split out more into specialties, with a common one being: researcher and storyteller. Side note: rate of progression can vary depending on scope and experience at this point – as career progression is less straight-forward at this level compared to early career (i.e. it depends more on the business need/environment).
A note on some common staff DS archetypes: Storyteller vs Researcher. A researcher DS has very strong technical skills (e.g. in ML, causal inference, or data engineering) and adds a lot of value solving complex technical problems. A storyteller DS is effective at impacting decision-making and creating structure in ambiguous problem spaces (article on what great data analysts do) – e.g. they might have some ability to align work with executives goals, etc. Both archetypes have to be able to teach and help uplevel others. Ultimately though, regardless of toolset or specialty, impact comes down to customer value creation, does their work improve the business?
Caveat
I am a staff data scientist currently (with ~11 years of experience depending on how you count it), so everything after this is just my own observations and not actual experience. Not everyone is going to want to move beyond a ‘team leader’ and go into a grayer, sometimes riskier, area of their career of ‘organization leader’ – similar to how not everyone is going to (or wants to be) an executive at a company (i.e. both often involve a lot of networking). These roles can somewhat become a more ‘political’ leadership type role (e.g. getting an org to become more scientific/experimentation minded is a very different and specific task than a typical IC career path). These types of DS levels also depend more on the specific industry and size of the company. For example, a tech company with less than 1-5K employees might not even have a senior staff type role, unless it’s a more data science oriented company. Anyways, let’s continue (or feel free to stop reading here)!
Staff to Senior Staff DS Transition
The progression from staff to senior staff generally involves a broader scope of impact. In fact I’d argue the rest of the levels at this point are about scope of impact: staff is team expert, senior staff is organization expert, principal is company expert, and senior principal/distinguished/fellow is industry expert. Though they are not management, these are leadership roles, and a common theme is the ability to build trust with key stakeholders and sponsors. Companies are teams of teams (with ‘org’ being the general term for a mid level between team and company – maybe including a few hundred people). At this level you start thinking about teams as a concept: are the org’s teams functioning well? What are their strengths and weaknesses? A staff DS does this for their team and a senior staff DS does this for the org (at least for a particular lens like ML). Only a very data science oriented company might have someone at this level focus on a more narrow space than an entire org.
A common mistake throughout all the staff+ levels is not looking for broader opportunities and keeping a pulse on the industry as a whole. No one will assign someone senior staff work, and you won’t reach it by doing more staff / team level work. Not everyone will make it to this level, as it requires a degree of business need and sponsorship (not just mentorship) – someone in leadership has to vouch for you. Can you lead a key project and make the whole org better? That being said, an organization doesn’t need 5 people writing standards, which is why there are fewer senior staff roles.
Example project: Finding solutions to major problems such as creating a core data pipeline infrastructure or ML foundational solution that is leveraged by other teams to deliver impact consistently – either by driving a migration or creating the solution.
Senior Staff DS
Senior Staff DS are organizational experts, typically working proactively on complex, ambiguous, and often riskier longer-term business solutions. Another way senior staff DS can add value is to give their talented teams a path forward sometimes by creating a multi-year progression/plan (i.e. like being a ‘professor’ at the company). No matter how smart an IC is in terms of execution, at a certain point they can start to feel lost when things get too broad and require a clearer vision/ direction. Particularly entrepreneurial DS who may have started their own companies in the past might do well at identifying org+ opportunities. Even so, they need clear boundaries for themselves. You cannot have a senior staff DS creating plans for the org AND doing a bunch of complex staff IC work in an area, they have to let some things go to focus on bigger picture tasks (in some ways making this level riskier and situation dependent) – a strong support team is needed for them to be successful. Typically this role requires 12+ years of experience with an undergraduate degree, 8+ with masters degree, or 5+ a PhD. Granted, the key here is sponsorship to major org projects and excellent delivery. At a certain point, you have to create your own role, and getting promoted is often after the contributions of a few major projects are recognized/rewarded.
Senior Staff to Principal DS Transition
This is where we move from org (e.g. a few hundred people) to company (e.g. a few thousand people) expert – depending on the company. Principal DS establish industry best practices and proactively prevent and fix company-wide issues, as well as taking ownership for a portion of the business. Maybe the data science team needs someone with a strong academic background who can stay ahead of industry trends (like GenAI or cloud computing, etc). Part of this role might be networking and knowing people in the industry or researchers. Internally, this DS level requires trust and sponsorship (i.e. political capital), but also drive and desire. Do you want to lead the entire company in a field like ML or data infra? Meaning going to conferences, recruiting, and upleveling the company’s expertise in a field? It can be very entrepreneurial to create a new platform within a company for example, but some people might rather run their own company at that point, or not have this degree of responsibility, regardless of the matching compensation. This is also the level where someone might have a job title created for them. Because they are going to own / build something out for the company and often need a team to support. Sometimes people would become people managers to some degree at this level just because it’s so hard to do this on your own. You cannot rely on a strong IC to make that vision happen, you need someone to figure out stakeholder needs AND know the technical ins and outs AND influence and work well with others. It takes time, which costs money, so it requires trust AND investment.
Example project: Evangelizing and building an experimentation culture or production ML standards at a medium to large-sized company, including platform migrations, training, adopting best practice (e.g. bringing in a new toolset for the company to use or conducting a year+ long migration). For a more specific example from Airbnb, building the open-source tool Airflow. My guess is after that launch, the IC (Maxime Beauchemin) who created it most of that became known as ‘the airflow person.’
Principal DS
This is getting into some fuzzy entrepreneurial ambiguous territory, as the principal role is specific to the company, but this is a company expert (i.e. setting the pace and ensuring industry best practices). They set standards for disciplines and organizations are are responsible for a portion of business goals, longer term strategy, and DS community principles and values. This person probably is very strong at recruiting high-level talent and working on cutting edge organizational or technological problems. A career analogy from a former software engineer colleague on building homes: junior level builds bricks, mid level builds walls, senior level builds rooms, staff builds houses, senior staff builds neighborhoods, principal builds cities, distinguished builds countries, etc. You could argue the path continues upwards, but at a certain point, levels become unnecessary since there is no comparison framework. What level is Andrew Ng or Nate Silver? I dunno, but probably not senior staff.
Principal to Distinguished DS Transition
This is the transition from company to industry expert, so we get into a weird territory of ‘what’s an industry?’ I’d argue creating a new coding language or technical system is an example of this. Suppose you created some new snake-based language called Cobra (‘move over Python, here’s Cobra!’). Imagine being the CEO of a company and trying to build out a platform in cobra and being like ‘hey, let’s hire the person who created it?! They’ll bring in the best coders and make us leaders in space! What a competitive advantage!’ But it’ll cost. That person has to probably turn down a lot of companies wanting the same thing, and they might need a job title created for them. Leveling is often not transferable across different fields, especially at this point (see levels.fyi for rough comparisons).
Example project (highlighting the shift): Creating a widely adopted language (e.g. R) or python package (e.g. scikit-learn) that is adopted beyond just the company. The person who created R would likely be known as ‘the R person’ for their career, and the impact shifted how the industry conducted DS work. Maybe you could argue my earlier example of Airflow would qualify – but mainly because its open sources (not just internal).
Distinguished/Fellow DS
How do you measure someone at this level? Industry leader/expert. Do they have to be an executive? A major criteria here is not to measure potential, but also reliability. It’s a major investment for a company to bring in someone at this level. Many leaders will need to be involved in the decision (hire or promo), especially since this person would be expected to mentor principal DS (and therefore need to command some respect/political capital). A distinguished engineer who creates principal DS capabilities or other distinguished engineers is very valuable indeed. That being said, retaining them might be difficult: your problems have to be interesting and complex enough – and if they are truly industry experts, you might be afraid of losing them (e.g. imagine if early creators Google left and formed a competitor early on – I imagine OpenAI is thinking of who to promote to this level who was involved in ChatGPTs creation). Granted, as an industry, you don’t necessarily need everyone to be eccentric geniuses creating industries all the time. You also need people who help to make sure the trains run on time.
Wrap Up
Getting to any of these DS levels require some degree of convincing the people who want to hire you that you can do the job. At early levels it’s more via negativa (the study of what not to do), i.e. screening out people who would struggle with the basic responsibilities and expectations. At the middle levels it’s more about reliability and showing consistent impact. At the higher levels, it’s more about influence and entrepreneurial risk taking. All are anchored on the same principles of cost-benefit, and a subjective evaluation (e.g. executive approvals) that can include some objective inputs (i.e. past achievements and role fit).
Sports make a great analogy here. Early in learning a sport it’s about understanding the rules and fundamentals (and yourself!), then it’s about building skills and practice. Later on, for example in the NBA, recruiting becomes about risk and bets (e.g. will Kevin Durant play well with the Warriors?). Data science careers are more complex than sports careers in some ways – since impact can vary by domain/technical specialty and things are often determined by the market/business need (or signaling prestige, like a PhD) – but in other ways they can progress similarly – like picking a sport to specialize in or stay an athletic generalist (see article on T-shaped people). Ultimately it depends on the economic/industry environment/your own needs and choices as to where/how you progress in your career past a certain point.
Daeus Jorento 