In today’s world of MVPs, POCs and scrambling to hit OKRs, it’s not uncommon for data teams to feel whipsawed. The middle tier of the organization is certainly no place for the faint of heart. As a leader within a data organization, what can you do to stabilize life for your engineers as they’re confronted with some kind of “iteration” or “pivot” every few quarters?

One tool that I might suggest is a written team vision that is both specific and timeless. You have to make it specific because none of your engineers want trite platitudes. And ought to aim for timelessness so that it requires nearly no revision, regardless of how the business itself may evolve.

Such a vision statement can contribute to each team member’s narrative about why they are happy to continue to labor with your group.

What follows is such a document prepared for a data team in 2021…

Team Mission & Vision

Imagine if our data team could step into a time machine, fast forward four years into the future… What discoveries would await us?

#1 Retention

The first thing that impresses us is retention. All of the team members who joined four years ago are still here. The energy among us is high, stemming from both the delight of working on meaningful projects and our deep camaraderie.

#2 Superpowers for Other Teams

In our (virtual) office, non-data-engineers are empowered by the tools and relationships provided by our data engineering team. This has boosted retention as employees can perform at an unprecedented level. We aim to help them become “badass” in their roles, as Kathy Sierra would say.

We don’t require stakeholders to learn our tools. (“You should just learn SQL!“) Instead, we help them level-up their work in a non-intrusive fashion.

#3 Know Your Data/Business/Industry

Thirdly, we find that our team is highly knowledgeable. Our team knows the following extraordinarily well:

• Our industry
• Our business
• Our customers
• Adjacent teams’ delights and pain points
• Our data
• Our systems
• Our processes

Our knowledge allows us to move among various teams using accurate language and context-specific jargon to discuss problems and convey our solutions. Other teams would tell you this… more than any other engineering team they’ve worked with, the vast majority of the data engineers really “get it.” Our intra-team knowledge sharing is intense, and we shepherd the noobs toward deep knowledge.

#4 Happy Accidents

It’s not uncommon to find that the next set of “challenges” (as the business imagines it) are not as difficult as anticipated. Our team’s forethought often gives us an edge as we work from one problem to the next.

• We don’t try to future-proof every element of our work.
• We use our deep knowledge to sow the seeds of serendipity.
• We anticipate the future and address all kinds of risks without bogging down our stakeholders.

#5 Project, Task & Contributor Selection

In the future, we find that we’ve appeared to have “good luck” in picking projects, tools, partners and new team members.

For example, we’ve avoided building an oddball sprawl of NIH-fueled tools and dead-end integrations. We’ve picked good tools and projects projects. We have recognized that these decisions (tool/project/task selection) are one of the biggest force multipliers under our control.

The areas where we choose to “double down” are guided by our deep knowledge. (see point #3 above)

We make the right tradeoffs. It’s not that we get every decision right on the first try, but we remember the most important aspect of “architecture” which is (to cite Martin Fowler) to mind those things that are hard to change. It’s unavoidable that we will make decisions and need to backpedal at some point. However, we have been successful at avoiding “impossible to change” situations. We haven’t been bitten in the butt. As much as we may be fans of a particular tool, our deeper motivators tie to #2 above… the ability to confer superpowers to other people who may not use a tool directly.

#6 Data Asset Orientation

We find that the whole organization refers to its work through the lens of data asset curation and data product development. This is a mark of our influence.

Our team has created a platform for innovation such that assets can be dealt with as assets, building in the proper protections against rot and supplying the proper nutrients for growth. We have established a platform such that new data products can keep pace with the evolving business model and market dynamics.

Our team does not operate with a mindset that there is just one monolithic narrative and product that we are trying to support. Instead, we work towards a development lifecycle that supports innovation both at the micro and macro levels. There is a well-worn path from a researcher’s moment of insight to publication as a standard data asset.

#7 Risk-Centric Measures of Progress

We think of progress in terms of burning down risk. While we cheerfully check boxes in our project management tools, the bigger point is that uncertainty diminishes.

Risk often presents itself in the phrases we use to describe what is not known…

• ”We don’t know how hard it will be to ____"
• "We don’t know how useful it will be to ____"
• "We don’t know how long it will take to ____"
• "We don’t know if our users will embrace ____“

The essence of progress is often that moment at which “we don’t know” becomes “now we know.” What this means is that sometimes we get “scrappy” not because we don’t care about “doing things right” but instead we’re sequencing our activities to try to burn down the overall risk as quickly as possible.

Some problems, if left unsolved will be lethal to our goals & initiatives. We want to identify those threats most likely to be lethal and disarm them. The specific risks to our current projects are written down. Inevitably, we cannot burn down every risk, but there’s a clear paper trail outlining the potential pitfalls we tried to avoid and those that we just had to live with.

#8 Heck Yeah

We are decisive. We reject “yes.” It’s either “heck yeah!” or no. (thanks to Derek Sivers)

#9 Show Your Work

Our work is well documented. We always “show our work” so that other teams know how to reproduce our work easily. This means that whenever we see “results” of any kind, there’s an easily discoverable link to the query or code that will allow a person (with the right permissions) to get those same results. This is very useful because it answers questions about origin and it allows people to easily make derivative works that are truly comparable. This mindset serves as an antidote to the common proliferation of black boxes large and small.

#10 Multi-Faceted Feedback Loops

Last, our feedback mechanisms are strong. We see the health of the feedback loop as the primary leading indicator of success (see my other post). Some of the evidence that our feedback mechanisms are working include…

• Folks outside our team can comfortably and accurately articulate the capabilities of our team and tools. They know the good and the bad… what is truly working well and what sucks. They know what projects at any time are going well and what ones are going sideways.
• Our team has a high level of self-awareness as well. They play to their strengths and fortify their weaknesses. They seek human feedback and provide visibility into the efficacy of any of the systems they support.
• We are always happy to field an AMA. We don’t shy away from dumb, hard or awkward questions.
• But we don’t just react to questions; we proactively offer the right level of communication using the right language and level of detail for the audience at hand. This doesn’t mean that we have too many meetings. Instead, we have refined our approach and meetings tend to have a strong signal-to-noise ratio and more often than not lead to direct action.

That’s It?

You might read each of the 10 points above and feel like it’s missing a lot of specifics about what we intend to build. Where is the description of the cutting-edge data engineering tools, systems and infrastructure?

Like it or not, specific tooling has been purposefully omitted. If you look at the big (multi-year) picture, our place in the organization and our mode of working is more prominent than the concrete means of achieving that.

We expect that in the future when a prospective team member asks an existing team member what they like about working at our organization, you get answers like, “Other people see me as a badass because I have helped them become a badass” more frequently than they talk about tools. We expect that the tools will change, but the way we work will not.

What do you think? Email me your ideas!