How Products Get Built

2023 Edition

Chapter 1 - The Origin of Opportunity

Organizations exist to meet the needs of a specific customer or constituent. The extent to which an organization achieves this core objective is directly proportional to its financial or social impact.

The process of finding a match between the needs of a customer and what an organization can do to fill those needs is the process I am aim to outline here.

Quick note - this is a living document that will change and evolve over time.

Where Good Ideas Come From

Good ideas for products, solutions and services are typically uncovered by people out in the world - those intimately familiar with an existing domain or market segment, or sometimes those completely unfamiliar with a domain but that have seen a similar problem solved elsewhere.

The bottom line is that good ideas come from talking to people and hearing about the various pain points they encounter in their jobs or personal lives. Good ideas can generally be considered our natural human response to attempt to alleviate the pain points of others.

In a business context, we have many different sensors that alert us to new challenges that are not being solved.

1. Pattern matching
   1. This happens when team members traverse across different parts of the companies and are involved in a wide range of conversations. Patterns begin to appear in conversations over time and highlight potential opportunities.
2. Analytics we collect
   1. We collect analytics in many of our existing products and services, that when viewed in aggregate can show us patterns in the data that we normally might not have noticed.
3. Revenue - near-term and future
   1. Revenue is the ultimate metric. Customers spending money on products or services indicates that value is being delivered. Opportunity to scale delivery exponentially presents an opportunity for exponential revenue potential and also possibly market-disruption activity.
4. Signals from the market
   1. External market signals and trends that we track can also provide some clues about where specific segments of our market are heading and where we might look to uncover opportunity.
5. Customer feedback
   1. Direct feedback from customers is perhaps the clearest signal we can receive, based on the mutual trust inherent to the relationship. Pattern matching applied across multiple trusted customers is a powerful resource for detecting new opportunities.

We often rely on multiple sensors to make decisions. However, we layer the inputs to either confirm or refute biases we carry. Signals detected indicate opportunity, and multiple opportunities require a system of organization to support exploration and make great product decisions.

Chapter 2 - Making Good Decisions

All good product work starts with a problem. And where we have defined problems, we need a way to manage them. Leveraging an Innovation Pipeline framework makes sense here, specifically to identify the various activities we’ll need to perform, but also to outline the different artifacts that we’ll produce at the different stages of the process.

The Innovation Pipeline

An organization’s Innovation Pipeline functions like a funnel, always accepting input signal for new problems to solve. The function of the pipeline is to work help an organization validate problems through an evidence-based process that tests hypotheses and challenges assumptions. The output of a correctly functioning Innovation Pipeline should be vetted solutions to validated problems that solve pain points for real users out in the world.

Below is the How and the What of our lightweight internal Innovation Pipeline.

What’s not clear from the chart-based view of the pipeline is the map of connective tissue that connects the different phases - the decision points that influence how and when something moves from one phase to the next.

Intuition vs. Evidence

Making product decisions about what to prioritize and when is the core responsibility of the Product Manager. It is best thought of as a constant compromise between intuition and evidence. Intuition and evidence are inversely proportional viewed through the lens of the Innovation Pipeline, while funding is generally correlated strongly with evidence, as it should be.

• Note the inflection point during “Discover”

Good decision making becomes more critical the further an opportunity advances through the pipeline - from both a funding standpoint, and an opportunity cost standpoint. When teams are occupied supporting an activity, they’re naturally unable to support something else. So it follows that while we actively make decisions to commit time to an opportunity, we’re at the same time actively making a decision to not work on another. Every decision is a compromise.

Funding

While we don’t currently attribute specific funding to exploration occurring on the left side of the pipeline, there are costs associated with the activity necessary to perform curation and discovery around new opportunities.

Typical funding required to support a new product opportunity breaks down roughly as follows:

• Assumed $200/hr billable rate per person hour.

With an estimated cost of nearly $50,000 (with a 30% buffer) to support the promotion of a single product candidate through the Discovery phase, it becomes even more apparent that we need to make very good decisions about which efforts to pursue, and that we can only pursue a select few beyond the Curate phase.

Chapter 3 - How to Start

Detecting Signal

As previously discussed, signal can come from a single sensor, or multiple. Conversations with customers, data points from industry, analytics from existing products and services - all are valid resources at our disposal. While data and data sources are important, detecting signal is sometimes more of an art form than a scientific pursuit.

Once you believe you’ve found something interesting, it’s good practice to share a brief overview with a small trusted circle. At this point in the process, you are the owner and you alone make the call on how the opportunity takes shape. Design by committee doesn’t work - don’t cede your creative authority on the opportunity to others at this early stage.

Generating an Overview

The best way to solicit feedback is by generating a brief overview of the signal you’ve detected. A general rule of thumb is that this is a few sentences describing the nascent opportunity. A paragraph - no more, no less. Talk about the who, the what, the gap you see and how you’re thinking about exploring it.

The goal of the overview is to quickly and asynchronously begin to socialize the opportunity as a way to “sanity check” amongst peers. A quick thumbs up/down is the only feedback required at this point.

Hypothesis

Once the opportunity has been lightly vetted by a few trusted peers, it’s time to formulate a working hypothesis. Ideally, this is something that you’ll work to try to disprove in the stages that follow. It’s an important distinction to note that you’re actively working to try to disprove the hypothesis, vs. prove it out.

A good hypothesis will aim to satisfy the desirability, viability or feasibility of the opportunity through the eyes of a person affected by it. Of the three, desirability is the most critical to solve for immediately - if no one is affected by the challenge you’ve identified, then it’s simply not worth pursuing. These three attributes commonly overlap as shown below:

An example working hypothesis, aimed at solving for desirability:

There exists an opportunity to build on our successes around problem identification, and extend the thinking around that work to encapsulate the solution space, by engaging the defense-focused VC community. In doing this, we can create a system that begins to map government problems to known commercial solutions, and create a knowledge repository of this activity for future reference.

A Quick Note on Viability vs. Feasibility

While in the very early stages of building a hypothesis to test, Desirability is almost always the first lens through which a problem must be viewed. However, it’s also important to perform a quick scan of the landscape to assess the problem’s Viability and Feasibility.

In this context, Viability being the general human, process and political landscape that could potentially be affected by a solution to the problem. IE - Who would stand to gain or lose should this problem be solved? What parts of the org or market would be disrupted? Is there someone or something that can kill a potential solution before it sees the light of day?

Feasibility, for our purposes, is viewed almost exclusively through a physical or scientific lens. IE - For any range of potential solutions, are we talking time-travel here? Or does the requisite technology already exist and just requires modification? How has this problem been solved elsewhere for a different use case?

Problem Statement

A strong working hypothesis, specifically viewed through the DVF lens, helps us satisfy the minimal criteria necessary to begin structuring a sound Problem Statement. Tackling desirability first enables us to think about the first key to constructing a good Problem Statement - the Beneficiary, or the person who is affected by the problem the most directly. There are always multiple beneficiaries for any problem, but there can only be a single beneficiary for any given problem statement. This helps keep the problem statement scoped narrowly enough so that it can actually be solved.

Once a beneficiary is identified, we can turn our attention to their Basic Need - the thing that they’re missing (this is the potential opportunity we’ve previously identified). And for the final piece, the Desired Outcome, we can simply think about the inverse of their basic need and anticipate what might happen should that basic need be fulfilled.

Assembled together, a problem statement looks like this:

Defense Program Managers need a way to quickly map organizational requirements to validated industry solutions in order to identify the correct solution providers to engage and ultimately contract.

It’s also important to note that words are incredibly important in the construction of a problem statement. Looking at the statement above, we can pull out a few key words and expand further on their intended meaning:

Quickly -> doing it by hand or “secondary research” is not good enough

Validated -> companies can’t be majority owned by non-US entities

Engage -> part of the problem is awareness, in general

Contract -> indicates prior experience with gov and understanding of acquisition

A great problem statement should be shareable with and understandable by anyone. The goal is to present a clear articulation of the opportunity and to expand the ecosystem around the problem so that others can help you solve it. The more people you can talk to about the problem, the better chance you have to solve it.

Chapter 4 - Curation (aka - Talking to Humans)

Problem Curation is the process we use to further understand and scope opportunities. It’s designed to continually de-risk the problem and increase our body of evidence supporting our hypothesis, thereby increasing our confidence that we’re on the right track and at the same time, reducing our reliance on our own intuition as the sole driving force behind our motivation to explore the opportunity.

Beneficiary Discovery

Exploring a new opportunity begins with this simple act of picking up the phone and talking to people who are currently affected by the problem, yet it’s the one thing that most people refuse to do. Understanding the challenge through the eyes of the Beneficiary is the single most valuable activity when setting out to understand the scope and magnitude of any new opportunity.

The process of getting out of the building and talking to people is called Beneficiary Discovery, partly because we’re looking to understand their view of the challenge, but also because we’re constantly trying to understand whether they’re the right people to be talking to in the first place. It’s not uncommon to discover that your original beneficiary is not the correct beneficiary to solve for, and changing beneficiaries mid-stream should be seen as a valuable pivot, vs. a failure of intuition.

Beneficiary Discovery should be thought of as an investigation, and it’s important to keep an open mind and follow the investigation wherever it leads. Oftentimes, we find nuance in conversations with Beneficiaries that alters our view of their Basic Need, thus significantly changing how we view the problem statement. Sometimes this also affects the Desired Outcome, but not always. The point is, at this point in the process, we’re still very much in the exploratory phases of our efforts and need to pay extra attention to not allow our own biases to guide the investigation.

Long-term Goal

This is what the end state looks like - what happens years down the road and the problem is solved. It’s ok if it’s wrong, but it’s a stake in the ground and something you can communicate to others to demonstrate that you’ve given some thought to the vision of where the effort could go. It’s important not to confuse this with the Desired Outcome - the Long-term Goal typically forecasts much further out.

In the above example of our initial Problem Statement, our stated Desired Outcome is “identify the correct solution providers to engage and ultimately contract,” which indicates a desire to form new partnerships, with better partners, faster. This alleviates the current pain being felt by the beneficiary, but does not help the reader understand the larger-scale implications of what this could lead to. In this scenario, a Long-term Goal for this Problem Statement, could be something like:

“We aim to build a bridge between government and industry that transcends the current atmosphere of innovation theatre. We believe that by learning about the workflows of government requirements owners, and forming an understanding of the core attributes common across all industry solutions, we can create a self-sustaining system that matches requirements to potential validated and vetted solutions.”

Current Landscape

This paints a picture of the current atmosphere in which the problem is manifesting itself. It’s useful to describe the context around the problem to provide others a glimpse into the world of both the problem owner and the beneficiary. Describing the landscape also allows you to include other contextual details like operational constraints or other people that surround the problem, but in the proper context (instead of just listed out in a spreadsheet).

The least-worst way to organize this information at this stage in the process is to begin filling out a Business Model Canvas/Mission Model Canvas. While not an ideal format for sharing information with others, the BMC/MMC is a useful tool for tracking the core components of your findings over time.

Preparing for Discovery

The end of the Curation phase occurs when a critical mass of evidence has been obtained that indicates that the problem exists and there are a sufficient enough number of people being affected by the problem. Determining “sufficient” is sometimes tricky - a general rule of thumb, your initial pool of people to talk to should be between 10-100 - enough of a sample size to generate meaningful signal, but not too broad that the signal gets diluted.

Curation will also include an early indication of market opportunity, validated by either existing customers or potential future customers. For any new product opportunity, target market should be in the thousands, or tens or hundreds of thousands, but there are many factors that could influence the ideal market size, such as eventual offering price, for example.

While Curation is typically a one or two-person effort, spanning a few weeks, Discovery will require a team to execute properly.

Chapter 5 - The Process of Discovery

Assembling a Team

A good Discovery team is cross-disciplinary - this doesn’t mean you need specific skillsets necessarily, just enough diversity of thought to ensure the team will be viewing the problem from enough angles to provide true 360-degree Discovery effort.

A typical team makeup might look like this:

The Design Sprint

The best way to make rapid progress when starting the Discovery phase of a product endeavor is by running a Design Sprint.

• 2-4 days
• Team all bought-in and calendars blocked
• Aim is to validate assumptions and leave with an MVP and test plan

Building a Good MVP

• Should follow the three rules:
  • Be cheap
  • Be generative
  • Be fast
• Goal is rapidly prove or disprove assumptions about the solution
• MVP’s should be iterative in nature and not attempt to solve 100% of the beneficiary’s pain point out of the gate. Done well, our MVP iteration process will gradually ease our beneficiary’s pain over time with each iteration.

Solution Idea

• at this stage can come in many forms, as simple as a paper sketch to as complex as a web site mockup
• solutioneering and testing at stage is important because if we’re off-target and a pivot is required, this is where we want to make it
• pivots during Discovery are common and expected, and they’re relatively cheap to make here
• pivots that occur after Discovery are expensive and mostly happen due to oversights earlier in the process or external factors that we cannot control or anticipate

Business Model Canvas

• The BMC is a great tool for keeping your team honest about what it’s capturing during Discovery experiments and how the findings relate to the larger goal you’re looking to achieve

Making a case for larger investment

• Provided we have not been able to invalidate our assumptions, we need to use the evidence we’ve collected to build a case for a larger investment to take our MVP to the next level.
• This often requires a sales or pitch effort to key stakeholders and/or financiers.
• Once the product is funded and approved, it becomes the job of the product manager to help ensure that the initial MPV that gets built, is inline with the vision of the Discovery team and then to leverage data from continued Discovery to ensure that it achieves true product-market fit.

Chapter 6 - Product Management

The Incubation Phase

Product Management can best be thought of as the discipline required to shepherd products through the Incubation stage in an Innovation Pipeline. We’ve done the initial due diligence, proven our hypotheses and made the case for continued investment. Our goal in this phase is to continuously refine our working hypothesis and ensure that our teams’ focus remains on delivering tangible pain-relief for our beneficiaries.

If we do our job correctly, we will exit Incubation with a product that has mostly, if not completely, solved the acknowledged pain point outlined in our problem statement.

We begin to build our product iteratively, and as quickly as possible we should be aiming to put a minimally workable solution in the hands of users to begin easing the pain they’re feeling. In order to accomplish this, we need to level a development strategy that assumes we do not have all of the answers and complete blueprint for our product from the outset, and thankfully this exists in Agile Development Methodology.

The archetype for a product launch team:

• Product Manager The “owner” of the product vision, who holds the future state of the product in suspense. Their primary job is to ensure that the entire team’s focus and efforts remains centered on building a product that addresses the pain point(s) of the product’s users.
• Product Designer The co-pilot to the Product Manager. Responsible for translating the findings and insights gained throughout the research and discovery process are rapidly tested through design concepts, prior to being released to the team to actually be built.
• Product Marketing Manager (can also be PM) In smaller teams, the responsibilities can also fall on the Product Manager. The main function of the marketing manager is to constantly engage the product’s user base and nurture the infinite feedback loop between product team-product-user base.
• Agile Project Manager (can also be PM) Responsible for translating the product design and research into user stories and epics that can be further broken down and understood by the engineering team to be built. This also includes organizing and keeping track of work being done to ensure that the agreed-upon goals for the product development are being met.
• Engineering team (Lead + 2-4 engineers) The builders. With a maximum team size of five engineers, this core groups is responsible for building the puzzle pieces and arranging them into a gradually more-complete final puzzle.

Agile Development

With our evidence and investment in-hand, it’s time to start building. Here we continue to follow the mantra of building iteratively and chipping away at the final product piece by piece. While our discovery process aims to remove all doubt and decrease risk significantly prior to writing a line of code, it’s impossible to completely remove all risk. Our software development process needs to take this into account and act as a “damper” to absorb any bumps we encounter along the road.

The way we do this is by structuring our work into small cycles, or sprints, to ensure that the product we’re building remains in-line with both our vision and the expectations of our users. Agile development has been widely adopted as an acknowledgment that we cannot make 100% accurate predictions and it’s better to admit when we’re wrong early and often, and make adjustments, before we spend too much time, effort and money heading off in the wrong direction.

From Lean to Agile

• Translating from Lean to Agile
• Creating the feedback loop and practicing it
• Active and passive analytics

Agile Meets Design

• Managing design and engineering in the same process
• Design operating a sprint in advance
• Testing design and using a prototyping platform

Delivery

• Ship continuously, communicate iteratively
• Leverage continuous deployment to enforce building real things
• Communicate externally on an iterative cadence to engage, but not overwhelm audience

Launching to Market

• Like a good MVP in the Discovery process, any good product MVP should be generative and fully instrumented with passive and active analytics
• Analytics provide passive feedback about whether or not we’re hitting the mark, but also provide invaluable insight into new feature/product ideas
• Finding strategic partners to discover with together

The First Customers

• First customers are almost always strategic launch partners
• These customers will help us know if we’re on the right path before we commit to heading down the wrong path
• Important to listen to feedback and validate with others

Transition and Sustainment

• The long tail of continuing to iterate over time
• Taking into account market, economic, competitor and user behavior dynamics
• Be open to discovering how customers try to use the product in unintended ways