Unlocking the Power of Data-Driven Product Management: A Step-by-Step Quick Guide to Collecting and Using Data Effectively

In this blog post, we will discuss how to effectively gather data to answer our questions and make actionable decisions.

1. Define the question clearly

Before we begin collecting any data, it is crucial to have a clear understanding of the question we are trying to answer.

Before collecting any data, it's essential to have a clear understanding of the question we're trying to answer. The question should be specific, measurable, and relevant to the problem or opportunity we're trying to address.

It's important to note that having a clear understanding of the question is not only necessary for the data collection process, but also for the analysis and interpretation of the data.

When the question is not well defined, it becomes difficult to identify the appropriate data to collect and the appropriate methods to use. Additionally, a well-defined question helps to ensure that the data collected is relevant and useful for answering the question, and it allows for the data to be easily communicated to others.

Having a clear understanding of the question is the foundation of any data collection effort and it helps to ensure that the data we collect is meaningful and useful.

For example, "Where do new users come from?"

2. Define our hypotheses

Once we have a clear question in mind, it's important to also define our hypotheses. Formulating hypotheses allows us to make predictions about the possible outcomes of our research, and it sets a clear direction for our data collection efforts.

Hypotheses should be specific, testable, and based on existing knowledge and information. They provide a framework for understanding the data we collect and make it easier to identify patterns and relationships.

Formulating hypotheses also helps us to identify the necessary data to collect and the appropriate methods to use. It's an essential step in the data collection process, as it helps to ensure that the data we collect is relevant, meaningful, and useful in answering our research question.

In this example, our hypotheses could be "Influencers, Organic Search, Facebook Ads..."

3. Define Actionable Decisions

Defining the actions we'll take before moving to get data is crucial to making informed decisions. By identifying the specific actions we'll take, we can ensure that the data we collect is actionable and not just informational.

It's important to understand that the data we collect should be used to inform decisions, not just to provide insight. Before collecting any data, we should clearly define what actions we'll take if our hypotheses are proven true or false. This allows us to ensure that the data we collect is actionable and will lead to tangible improvements in our products.

For each hypothesis, we need to ask ourselves "What action can we take based on this information alone?". If the answer is "none", we should not waste time and resources getting data.

In this example, our action could be:

• If we discover that most of our users come from:
•  Influencers: we'll double the investment in Influence Marketing;
•  Organic Search: we'll double the investment in SEO;
•  Facebook Ads: we'll double the investment in Facebook Ads
•  Other sources: we'll discover more about those sources to understand how to leverage them.

Even if we have a clear action for the hypothesis, we also need to ask "Based on all the other things we already committed to doing, is it worth doing this?". If the answer is "no", we should stop wasting our time.

4. Utilizing Existing Data

Use available data first. Utilizing existing data can save time, resources, and potential user burden by avoiding unnecessary data collection.

Additionally, existing data can provide a starting point for our analysis, and it can also help to validate or disprove our hypotheses.

Keep in mind that the objective is to take action based on a data-informed decision, the objective isn't to collect data. Data bring confidence, but are actions that bring results.

In our example, we already know that 73% of new visitors come from Organic Searches. This information is enough to make a decision without needing to survey users, or it can also be used as a benchmark to compare the results of our new data collection efforts.

Furthermore, existing data can be obtained from various sources such as web analytics, internal databases, or public data sets. Also includes data that may have been collected previously within the organization but not analyzed or used yet. By using existing data, we can leverage the resources that have already been invested in data collection and gain insights that would otherwise be unavailable.

Utilizing existing data is a cost-effective and efficient way to gain insights and make decisions without needing to survey users.

5. Collecting New Data

Collecting new data may be necessary when existing data is not sufficient to answer our question or when we want to gather more detailed or specific information. In these cases, it's important to choose the appropriate method of data collection, considering factors such as cost, time, and user burden. Event instrumentation and surveys are two common methods of data collection, but there are other options such as experiments, interviews, and focus groups.

When collecting new data, it's important to be mindful of the cost and burden to the user, as this can affect the quality and representativeness of the data collected. For example, surveys can be frustrating for users if they are not well planned and can lead to low response rates, inaccurate data, and user dropouts. To mitigate this, it is important to ensure that the survey questions are clear, concise, and relevant to the research question and that the survey is as short as possible without sacrificing the necessary information.

Additionally, it is important to ensure that the data collection method is appropriate for the research question and the target population. Surveys, for example, may not be the best method for collecting data from a highly technical or specialized population. In such cases, other methods such as interviews or focus groups may be more appropriate.

Conclusion

As product managers, it's important to have a clear understanding of the question we're trying to answer, define our hypotheses, and know what actions we'll take based on the data we collect.

Before collecting new data, we should first try to answer our question using existing data. And when collecting new data, it's important to be mindful of the cost and burden to the user.

By following these guidelines, we can effectively collect data that informs actionable decisions and leads to a better product for our users.

Summary

1. Define the question clearly: Before we begin collecting any data, it is crucial to have a clear understanding of the question we are trying to answer.
2. Define our hypotheses: Once we have a clear question in mind, we should also define our hypotheses.
3. Define what actions we'll take: It's important to understand that the data we collect should inform actionable decisions. So, before collecting any data, we should define what actions we'll take if our hypotheses are true or false.
4. Use available data first, always: Before collecting new data, it's important to first try to answer our question using existing data. That information may be enough to make a decision without needing to collect data.
5. Collect new data, if needed: If we can't answer our question using existing data, we may need to collect new data through event instrumentation or surveys. However, it's important to be mindful of the cost and burden to the user when collecting new data.

Work smart. Keep growing 🚀