Product Operations — Turning data into action

Which skill sets are right for you and your organization?

As a growth stage company, you have found product-market fit and are now scaling. Congrats! Now, you need to assess your products on a regular and frequent basis, as you are considering different directions (e.g. new products, M&A, etc) to focus your resources for growth.

You may have heard of product operations as a business function for product insights, but just didn’t know where to start. No fear — the key is just starting! Below is a guide for baselining your product’s performance, which is the first step to product operations. Note that the activities shared below represent the most common pattern for understanding a product portfolio’s performance for a B2B SaaS company; there are a number of dimensions that you will want to assess but you can start here and add analysis as you see fit for your products.

You will need to expend a bit of effort to go through this exercise but the key is just starting the process — and the results are worth it!

Step #1 — Identify Your Total Costs

The first step in baselining your portfolio is getting an understanding of your research and portfolio development costs for the last three years, from your financial statements.

The high level costs should include two views of your total costs:

Total costs, in total and by product — These costs will include all of Product and Design, Development, Quality Assurance, and Operational Support and will include your labor and non-labor costs. These costs should include all of the activities required to operate the environment and any applications, tools, license fees for third-party components etc.
Total costs to maintain your current products and product operations, assuming no new customers. This cost is different in that it also includes any contractual obligations, such as custom effort.

When calculating these costs, ensure that you address accounting treatments (i.e. net of accruals, include capitalization).

Pro-Tip: A common mistake is only looking at build costs and not ones associated with maintenance.

Step #2 — True up costs against detailed data sources

Oftentimes, we see that the financial statements do not fully nor accurately provide the true costs of your products or provide a full breakdown by product. To get a real understanding, you will need to consider other data sources and ensure that all costs are accounted for through a bottoms-up review.

For non-labor costs:

Accounts Payable — Invoices are sometimes not mapped correctly or at all. To ensure that all non-labor costs are accounted for, review your vendor costs (e.g. third party applications). We recommend that you drill down to detailed invoices for complex costs, such as cloud hosting and services, as these costs may have been pre-negotiated, are hard to forecast, and require allocation across products (if you have more than one). Your billing system (e.g. Netsuite) can provide paid invoicing data. You may also want to consider reviewing contracts or data from your procurement system (e.g. Coupa) to get an understanding of the longer term contractual commitments.

For labor costs:

Contracted Labor — Depending on whether you have contracted labor and/or temporary employees, you will need to review your accounts payable or HR systems. Ensure that you capture all contracted labor costs, including those for formal development activities.
Census Data — To determine whether all labor costs are captured for your products, you will need to review your company census data by person, location, and role. Ensure that all team members are included in the costs, as we sometimes see team members from non-product teams working on, or re-allocated to other activities. Also make sure that your teams are allocated to the right products. Census data is usually captured in HR systems, such as Workday.
Workforce Allocation — Use workforce allocation data as another level of data to determine resource allocation to specific products. This data needs to be cross referenced with the census data to determine where effort is being expended in terms of product, customer, etc. There is a high likelihood that this data may need to be used as a swag and directional, as we have seen that processes and inputs are not always formalized. And it’s ok! The exercise creates a baseline and is a start to getting more or additional information to inform work allocation. It also informs what future iterations need to happen to make the processes and data more actionable. The source for this data is usually workflow management systems like Jira.

Once you have these costs, review and reconcile them against the total costs noted in the financial statements. The sums of this data should match the top-line financial summary data.

Step #3 — Understand other costs that may impact your products

As companies scale, other cost drivers may be masked or the magnitude may not be readily evident. To assess these drivers, you will need to look at the following data to assess scope and impact.

Defect Tickets — Defect tickets are a source of rich information. They reveal where and how much R&D is spending their time as well as potential issues with development processes. The source of this data is usually a workflow system, such as JIRA or ServiceNow.
Support Tickets — User and customer support tickets can reveal critical portfolio defects, enhancements needs, and other portfolio opportunities. The source of this data is usually a customer ticketing system, such as Zendesk. Be aware that not all opportunities may be captured in the tickets if your user support teams are not fully capturing this data and/or manually addressing them as part of their workflows.
Onboarding Process — To consider potential impacts to product, review the onboarding process and effort. Are there any efforts in this process? Are there any manual activities that can be automated, creating opportunities for efficiencies, faster time to market, better customer experience etc.? Depending on your organization, the data for onboarding may be captured in a CRM like Salesforce, or a workflow management tool, or a customer ticketing system.

Step #4 — Assess your growth drivers and opportunities

Once you have gotten all direct and indirect costs accounted for, you can assess the potential growth drivers for your products and business. This information is important for understanding what your customers are paying for, and where you could potentially grow, in terms of features and portfolio offerings.

Revenue and Bookings — To understand how your business is performing at the topline, get revenue and booking data by product. The primary focus will be recurring revenues (e.g. ARR, MRR) and type, though one-time costs such as implementation services should also be noted. This data can also be sourced from your financial statement and systems.
Pipeline and Win/Loss Reporting — Your pipeline and churn data will reveal potential revenue, customers, opportunities, and timing. You will need to get this data by product and understand the average duration of the sales cycle. Additionally, you’ll need to understand why customers may not be buying your products and what some of the drivers of churn are for your customers — and what stage pipeline churn is happening. This data is also usually sourced from a CRM like Salesforce.
Customer Engagement — Customer engagement data may be captured qualitatively and disparately. Depending on how your business is structured, you may have customer success teams, sales teams, etc. liaising with your customers to identify unmet needs. If you distribute an NPS survey, you will want the NPS as well as any customer engagement data to understand how your customers are viewing your products and potential opportunities.
User Engagement — Engagement data, such as feature and product usage, are great indicators of your products’ acceptance and resonance. At minimum, you will want to track monthly average users, log-ins, session times, user retention, and features used. User engagement data may be captured in applications such as Pendo and Amplitude.

Step #5 — Consider all data and turn it into action for your products

Now that you have your data, you can put it all together for a baseline view of how your products are performing. Some questions that you can ask and that may require an additional layer of analysis and research include:

Are there any costs that are surprising and are potential risks?
Are there any costs that can be redistributed to new products and features?
What are feature or portfolio opportunities that can be tested?
What are unmet customer needs that can be addressed with new products or services?
How can you better improve the customer experience, increasing engagement and retention?
What is your true cost of expanding your products’ footprint? How much of your workforce are you actually tying up to maintain existing portfolio vs creating innovation?
Are there new monetization opportunities from reviewing pricing & packaging or improving cross-sell between product lines?
How much of your revenue is at risk as longer-term contracts come up for renewals? Are there actions available to mitigate potential future churn increase?

With this baseline, you can make forecasts, develop a strategy, and create a plan. You can also measure how your plan is performing as you execute against it. With this, you are ready for the next step towards a product operations function — automating the data and centralizing various views of the data into a business intelligence or data visualization tool. You will want to track performance and changes to your baseline weekly, so that you can quickly pivot your teams to reach your products’ goals. Share with your team and discuss how you can start being a more metric-driven decision making organization.