Product Analytics

Continuous optimization is needed to meet the expectations. This, in turn, depends on the correct interpretation of conclusions and recommendations based on current and historical effectiveness research. Quantitative analyses leading to these recommendations are generated by analysing data from metric tools at the product level.

How does product analytics work?

The analytics value for comprehensive product development comes from the combination of proven data work and behavioural analysis. Users’ interactions with a digital product, which is usually a website, store or application, are examined.

When assuming the role of a product analyst, you should focus not only on what users interact with and what causes their behaviour, but also how the digital product’s algorithms react to them and what the interface communicates with each behaviour exchange.

Research requires data, the collection of which is divided into metrics. They can be analysed to assess different areas of the digital product and solve specific usability or performance problems.

Data analysis has great potential, especially in products with a large volume of users. Thanks to it, you can verify development, marketing and business activities as well as identify phenomena and processes taking place in the application. Including those that require optimization.

What are the benefits of product analytics?

Thanks to the analysis reports, you can optimize the range of functions to choose the most used ones – reducing product development costs to prioritize and delivering value to users with the functions that are most natural to them to use.

In extreme cases, it may turn out that the product, which has been developed for years, is used differently than the product owner and the team of designers intended. Thus, the company wastes time, money and team energy on developing functions that most users will never reach, and certainly will not pay for. Product analytics allows you to discover, redirect investments and increase the competitiveness of a product on the market.

Another advantage is complementing the results of qualitative research and constantly providing challenges to UX researchers, who can confirm their conclusions and better select areas for optimization. Next, look at the observed phenomena more broadly and verify them with the entire population. The cycle of exchanging results between product analysts and UX specialists results in new questions, confirmation of obtained insights and evaluation of solutions.

The data also shows the effects of technical problems – for example, the maximum page load time that most users are able to withstand. Knowing the degree of impatience with particular problems, you can more efficiently determine the hierarchy of errors to be worked out.

How to get started with product analytics?

The analysis process begins with determining its direction.

The starting point could be an impulse from the product team that they need product knowledge, in other words, answers to questions. It is the business owner who initiates the analytical process.

At the next stage, asking a question should be analysed. What will we learn from the answer? Are we able to use the answer to improve the business? Are we able to choose metrics and methods of analysis?

How to work with A/B tests?

A/B testing is a popular method of acquiring knowledge. The work begins with the formulation of a hypothesis and designing a test verifying it. At a given time, some users are presented with the old version of the interface element or subpage, and some meet the new one. Participation in the test of the currently functioning version is very important as it is a benchmark for assessing the quality of the changes.

A/B tests can be efficiently designed in 5 steps:

• Choosing the target group. Depending on the hypothesis, it can be all users or just a group that frequently uses a given function.
• Determining the scope of the change and the number of versions. As a rule, these are two versions of the interface – including a new one with a corresponding hypothesis difference in the problematic area (dependent variable. The one that we can change by our actions, eg change the password, color, layout, etc.).
• The division of the target group into fixed subgroups: the test group (sees the version after changes, e.g. a different advertising slogan) and the control group (sees the version without changes).
• Designation of metrics:
  – on which we want to act (dependent variable). For example, conversion, average session length, number of subpages per session, number of clicks
  – which are not expected to be affected by the changes (control variable). For example, cart abandonment or page bounce rate. An apparent increase in a desired index may have unexpected consequences in the deterioration of another important metric.
• Closing the test in time. The period depends on the characteristics of use. For products that are used every day, the test can be closed in a dozen or so days. On the other hand, where users spend more time on weekends or in high season, it is necessary to wait for the test results for several months or longer.

What metrics to use?

The most popular metrics that should be considered in product analytics can be divided into 3 basic categories.

Measuring:

• financial results, e.g. MRR, iMRR, ROI, conversion,
• traffic, e.g. DAU (active daily), MAU (active monthly), number of new, active users
• commitment, e.g. number of sessions, duration; number of uses of a specific functionality, number of key actions per session

In addition, product performance data (and therefore technical data) – such as the loading time of subpages – are helpful.

The beginning of work with analytics should be preceded by the development of an analytical strategy, i.e. determining what metrics and for what purpose we measure, what are our business goals and how the progress in achieving these goals can be measured. What is the measure of our business success?

It is worth remembering in the context of the entire organization that data collection must be related to important business assumptions. Without it, even the richest resources will be of little use – information collected without purpose is of no value.

In order to work with data successfully, at least two people should be responsible for analytics – each for one area:

• business — in which research questions are asked, and then ensure the effective use of the received answers in the organization
• analytical – which is to answer these questions, turning data into practical knowledge and tips.

Additional support is also provided by programmers who, based on the analyst’s guidelines, create technical solutions needed to download data and save them in databases or measurement tools.

With this division, it is better to ensure that the analyst will solve the problems that are really pressing for product development, and that the financing of the obtained answers will pay off in the course of optimization.

Analyst soft skills

Soft skills are necessary to effectively translate the information extracted during the analysis into human language – and sometimes also to convince decision-makers of the importance of the discovered problems and the legitimacy of investing in their solution.

The analyst’s task is to complete the answers to all research questions and verify the hypotheses. If this is not possible, all omitted questions and hypotheses must be listed in the final report.

Final report

The presentation of the results is as important as the quality of the analysis itself. The result of the work is a report created in the most legible form. This is where digital tools help. From the traditional Excel/PowerPoint tandem, through Data Studio, to the more advanced Tableau software. The brightness of the prepared analyses has a large influence on the effectiveness of optimization.

Presentations with graphs and data visualizations, supplemented by author’s comments, are commonly used. It is easier to refer to such materials and formulate feedback. It is very important to be open to the discussions of the report recipients, because the results contained in the report, interpreted by different people in the organization, looking at the product more in terms of customer service or marketing, can stimulate ideas that will further complement the material and contribute to more effective optimization.

Thanks to this, product analytics reports can be used by many departments that support them, which use knowledge about users on a daily basis – such as customer service, sales or marketing.

It is equally important to properly archive the results of analyses and reports. Reports should be easily accessible to all stakeholders so that they can always be consulted.

Summary

Product analytics is an area that is especially worth implementing at the stage when the vision of the product is already stable, the goals set for it – clear, and the market strategy – coherent. In such a situation, qualitative research – often present from the very concept phase of the product – is extended to work with quantitative data

Regular analysis supports decision-making in the conditions of changes on the market, as well as natural changes in the product and its relations with users during development. It also helps detect rising risks and opportunities earlier. Then, the best tactical decisions (e.g. correction of the hierarchy of function importance) and strategic decisions (such as recognizing the potential of a new, significant target group) result from the combination of intuition and experience of managers and current, objective data about the product and users.