Managing an optimisation programme is a complex task as it involves multi-disciplinary teams working together in sync. This raises the question of ownership of key processes. Ultimately, putting together a successful optimisation team depends on the level of maturity of the business and its budget. In this post, we’ll explore how our CRO Maturity model helps to define the processes you should put in place as well as who should should own them depending on the level of cultural adoption within your company.

Charlie Romeo Oscar's CRO Maturity Model

Charlie Romeo Oscar’s CRO Maturity Model

Optimisation Team structures

Starter

Companies that have no official optimisation programme may still have individuals using experimentation principles in their work. For example the email marketing team may do headline A/B tests or the acquisition team may experiment on copy. As such there is no team structure associated with this maturity level.

“It’s important to choose a team or individual who is able to educate others on the culture of experimentation.”

How to get to the next level?

Start by hiring an optimisation expert who will coordinate efforts within teams, share knowledge between them and start putting in place a testing programme. Since little to no formal testing will have happened on key digital properties (e.g. e-commerce site, app, mobile site), the first few tests will serve as a proof of concept to try and engage senior stakeholders and prove the value of optimisation. At this stage it’s important to choose a team or individual who is able to educate others on the culture of experimentation to avoid disappointment if uplifts are not there at the beginning.

Intermediary

Companies that are getting started with an official optimisation programme tend to start with a small team of one Optimisation Manager/Project Manager who is responsible for building a strategy and executing it. It’s rare to see dedicated analytical, technical or design resources at this stage so the optimisation programme runs at a slower pace and tests take a long time to be planned and built. Due to the lack of analytical resources, the roadmap isn’t as data-driven as it should be but the first few tests tend to focus on putting in place UI improvements and inform product feature decisions anyway.

“Having dedicated technical resources will significantly improve the speed at which you can deploy tests.”

How to get to the next level?

Depending on the focus of the company and what the bottlenecks have been so far, think about dedicating more resources to the optimisation programme. For example, having a full-time data analyst will improve the quality of the roadmap and will help to prioritise it, as well as improve the quality of insights gathered. Having dedicated technical resources will significantly improve the speed at which you can deploy tests, too.

Advanced

At this point, companies have built a team of optimisation experts comprising strategist(s), analyst(s), designer(s) and technical resource(s). The bottleneck doesn’t come only from internal processes anymore but potentially from traffic levels since the volume of tests is increasing. There is greater engagement with the rest of the organisation (for example through an optimisation newsletter). The team is focusing on being always more data-driven, and on getting testing requests from multiple sources with various areas of expertise. The pitfall to avoid at this stage is trying to test everything. Although this is ultimately the goal of most optimisation programmes, one team cannot be responsible for taking requests from all sources and expect to treat them with equal importance. Data is therefore critical to prioritise.

“The optimisation team can’t scale at the same rate as the drive to experiment.”

How to get to the next level?

The key thing to overcome is that the optimisation team can’t scale at the same rate as the drive to experiment, therefore the structure has to change. Until now, the optimisation team has been mostly sitting with product or with marketing. To become a world-class optimiser, the optimisation team has to become a support function. Product and engineering teams who are focused on specific scopes (for example search results, product pages, purchasing funnel, landing pages, etc) become the owners of their own continuous optimisation. The centralised optimisation team becomes the provider of internal strategic services, project management, and maintenance of tools.

World Class

Companies who have made experimentation a core part of their culture show that they have implemented these processes directly into their product development and marketing cycles. For example, Spotify, Google, AirBnb, and more…

“There is always room for improvement.”

How to continue to improve?

Although the framework for continuous optimisation is now in place, there is always room for improvement. Teams should focus on improving knowledge management (especially within large testing organisations), the implementation of automation/machine learning, and the development of better tools in order to overcome optimisation bottlenecks, to name a few.

Why not simply copy the world-class model straight away?

As good as continuous optimisation is, it requires a huge cultural shift if the company is not built on these principles already. Furthermore, it requires a large ongoing investment, which is risky if made at an early stage since the critical underlying processes will not be in place yet. Therefore this is why the maturity model helps to understand the gradual steps to take on the journey to world-class optimisation.

Outsourcing vs in-house

I believe that in order to be successful, the optimisation programme cannot be fully outsourced. However, a hybrid model where an agency or external consultant provides support has multiple advantages:

  • Outsourcing parts of the optimisation process increases the company’s maturity level in a aim to get better results faster and put the right processes in place internally in the long term
  • Retaining internal ownership of the programme means that learnings are still kept and subject matter experts can still have an input into the roadmap

 

Sign up to the CRO Newsletter to stay up to date