Creating a Culture of Experimentation Within Your Organization

Experimentation culture is a hot topic these days, and for good reason! What is a culture of experimentation you might ask? In this case we’re talking about a scaled approach to online testing that is encouraged on an ongoing basis throughout an organization, with an established process and approach.

The likes of Pinterest, Spotify, and Visa are keen to foster a culture of experimentation because, in a world where customer experience is everything, a solid experimentation culture can and does make the difference between capturing audience loyalty, or losing it to a competitor. Successful and rapid innovations in customer experience are based on data-driven hypotheses and testing.  

Enterprise organizations are scaling their experimentation efforts because they have found that increasing the number of experiments done in a shorter amount of time means that the optimizations from the experiments end up benefiting the company and outweighing the risks. In other words, even a low success rate with a large volume of experiments diminishes the financial costs of the experiments that fail. This blog post will review how you can start building a culture of experimentation within your own organization.

Getting Started

Your marketing team may already conduct testing to optimize the customer experience, but to scale the program to deliver significant impact to topline revenue goals, leading digital companies are creating a culture of experimentation by building experimentation programs that involve contributions and skills from across the company.

Getting a culture of experimentation started requires  two things: Data and executive buy-in. It is crucial for executives to lead the way in supporting and promoting a culture of experimentation. In most cases you will need the data to get executive buy-in. Data will be integral in pitching your executives on an optimization program to help them understand the value and potential benefits. Data will allow you to talk objectively about expected results and avoid strategic decisions made from assumptions and opinions. It’s often helpful to stress the gaps in your current process. It may be assumed especially in enterprise organizations that your marketing team is performing optimization through testing already and that optimization efforts are the responsibility solely of the marketing team, but in order to scale your experimentation and start making a considerable impact on your high level business goals, you will need to increase the volume of your experiments and involve a more diverse set of skilled team members. When discussing the business case for experimentation, make sure you are armed with internal efficiency data to show executives the current state of optimization at your organization, as well as market data to tell a story about the potential conversion rate an optimization program could provide, competitive analysis, and ROI considerations. Experimentation culture really starts with executives embracing the unknown, encouraging employees across the company to contribute ideas and considering failures as data inputs and a natural part of the process.  

Building the Team

As mentioned above, it is often assumed that experimentation is the responsibility of the marketing team. Initially, that may be true, but to scale and increase the volume of your experiments, you may want to consider bringing in the following roles:

• Program manager; to manage the experimentation program, and make sure all experiments will impact the right business goals, and management of program KPIs
• Data analyst; to ensure experiments are formulated properly, to analyze the results of the test, and report on insights from multiple data sources
• Copywriter; to help test the best message to reduce friction and inspire the customer
• Developer; to help with any frontend code required for designs or functionality testing
• UX designer; to work with the team to optimize conversion paths

The more employees from around your organization are aware of the customer’s brand experience and how their roles affect it, the more they will feel ownership around improving it. With more of the organization involved, the deeper your overall understanding of the customer will be, and the more creative your experiments and testing will be. Pinterest found that the best way to scale their experiments was to create a data analyst task force to help employees formulate their ideas into testable hypotheses. That way they could ensure that all their experiments would produce valid test results.

Develop a Repeatable Process

In order to scale any kind of experimentation, you need to establish a process that is repeatable, consistent, and one in which anyone in the organization can get involved. 

Your process should include:

• Parameters around the kinds of testing your tools enable
• Roles and responsibilities on the experimentation team and how to submit an idea
• How to formulate a hypothesis
• How the test gets reviewed, critiqued, shared, and documented

The clearer the process is defined, the easier it will be for employees to get involved, and the more creative and diverse testing ideas you will produce as a company. Bookings.com provides an online form for employees to fill out with the experiment idea and information which is visible to the whole company to encourage not only contribution but collaboration.

Establish success metrics for your experimentation program that tie directly to business goals.

Some metrics to consider:

• Number of tests run 
• How many tests produce a positive effect?
• Time it takes to deploy a test
• Number of channels/products  tested
• Personalization program ROI

Documentation

As you may know, with any kind of change management and governance, documentation is key to ensuring that your process will succeed and scale. Consider creating documentation for the following information:

• What is your overall testing methodology?
• How do employees get involved and who is responsible for various parts of the testing process?
• How can employees easily submit test ideas? What were the results of past and ongoing tests? This can help people understand what’s been tested before.
• What is the schedule of tests? Consider creating a publicly viewable calendar with your test schedule so that tests won’t overlap.

Chances are your organization already has a central hub for information to add this documentation like an intranet, wiki, or even just Google docs. Your program manager should make sure that documentation is up to date and accurate.

Internal Marketing

Communicating the program’s goals, progress and outcomes across the company at all levels is essential to nurturing a culture of experimentation. 

When you reinforce outcomes within the context of your organization's commitment to delivering the best customer experience and showing the business impact of conversion optimization, you help the organization understand the value of the program. 

Be sure to celebrate the successful experiments and reward employees for contributions with social recognition. Some companies have gamified the process, creating leaderboards based on some of the KPIs you’ve defined in your process.

Rinse and Repeat!

It may make sense for your organization to leverage an expert agency to fill some of the roles needed to scale experimentation, or you may find interest and skills within your organization to build an internal task force. Whatever process works best for you, it's clear that the key to a successful program is leadership nurturing the company culture by allowing employees to experiment and considering surprises or failures as learning opportunities.

We hope you enjoyed reading the Phase2 blog! Please subscribe below for regular updates and industry insights.