Should You Test Prices Online?

by Linda Bustos

There is a pricing sweet spot to maximize profit (not necessarily conversion rate) for each of your products. Unfortunately, without testing methods, it’s very difficult to determine how to optimally price your products online.

Is A/B or multivariate testing the answer?

But split testing live with real customers carries risks. It’s not illegal, but customers who access your site from multiple devices or who clear cookies regularly may spot your inconsistencies. It may also mess with your paid search, shopping engine or email ads that contain prices.

To answer the question “should you test prices online?” I enlisted the Web’s top thinkers in testing to weigh in with their opinions, with a few of my own:

Just say no

Carlos del Rio, Director of Conversion Analysis & Digital Strategy with Unbounce :

The short answer is yes. But, there is a major caveat. Don’t do public testing. There are many places where it is illegal to arbitrarily vary your price, and any specific segmentation of that group can lead to other ethical/legal issues. Price testing should be handled like market research. Choose a controlled group and run them through the test scenario and offer them all the same end compensation for their time.

Justin Rondeau, Editor & Evangelist of WhichTestWon.com :

I’ve never really liked the idea of price testing, and each day we see negative reactions to companies who have conducted price tests via different segmentations, e.g., device used, geolocation, etc.

Prices will undoubtedly impact conversions and unless you are selling luxury goods the best deal will likely win out. However, the second it gets out that you are profiling users based on their device (like Orbitz did) you will lose a lot of credibility with prospective customers.

In my opinion, marketers should keep price testing out of their A/B and MVT tests. There are just so many other ways to optimize a site that will produce solid lifts without sacrificing integrity.

Do it, but get the timing right

Jon Powell, Senior Manager, Research and Strategy at MECLABS (Marketing Experiments / Marketing Sherpa )

Yes. The real question is “when is the best time to test price?”

If you already have established pricing, then I would avoid testing price first. Price is a key component in the perceived value exchange. Customers are weighing that number against how well they understand the value I am communicating to them. I should make sure I’m not dropping the ball on that before I start cutting into my company’s revenue potential (or taking the greater risk in trying to increase it).

I would first design a test sequence to determine if the presentation of the product(s) can be optimized to achieve an increase in my key performance indicator. I would also design a sequence of experiments to understand a channel’s impact on buyer behavior here as well before concluding that prices need to change.

What is the best approach to do so?

Once one gets to the point of testing price, the best approach starts with creating price points based on their projected impact to a greater business metric/need, like total revenue, new customer acquisition, etc. No matter which metric you choose, an optimal price point will be one that has a predictably positive financial impact on the business, be it short term or long term.

In other words, you’re going to have to do some math before you start your test. The only situation you will want to test a price that doesn’t have some sort of positive effect on business metrics is when you are simply testing to determine if you need to find a cheaper way to produce and make available your product (sometimes the majority of end users just won’t pay beyond a certain threshold).

Once you determine the metric you are trying to impact and why, you can use an approximation model to calculate suggested prices to test. If traffic allows, you will want to test a range of prices, and not just cheaper. Sometimes end users expect a higher price to completely buy-in to the perceived value of the product, and sometimes that higher price point will be responsible for bringing in an optimal amount of revenue, despite a reduction in orders.

Finally – in your test set up, do not forget to set up the additional tracking and auditing so that what you see can be verified and audited. Often times in a test, we make unintentional discoveries as a result of the additional detail, or in this case, the purchase data collected.

Do it, but stay in control

Chris Goward, Co-founder and CEO of WiderFunnel :

Price testing can be very revealing for e-commerce. The traditional economic model that says lower prices always increase demand doesn’t hold up in some cases.

Retailers are familiar with the effect the “95″ cent ending price has. It can communicate a discount and create more demand than other five cent price movements. Recent research also shows price size, boldness, color, sound and cultural considerations can boost sales.

Can you really raise your prices *and* increase your sales? In some cases, yes!

To test price, you have to make sure your testing tool is keeping a valid control and maintaining consistency across visitor sessions. You don’t want people in different sessions to see the price change. (The new Google Analytics Experiments wouldn’t be a good tool to use, for example.)

For price testing you should also optimize for net contribution margin, rather than just conversions or revenue. Your goal needs to be to optimize total profit as lower sales with higher margin could be a bigger win than higher sales with lower margin.

Try testing price treatment as well as the price amount too. Do color, size, cross-out comparison and even numbers make a difference? You should test that!

Rich Page, author of Website Optimization: An Hour A Day and co-author of Landing Page Optimization :

I think price testing is fine, and can have a good influence on conversion rates. The issue is more whether the marketer actually has control over the price being charged to test it.

Don’t presume you know what will work best, test many different combinations in an MVT for price display (colors, size, location), and savings (monetary versus percentage savings) to find the best converting.

Also, don’t forget to continue your test prices on your product pages through the rest of your shopping cart and checkout, otherwise you will risk confusing (and annoying your visitors).

Campaigns as a workaround?

Email and paid search campaigns support testing and allow you to some more leeway in price testing in offers and on landing pages. But as workarounds they are not perfect.

Email subscribers are a segment to themselves. Making pricing decisions based on existing customer response, or conversion on products visitors were not necessarily in the market for is not good practice. These tests can help you understand promotional marketing, but not optimal catalog pricing.

Paid search ad and landing page tests are better, but also have drawbacks. You can test various prices in search ads for response, but it’s tougher to control price visibility once the visitor navigates away from the landing page and back to the product through your menus.

Discrete price testing doesn’t reveal optimal price

In his latest book Landing Page Optimization: The Definitive Guide to Testing and Tuning for Conversions , Tim Ash describes one of the limitations of site-wide price testing.

Most companies treat price as a discrete variable. If you try to test price as a discrete variable (e.g., you test three distinct prices—your current price, a specific lower price, and a specific higher price), you are only getting information about the exact prices you choose to test. You will know which one of the tested prices is better. But you will not know if any of them are at the best price for maximum profit. The only advantage to this approach is that it works with your landing page optimization tools, and can be tested like all the other discrete variables. If your only alternative is not to test price at all, then you should use spot testing—a little bit of something is better than a whole lot of nothing.

Price tests may have a very short half-life

Pricing is a variable that can have a very short “half life.” A price that “wins” in January when wallets are typically tighter may not be the optimal price in May, September or December. Economic changes, competitive changes, the launch of a new generation of a product or the addition of more product substitutes to your catalog can affect optimal pricing. A price testing strategy should factor this in, and regular re-tests should be performed.

There’s no definitive answer to whether you should test prices or not. There remains arguments for and against. If you do decide to go forward, consider the strategies, controls and measurement tools you need to ensure you are not testing prematurely, testing too narrow a range of prices, and are optimizing for profit, not conversion rate.

Get Elastic