For brands, taking the time to identify winning creatives via regular A/B testing creatives is a key lever of digital advertising campaign optimization and iteration. Whether your creative ad production is in-house or you work through an agency partner, here are some essential A/B testing strategies to consistently identify winning ads.

The Importance of A/B Testing in Digital Advertising

According to 99 Firms, A/B testing can vastly increase conversions by up to 400%. It also enhances revenue. In fact, Microsoft’s Bing search engine achieved a 12% increase in revenue due to A/B testing.

Along with revenue and conversion wins, A/B testing can drive value in your digital advertising campaigns in the following ways:

• Audience Insights

A/B testing helps identify which combinations effectively engage your audience, giving you valuable information about your audience, their preferences and their engagement triggers. This insight can benefit many aspects of your conversion funnel, not just your creative strategy.

• Design Direction

Along with audience insights, A/B testing allows for the collection of creative information for designers, scaling your creative strategy and creation of high-performing ads.

• Increased ROI

By systematically testing ad variants in elements such as call-to-action buttons, messaging, and end cards, businesses can make data-driven decisions that lead to continuous improvement and increased return on investment (ROI).

How to Unlock Winning  Creatives with A/B Testing

Develop a Testing Strategy and Allocate a Budget

Begin by establishing clear objectives for what you wish to achieve with your testing. Are you aiming to improve click-through rates, boost conversions, or enhance user engagement? Once your goals are set, decide on the key elements you want to test, such as CTAs, ad copy, imagery, and interactive elements like end cards.

Next, allocate a specific portion of your marketing budget towards A/B testing. This R&D budget will allow you to explore different creative variations and measure their impact without compromising your overall campaign funds. Allocating this budget inline with your ads strategy. For example, if static ads are your main channel, prioritize your budget accordingly but also consider the potential of interactive formats like playables, which might require a different budgeting approach due to their complexity and engagement potential.

Clearly Define Goals and Metrics

Before conducting A/B tests, clearly define the goals and metrics you want to evaluate with your test. For different ad formats, you might measure different metrics. For example, at CRAFTSMAN+, for playable ads, we generally focus on IPM (Installs per Mille). This metric measures the number of times a mobile app is installed per thousand ad impressions. In user acquisition campaigns, it’s a key metric for understanding how creatives, targeting strategy, and/or app store optimization impacts conversions.

Always Be Testing

The digital advertising landscape is dynamic, with audience preferences and platform algorithms constantly evolving. To stay ahead, adopt an "Always Be Testing" (ABT) mindset. This proactive stance helps mitigate the effects of ad fatigue, where user interest wanes over time. As you build your testing strategy, create a repository of ad variants so your brand is ready to serve new iterations when performance plateaus due to ad fatigue. For playable ad campaigns, we recommend partners create at least 20 playables before launching.

Iterate Key Variables

There are certain key elements in an ad that marketers should always be testing. These include CTAs, ad copy, and end cards. Making small, iterative tweaks to these elements can efficiently unlock small gains that add up to significant performance wins in the long term. 

For example, at CRAFTSMAN+, we recently worked with a mobile game developer to test the impact of switching out static end cards for interactive end cards in a video-based UA campaign. The results were phenomenal. Not only did interactive end cards generate a massive uplift in conversions, it enabled our partner to scale their spend across multiple different channels.

The takeaway?

Testing variations of key elements — like changing the phrasing of a CTA — can have powerful results. These outcomes underline the importance of continuous iterative testing. It’s a cycle that should never stop, as it constantly brings new insights and improvements to your campaign strategies.

Test for User Psychology and High-Level Logic

A/B testing isn't just about tweaking small elements; it’s also about looking at the broader picture. This includes testing elements that impact user psychology. For instance, do users respond better to a winning or losing scenario in an ad? Test elements that help your team understand how different narratives and emotional triggers — like humor or FOMO (Fear of Missing Out) — impact user engagement. 

Depending on the type of campaign, especially when it comes to playable ads, it’s also crucial to evaluate different characters, background images, and gameplay mechanics. The goal is to see which emotional angles and elements resonate most with your audience.

Common Mistakes to Avoid in A/B Testing

To maximize the effectiveness of A/B testing, there are certain pitfalls to avoid:

• Avoid Testing Too Many Variables Simultaneously: This can muddy your results! Design A/B tests to test only one element at a time to accurately analyze the impact of each change
• Have a Clear Hypothesis: Each test should be conducted with a well-defined hypothesis to guide your understanding of the results. For example, you might hypothesize that serving a playable ad featuring a female character to a female segment of users will increase IPM.
• Be Mindful of Seasonality: Seasonal variations can significantly impact test results. Testing should account for these variations to maintain accuracy.
• Optimize for Channel Placement: In today's omnichannel world, ensuring that your creatives are optimized for the specific channel they are served in - whether mobile, CTV, Digital out of home (DOOH), etc. - is crucial.
  

How to Leverage A/B Insights Through a Partner

For many brands and large developers, campaigns and creative testing is undertaken through agency partners. In these cases, it can be more complex to understand what creative concepts and elements drive results. Our recommendation to brands working through partners for their A/B testing is to strategically name asset files in order to better track results. This way, whether reports are clearly laid out or not, you can determine which ad variants are performing.

The Future of Creative Testing: Embracing New Technologies

The integration of AI and automation in creative testing is an exciting development. AI can be used to generate a vast quantity of high-level concepts to test and automate testing processes, such as “killing off” ads that don't meet certain performance thresholds. 

At CRAFTSMAN+, we are committed to building tools that continuously empower marketers with ad creation and testing solutions at scale. Our latest product, CANVAS, provides a comprehensive suite of tools to design, resize, and export static ads for omnichannel, multilingual, and geo-sensitive campaigns in a fraction of the time traditionally required. With tools like our upcoming launch of CANVAS, brands can efficiently scale the creation of ad variants, making it feasible to test a wide range of creatives within their allocated budget. 

For more insights on building winning creatives, check out my recent webinar with Singular and experts from Liftoff, MobileAction and Forbes on building creatives that convert.

Unlocking Winning Creatives in 2024

CRAFTSMAN+ can help you scale your ad creation across interactive, video and static formats. Our CRAFTSMAN Studio provides a seamless way to optimize your ads, offering both ready-made templates and custom tools for bespoke creation. Level up your advertising strategy and get a sneak peak at CANVAS by requesting a meeting with our team of creative experts.