Friday Field Notes: Your Media Plan Might Be Running on Old Assumptions

When was the last time you tested a channel you had already written off?

It is a useful question because a decision can outlive the evidence behind it. A platform underperformed. A format disappointed. The team moved on. Eventually, “it didn’t work in that test” becomes “it doesn’t work for us.”

This week, a post from Matej Lancaric brought that question back into focus through Supercell’s use of playable ads on Snapchat. His accompanying article was published October 1; this is a closer look at an existing example, rather than a newly announced result.

What Supercell actually tested

According to Snapchat’s published case study, Supercell tested playable ads for Clash Royale in a US app-install campaign against its business-as-usual strategy. Snap reports 19% higher return on ad spend and 37% lower cost per install.

One detail matters: Supercell was already a long-term Snapchat partner. This was an established advertiser testing a different format within the platform.

That makes the example relevant to two decisions: whether to revisit a channel, and whether to revisit what we run there.

The results are platform-reported, specific to this campaign, and do not establish that Snapchat outperforms Meta or TikTok. The public case-study text also leaves questions about budget, sample size and revenue measurement windows unanswered.

A channel decision is also a creative decision

Last week’s Field Notes explored AI’s movement from generating assets toward creating experiences.

The next question is where those experiences get tested.

A video asks someone to watch. A playable gives them an opportunity to try a mechanic, make a choice or experience a small part of the product before installing.

My takeaway is that an earlier channel test may tell us something about the particular experience we offered there. It may leave other creative approaches unexplored.

That gives the next brief a more useful question: what could someone do in this ad that would help them understand why they might want the product?

Give the next test a clear job

For a team considering a retest, I would start with one hypothesis and a bounded budget.

Choose a concept with evidence behind it. Adapt it to the placement’s requirements, check the interaction and loading experience, and confirm that tracking works before spending.

Define the comparison in advance. Testing playables against an existing campaign on one platform answers a different question from comparing Snapchat with another channel. Keep the audience, geography, operating system and measurement windows as comparable as the test allows.

Then give the acquired users enough time to generate the outcome being measured. Two weeks may provide an early signal; it will not automatically settle a longer-term revenue question.

What remains open

A lower install cost deserves attention, but the follow-up is whether those users retain, generate revenue and help the business grow. Better attributed performance alone does not establish incremental growth.

For the next planning cycle, the practical priorities are straightforward: revisit one dismissed channel or format, document the assumption behind that decision, and define what evidence would justify changing it.

At CRAFTSMAN+, that is a useful connection between creative strategy, production and measurement. The next brief can carry forward what the last test taught us while making room to challenge what we think we already know.

Which assumption in your media plan deserves another test?