MyAIDevelopment · Product · Operations notes

Measurement · Attribution · Incrementality · Privacy

"Did this ad create revenue?" is the hardest question in marketing. The key is distinguishing correlation from causation.

Three Measurement Tools

Method Question Strength Weakness
Attribution (MTA, last-click, etc.) Which touchpoints came before the conversion Fast, granular Not causal; sees only trackable touchpoints
Incrementality test (lift, geo experiments) How much would conversions drop without the ads Estimates causality Cost and time; one question at a time
MMM (Marketing Mix Modeling) How much did each channel's budget contribute to results No individual tracking; includes offline Needs a long data history; sensitive to model assumptions 
flowchart LR
    A[Attribution] -->|Hypothesis| T[Incrementality Test]
    T -->|Calibration| M[MMM]
    M -->|Budget Plan| B[Budget]
    B --> A

The IAB's State of Data 2026 report also discusses how privacy changes, closed platform-specific measurement, and cross-channel inconsistency make it hard to connect media exposure to business outcomes, and treats AI as a tool to strengthen attribution, incrementality, and MMM.

In practice, no one uses just one. The recommended combination is to form hypotheses with attribution, validate them with experiments, and calibrate MMM with experiment results. Meta's Robyn documentation also describes calibrating MMM with lift-experiment results.

Limits of Attribution

Bad example:

Brand keyword ads show a 20x ROAS. Let's double the budget.

Good example:

Turn off brand keyword ads in some regions for four weeks and compare the change in total conversions to estimate the incremental effect.

Incrementality Tests

Test group: exposed to ads
Control group: not exposed (randomized or by region)
Lift = (Test conversion rate - Control conversion rate) / Control conversion rate

Caution: small samples produce unstable results. Before the experiment, decide the observation period, the minimum effect size, and the decision criteria.

A Worked Example

This is a generic illustration of how to read a user-level lift experiment. It does not use the example product from documents 04 and 07 (KRW 2M a month in search ads, 10 paying accounts a quarter); it assumes a service with much larger ad reach. Here a conversion is a free signup. Every number is illustrative.

Conversion = free signup
50,000 people per group (randomly assigned)
Control conversion rate 2.0%  → 1,000 conversions
Test    conversion rate 2.4%  → 1,200 conversions

Lift                    = (2.4 - 2.0) / 2.0 = 20%
Incremental conversions = 50,000 × (2.4% - 2.0%) = 200
Test ad spend           = KRW 10,000,000
Incremental CAC         = KRW 10,000,000 / 200 = KRW 50,000

If the platform credits all 1,200 test-group conversions to the ads, CAC looks like about KRW 8,333. The conversions the ads added are 200, and budget decisions use the incremental CAC of KRW 50,000. That is a cost per free signup; to turn it into a cost per paying customer, divide once more by the signup-to-paid conversion rate.

At the scale of the example product in documents 04 and 07, this design is underpowered. As calculated below, detecting 2.0% → 2.4% needs about 21,000 people per group; with KRW 2M a month in search ads and about 10 paying accounts a quarter, the sample is hard to fill even for signups, and a difference in paid conversions cannot be detected at all. That is why the channel mix in document 04 checks this channel with a quarterly on/off or geo test.

Decide Before the Experiment

MMM

MMM does not need individual-level tracking, so it is resilient to privacy changes. In exchange, meaningful results often require at least one to two years of weekly data and enough budget variation.

Privacy Changes: What Actually Changed

Area Status (dates)
Chrome third-party cookies 2024-07 shifted from a scheduled phase-out to a user-choice approach → 2025-04 announced it would not roll out a separate choice prompt either. Users control it in existing settings
Privacy Sandbox 2025-10-17 announced retirement of most APIs, including Topics, Protected Audience, and Attribution Reporting. CHIPS, FedCM, and Private State Tokens remain
Safari 2020-03 WebKit announced full third-party cookie blocking by default
Firefox 2022-06 Total Cookie Protection (per-site cookie partitioning) made the default worldwide
iOS ATT Since 2021, apps need user permission to track. France's competition authority fined Apple EUR 150 million (2025-03-31), Italy's AGCM about EUR 98.6 million (2025-12), and Germany's Bundeskartellamt made commitments binding, including neutral prompts (2026-08-17)
iOS ad measurement AdAttributionKit (2024) follows SKAdNetwork, with aggregated postbacks and no individual identification
Google Consent Mode v2 Since 2024-03, consent signals for ad measurement and personalization (ad_user_data, ad_personalization) must be passed for EEA users

In summary:

First-Party Data and Server-Side Sending

To reduce tracking loss, platforms offer ways to send consented first-party data from the server.

This is not a way to bypass consent. Use it only within the scope of consent.

Common Mistakes

References