MKT.10 - Evaluate the Contribution and Cost of a Marketing Intervention

Type: Marketing Method pattern Status: Stable

MKT.10:1 - Problem frame

Use this pattern when a decision to continue, change, expand or stop marketing work depends on what that work contributes and what it consumes. Examples include choosing between invitations, deciding whether a supported trial improves customer results, or reconsidering a channel whose attributed revenue looks attractive but whose incremental contribution is unknown.

The result is an answer bounded by a particular intervention, alternative, population, outcome and period, with the relevant costs and uncertainty. The answer may support a decision, delimit a smaller trial, or show which missing evidence prevents the claimed conclusion.

The intended reader can name the business decision and engage the people who supply the observations, causal analysis and financial account. A simple count may be sufficient to describe completed work. Use the causal parts only when the decision or claim depends on what the intervention changes; an ordinary record of what happened does not require an experiment. Specialist statistical identification and estimation remain with the people competent to supply them.

MKT.10:2 - Problem

A dashboard can show impressions, replies, purchases, satisfaction and expenditure without explaining whether an intervention helped. People who receive more support may have harder problems; customers already more likely to achieve the outcome may select themselves into a trial; demand may rise during the campaign for another reason. Assigning revenue to the most recent contact describes a rule of attribution, not the result under an alternative contact policy.

Begin from the decision and its contrast. Define observations that retain their meaning, obtain an adequate comparison when a causal conclusion is needed, and combine that result with the whole economic and operational consequence. Choose the action in light of what the answer establishes and what a mistaken choice would cost.

MKT.10:3 - Forces

MKT.10:4 - Solution

MKT.10:4.1 - State the decision and the result that would change it

Name the actual alternatives: retain the existing invitation, add guided help, use another channel, reduce volume, or stop the intervention. Keep the useful customer result and the provider’s decision visible. An increase in replies can answer a reach question while remaining insufficient for an expansion justified by better customer performance.

Specify whose result matters, what would count, when it must occur and what downside can defeat the action. Include different participants when their outcomes differ: the purchaser can save effort while the user acquires extra work. MKT.6 supplies the agreed result; MKT.5 supplies the intended experience conditions. Neither supplies an observed effect by itself.

Determine the kind of answer required. “How many trials were completed?” is descriptive. “Which customers are likely to request help?” is predictive. “How many additional useful completions would this invitation produce compared with the current invitation?” is causal. These questions can use related data but require different grounds. A descriptive answer may be enough to stop an infeasible service even when its causal benefit remains unresolved.

Choose a practically consequential difference before interpreting a favourable fluctuation. Its basis can be the value of the result, the burden of the intervention and the consequences of a mistaken decision. Use a justified local threshold or a comparison of alternatives; do not invent a universal minimum response rate.

MKT.10:4.2 - Make each observation interpretable

Define the observation unit, eligible population, period, source and counting rule. For a response rate, say who had an opportunity to respond and what response counts. Distinguish unique people, organisations, opportunities and events. Several purchases by one organisation need not be several independent observations; a missing report is not necessarily a failed result.

Match the measure to the claimed outcome. A completed form can establish a submitted request. It cannot by itself establish informed choice, successful work or a favourable experience. Ask for the additional observation only when the decision depends on it. Where a customer reports an experience, preserve the question, timing, scale and respondent; an ordinal rating does not automatically support arithmetic differences, monetary conversion or comparison with another questionnaire.

Check how the data are produced. A new reminder may change who answers a survey, a new interface may change what gets logged, and better records may increase the apparent number of problems. Account for these changes before interpreting a trend. Retain consequential uncertainty about missing observations and selective follow-up.

Agree how much observation is needed and how to obtain it without defeating the intervention or its receiving service. Use existing adequate records; ask for a limited observation when it can resolve the decision. MKT.12 supplies a focused inquiry where the unresolved issue is what the customer’s work or answer means.

MKT.10:4.3 - Construct the causal contrast and its credible comparison

When contribution means a causal effect, define the intervention and comparison as ways of acting, including intensity, timing, support and consequential variants. Specify the target population, outcome and horizon. An offer of help and actual use of that help are different interventions. An invitation sent to one team may also affect another team through shared staff or shared information.

Use C.28.CM, Construct and Challenge a Causal Model to explain why the contrast could change the outcome and what else could produce the observations. Recover common causes, selection, intermediate results and feedback that change the inference. A journey diagram or chronological sequence is not a causal model merely because arrows connect its events.

Then obtain an identification answer through MMP.15, Identify an Intervention Effect from Available Data when this requires specialist analysis. The request names the contrast and available data, including how participants entered them. The supplier returns which effect the data can identify, under which assumptions, or a useful bound or reason it remains unidentified. More accurate fitting cannot repair an effect the data do not identify.

Select a feasible comparison suited to the question. Random allocation can make groups comparable for the effect of the assigned intervention if allocation, follow-up and interference conditions are adequate. It does not automatically identify the effect of actual use among self-selected users. Observational comparisons require defensible assumptions about relevant differences, usable variation and measurement. A before-and-after comparison requires a reason that other changes do not explain the result. Asking for “a control group” without these conditions supplies no causal answer.

Design the comparison with its operators. Avoid allocating a promise the service cannot fulfil or depriving people of an existing commitment. Arrange follow-up, exception handling and a stopping decision if the work becomes infeasible. If the credible comparison is unavailable, return the supported descriptive result and the unresolved causal question. A bounded action under uncertainty can still be chosen explicitly.

MKT.10:4.4 - Estimate the supported effect and challenge its use

Give the identified question and qualified observations to an adequate estimation method. Retain the estimated magnitude, uncertainty and consequential assumptions. Statistical precision addresses variability under the method; it does not remove selection, measurement error, missing follow-up or a wrong causal model.

Check the conclusions that matter for the decision. Could plausible missing outcomes reverse it? Does a result disappear when an unsupported comparison is removed? Does an average conceal a group for whom the intervention is impractical or harmful? A subgroup claim needs adequate grounds; dividing the same small dataset repeatedly does not create them.

Keep discovery and confirmation distinct when many messages, channels or outcomes were examined. A selected favourable result may justify a new bounded comparison rather than immediate expansion. If a decision permits repeated inspection, agree how the resulting uncertainty will be handled with the analyst instead of repeatedly stopping at the first attractive number.

Before applying the result elsewhere, compare the new population, baseline service, intervention and support conditions. A trial with spare specialist capacity may not describe a larger programme that queues every customer. Establish whether the earlier estimate applies under those conditions, or obtain a new comparison. A precise result for the wrong use remains an inadequate answer.

MKT.10:4.5 - Compare the whole contribution with its burden

Use MA.8, Account for Customer and Product Economics Over Time when the decision depends on economic consequences. Preserve the target unit and horizon, distinguish historical costs from costs changed by the future alternative, and place receipts and payments at their relevant dates. Count acquisition, preparation, adaptation, delivery, support, recovery and remaining obligations where they change the comparison.

A customer’s purchase value is not automatically incremental profit. Ask what revenue and expense would differ under the alternatives, which contributions would have happened anyway, and whether later work has been included. Avoid counting the same additional outcome under several channels or counting both its estimated monetary benefit and the same benefit inside a customer-value total.

Combine uncertainty honestly. An uncertain causal effect and an uncertain margin produce an uncertain economic conclusion. Show the threshold at which the choice changes or compare credible ranges when a point estimate would conceal the issue. The largest expected contribution can still be infeasible because of cash timing, scarce staff or obligations to current customers. MKT.11 supplies actual delivery conditions; its missing capacity is not repaired by a favourable average return.

Use nonmonetary consequences in their own terms where no adequate conversion exists. A client’s ability to complete important work, burden and experience can affect the decision without a fabricated price. State whose objective is served and what trade-off the decision maker accepts.

MKT.10:4.6 - Return the decision and the conditions for revisiting it

Return the compared alternatives, what the observations establish, the causal conclusion if supported, the relevant burden and the decision that follows. State what remains uncertain and why that uncertainty permits or limits the action. Continue, change, run a smaller informative comparison, defer expansion or stop according to this answer.

Send the consequence to the receiving work. MKT.3 can change the invitation; MKT.5 or MKT.6 can change the proposed contribution; MKT.11 can change the supply arrangement; MKT.13 can bound deployment of the commercial model. Keep a needed action and its responsible recipient explicit rather than returning an uninterpreted dashboard.

Reopen the answer when the intervention, population, measurement, service capacity, economic conditions or a material causal assumption changes. Keep the original result with its scope; a later disappointing outcome does not justify rewriting what the earlier comparison actually observed.

MKT.10:5 - Archetypal Grounding

MKT.10:5.1 - A promising invitation with an uncertain useful effect

A provider wants to add guided onboarding to an invitation. The decision concerns another comparable group of 1,000 eligible customers. The useful outcome is verified completion of a specified task within 14 days, not clicking the invitation. Existing commitments remain available under both alternatives, and the provider has checked that the additional help can be delivered without reducing that service.

In a constructed trial, 2,000 eligible customers are randomly assigned equally to the new and current invitations. Assume complete follow-up, no material spillover and the same stated outcome rule. There are 120 useful completions among 1,000 assigned the new invitation and 100 among 1,000 assigned the current invitation. The estimated effect of the assigned invitation is 12% minus 10%, or 2 percentage points. Comparing only people who used the help would answer another question and would lose the original allocation’s simple comparability.

For this illustration, an approximate 95% confidence interval uses the difference plus or minus 1.96 times its standard error. With independent binary outcomes, that standard error is the square root of (0.12 × 0.88 / 1,000 + 0.10 × 0.90 / 1,000). The interval is approximately −0.7 to 4.7 percentage points. The positive point estimate is compatible with both a small loss and a worthwhile gain. It does not establish that the intervention improves outcomes. The interval also depends on the stated trial and estimation conditions; it says nothing about an omitted delayed outcome.

Suppose a qualified prospective account values each additional useful completion at 40 monetary units of contribution after its associated delivery expense, and another campaign for 1,000 customers would require 500 units of additional fixed preparation and help capacity. Assume these flows occur within the decision’s funded period. The point estimate gives 20 additional completions, a contribution of 800 and a net increment of 300. Roughly carrying the effect interval through this simplified account gives about −800 to 1,400, even before uncertainty in the value per completion. A positive expected increment alone is therefore a weak basis for an expensive irreversible expansion.

The decision maker chooses another affordable bounded comparison because the unresolved effect can change the decision. If even the favourable plausible effect could not cover the prospective burden, stopping would be more useful than collecting more data. If customers independently need the help to receive an existing promise, that obligation must be supplied regardless of this marketing experiment.

The next proposed group includes organisations with restrictive data access and would exhaust specialist capacity. It is no longer the same deployment condition. MKT.11 establishes a feasible service variant before its benefit is tested; multiplying the earlier 2% by the new audience would conceal the changed intervention.

MKT.10:5.2 - More support accompanies more failures

An observational report shows that customers receiving many support calls fail more often. Staff explain that they contact customers after detecting serious difficulties. The report may identify a high-burden group, but it does not establish that calls cause failure. The causal model retains initial difficulty and the timing of support; the analyst considers whether available observations permit an adequate contrast. If they do not, the provider can improve the handover and record the needed conditions while withholding a causal effect claim.

MKT.10:6 - Bias-Annotation

Attribution systems, selective follow-up and incentives to show campaign success favour visible positive results. Customers who are easy to observe can differ from those who leave silently. The person commissioning an evaluation may prefer a precise answer even when the real uncertainty concerns identification. Preserve alternative explanations, missing outcomes and the possibility that no additional intervention is warranted.

MKT.10:7 - Conformance Checklist

• The answer serves a named decision and preserves its alternatives, population, outcome and period.
• Observation rules distinguish the events and participants that affect interpretation.
• A causal claim has an explicit intervention contrast and an adequate identification basis; description or prediction is not silently promoted to effect.
• Estimation uncertainty and consequential causal or measurement limitations remain visible.
• The whole incremental burden, dated economic consequences and actual capacity qualify the decision.
• The receiving work can act on the conclusion and recognise a change that requires revisiting it.

MKT.10:8 - Common Anti-Patterns and How to Avoid Them

MKT.10:9 - Consequences

The evaluation becomes useful for an actual action and limits claims that the data cannot support. It can reveal that a small descriptive answer is sufficient or that an expensive causal analysis would not change the decision. Establishing a causal effect or applying its estimate to a new population, for example, may require additional observations, specialist work or a trial; unresolved uncertainty can legitimately limit expansion.

MKT.10:10 - Architectural Rationale

Marketing chooses the intervention question and uses its answer. Causal modeling supplies the explanatory structure, mathematical modeling supplies identification and estimation, and management accounting supplies the economic comparison. Keeping these contributions explicit allows a practical decision without turning a dashboard, statistical fit or revenue attribution into proof of effectiveness.

MKT.10:11 - SoTA-Echoing

For observational marketing models, adopt the distinction in Google Meridian’s causal-inference guidance between model fit and causal grounds: alternative well-fitting models can imply different returns, and consequential assumptions need domain justification. This supports bounded interpretation rather than selection by predictive accuracy alone.

For the construction of an answer, adopt C.28.CM and MMP.15’s developed separation of causal contrast, assumptions, identification and estimation, then join their result to MA.8’s prospective account. This adds work when an action depends on an effect; an adequate ordinary description remains sufficient for a descriptive question.

MKT.10:12 - Relations

MKT.3/5/6/7/9/11/14 supply interventions and receiving decisions. MKT.8 can locate an observed difficulty but does not identify its cause. MKT.12 can qualify the customer’s situation and the meaning of an outcome. MKT.13 uses an effect estimate only within its actual deployment conditions.

C.28.CM supplies the causal model, MMP.15 the identification answer and appropriate mathematical modeling the estimate. MA.8 supplies customer and product economics over time. A missing specialist result returns as a bounded unresolved contribution, not as an invented precise number.

MKT.10:End