An algorithmic technique for solving optimization problems by breaking them into overlapping subproblems, solving each subproblem once, and storing the results to avoid redundant computation. Dynamic programming underlies many of the optimization algorithms that power bidding, attribution, and scheduling tools in the advertising technology stack.
Also known as DP algorithm, memoization, optimal substructure
Dynamic programming applies to problems that exhibit two properties: overlapping subproblems, meaning the same smaller computations appear repeatedly in the larger problem, and optimal substructure, meaning the optimal solution to the whole problem can be built from optimal solutions to its parts. When both properties hold, storing solutions to subproblems, a technique called memoization, avoids recomputing the same calculations and reduces what would otherwise be exponential computation time to polynomial time.
The technique was formalized by Richard Bellman in the 1950s for operations research problems, and Bellman’s equations remain central to reinforcement learning, where an agent must find the optimal action sequence through a series of decisions that each depend on the outcomes of prior ones. Dynamic programming gives the mathematical structure for computing those optimal policies efficiently.
Practical applications extend well beyond reinforcement learning. Sequence alignment algorithms that underpin natural language processing use dynamic programming. Optimal bidding strategies in auction-based ad markets are dynamic programming problems: the optimal bid at each impression depends on budget remaining, time left in the campaign, and the distribution of remaining opportunities. Attribution models that allocate credit across touchpoints use dynamic programming variants to find the allocation that best explains observed outcomes.
Most of the optimization tools a working ad agency uses to manage campaign budgets, bids, and schedules are solving dynamic programming problems under the hood. Understanding what that means helps agencies configure those tools correctly, interpret their recommendations, and identify when the optimization objective the tool is solving is not the one the campaign actually needs to optimize.
Budget pacing is a dynamic programming problem. Deciding how to distribute a campaign budget across impression opportunities over a flight requires accounting for how spending today affects what is available tomorrow, and how the value of tomorrow’s impressions depends on how many remain. When pacing underperforms, the cause is often that the platform’s optimization objective, minimizing cost, does not match the campaign objective, maximizing specific business outcomes.
Attribution across touchpoints borrows from it. Multi-touch attribution models that solve for the credit allocation that best explains observed conversion sequences are using dynamic programming logic. Understanding that the allocation problem has this mathematical structure helps agencies interrogate attribution model assumptions more precisely, rather than treating attribution as a black box that produces numbers.
Reinforcement learning is dynamic programming at scale. The bidding optimization systems, automated campaign management tools, and recommendation engines that use reinforcement learning are built on Bellman’s framework. Knowing that connects the abstract algorithm to the tools on the agency’s platform stack and makes conversations with vendors about how those tools learn more tractable.
An agency is running a time-sensitive product launch campaign with a fixed budget that must convert efficiently in the first three days to establish early sales velocity. The platform’s standard budget pacing algorithm distributes spend evenly over the flight to minimize cost per impression. The agency overrides the default to front-load 60% of the budget in the first two days, reasoning that early conversion signals will improve the algorithm’s audience targeting quality for the remainder of the flight. The decision reflects the dynamic insight that spending patterns in the first period affect the optimization quality in subsequent periods, not just the immediate impression cost. The front-loaded campaign delivers 40% more conversions in the three-day window than an evenly paced equivalent at the same total budget.
The automations and agents module of the workshop covers how AI-driven campaign tools make decisions, what their optimization objectives actually are, and how to align those objectives with the outcomes clients care about.