A representation of the current condition of a system, agent, or environment at a specific point in time, containing the information required to make a decision or prediction. In reinforcement learning, the state summarizes the relevant history of an agent-environment interaction and is the input to the policy. In recurrent neural networks, the hidden state is an internal memory that carries information from previous time steps. In customer journey modeling, the state captures the customer’s current position in the funnel.
Also known as system state, agent state, hidden state
In reinforcement learning, the state is a representation of the environment at a given time step that contains all information an agent needs to decide which action to take. The Markov property, which most RL frameworks assume, states that the optimal action depends only on the current state, not on the history of how the current state was reached. This assumption simplifies the decision problem: the agent needs only to learn a policy that maps current states to actions, not a policy that considers the full history of prior states and actions. State representation is a critical design decision: if the state does not contain information that is genuinely relevant to the optimal action, the learned policy will be suboptimal in ways that are difficult to diagnose.
In recurrent neural networks, the hidden state is a fixed-dimensional vector that the network updates at each time step. The hidden state at time t is a function of the hidden state at time t-1 and the input at time t, accumulated over the sequence. This recurrent state serves as a compressed memory of the sequence up to the current position: information from early in the sequence is encoded in the hidden state and can influence predictions at later steps. LSTM architectures manage hidden states through a cell state (long-term memory) and a hidden state (short-term working memory), with gating mechanisms that control what information flows into and out of the cell state, preventing important long-range information from being overwritten by irrelevant recent inputs.
In customer journey and marketing automation contexts, state refers to the customer’s current situation, which determines what communication or experience should be delivered. A customer state might encode: recency of last purchase, current loyalty tier, recent browsing category, last email engagement date, and current campaign exposure. The state is an input to rules-based journey orchestration systems (if state matches condition X, trigger action Y) and learned RL-based orchestration systems (a policy that maps the state vector to the optimal next action). The richer and more accurate the state representation, the more precisely the orchestration system can personalize communications.
A working ad agency building customer journey automation, RL-based engagement systems, or personalization engines for clients is making implicit state representation decisions in every deployment. The customer attributes included in the state vector, the features engineered from raw behavioral data, and the time horizons used to compute behavioral summaries collectively determine what information the orchestration system can act on. A state that is too narrow (missing relevant behavioral signals) produces a system that cannot differentiate customers who deserve different experiences. A state that is too wide (including irrelevant noise variables) increases dimensionality without improving decisions and may cause overfitting in learned policy systems.
Customer lifecycle state variables that capture recency, frequency, and monetary dimensions simultaneously enable more targeted journey orchestration than any single metric alone. A customer state vector that includes days since last purchase (recency), number of purchases in the trailing 90 days (frequency), and trailing 90-day spend (monetary) provides the three independent dimensions of customer relationship health that the RFM framework captures. A customer with high frequency, high monetary value, and increasing recency is a growing high-value customer who warrants retention investment. A customer with historically high frequency and monetary value but rapidly increasing recency is at churn risk. These distinctions are visible only when all three state dimensions are included; a state that includes only frequency misses the recency signal that indicates churn risk among high-frequency customers.
Hidden state representations in sequential behavioral models enable systems to distinguish customers with the same current attributes but different behavioral trajectories. Two customers with identical purchase frequency, spend, and recency may be on completely different trajectories: one is accelerating (growing engagement) and one is decelerating (declining engagement). A point-in-time state representation that captures only current attributes cannot distinguish them. A hidden state representation that accumulates information about how each customer’s behavior has evolved over time distinguishes these trajectories, enabling the orchestration system to invest in retaining the decelerating customer before they reach churn while conserving resources on the accelerating customer who is already engaging naturally.
Partially observable state in real-world customer engagement systems means the agent must make decisions with incomplete information about the customer’s true situation. A marketing automation system has access to its own channel interactions with the customer (email sends, SMS, push notifications) but typically lacks visibility into competitor interactions, offline purchase behavior, major life events, and other factors that influence the customer’s receptiveness. This partial observability means the state the system acts on is an approximation of the true state of the customer relationship. Systems that acknowledge this uncertainty through probabilistic state representations or multiple behavioral signal aggregation are more robust than systems that treat their observed state as a complete picture of the customer’s situation.
An agency is designing a customer engagement state representation for a direct-to-consumer pet supply brand client that will feed a rules-based journey orchestration system. The client has 380,000 active customers with purchase, email, app, and support interaction data available. The agency designs the customer state as a 28-dimensional vector updated daily. Recency signals (8 dimensions): days since last purchase, days since last email open, days since last app session, days since last support contact, days since first purchase, days since last browse session, and a binary flag for purchase in the past 14 days and 30 days. Frequency signals (7 dimensions): number of purchases in trailing 30, 60, and 90 days, number of email opens in trailing 30 days, number of app sessions in trailing 14 days, number of support contacts in trailing 90 days, and current subscription status binary. Value signals (6 dimensions): trailing 90-day revenue, trailing 12-month revenue, average order value in trailing 12 months, number of product categories purchased trailing 12 months, and loyalty tier (1 through 4). Behavioral trajectory signals (7 dimensions): change in purchase frequency from 60 to 30 days versus 120 to 60 days, change in average order value from first half to second half of trailing 12 months, email engagement trend (3 direction flags: improving, stable, declining), and a predicted 90-day churn probability from a separately trained logistic regression model. The 28-dimensional state vector is used as input to journey orchestration rules that determine which of 7 communication tracks (onboarding, high-value engagement, lapsed win-back, churn prevention, cross-sell, subscription upgrade, and low-engagement nurture) each customer should be placed in. Over the 90 days following deployment, the multi-dimensional state-based orchestration system produces 23% lower lapse rates and 31% higher reactivation rates for previously lapsed customers compared to the prior rules-based system that used only recency and purchase frequency without trajectory signals.
The generative AI foundations module covers state representation in reinforcement learning, hidden states in sequential models, and practical customer state design for journey orchestration and marketing automation systems.