Understanding Cause and Effect in Marketing: A Simple Guide to Quasi-Experiments

Imagine you’re a marketing manager trying to figure out if running a new ad campaign actually makes more people buy your product. Ideally, you’d run a perfect experiment: randomly show the ad to some people and not to others, then compare their purchases. This is a true experiment, and it’s the gold standard for figuring out cause and effect.

However, in the real world of marketing, perfect experiments aren’t always possible, ethical, or affordable. That’s where quasi-experiments come in.

What are Quasi-Experiments?

Think of quasi-experiments as “almost experiments”. They are used when we want to understand if one thing (like an ad campaign or a change in pricing) causes another (like an increase in sales or a shift in consumer behavior), but we can’t randomly assign people to different groups like in a true experiment. Instead, quasi-experimental research looks at situations where some change or event happens naturally, without the researcher intentionally controlling who gets the “treatment” and who doesn’t.

For example, imagine a new law is passed in one region but not another, or a company changes its policy for some customers but not others. These situations create a natural division that researchers can study to see if the change had an effect. The key is that this division wasn’t created randomly by the researcher.

Why are Quasi-Experiments Important in Marketing?

Quasi-experiments are incredibly valuable for marketing scholars and practitioners for several reasons:

• Understanding Real-World Impacts: They allow us to study the effects of real marketing actions and events as they happen in the actual marketplace.
• Making Informed Decisions: By understanding causal relationships, marketers can make better decisions about which strategies to implement and how much to invest. For instance, knowing if a price increase truly leads to a significant drop in sales helps in setting optimal prices.
• Answering Important Questions: They help answer crucial marketing questions about consumer behavior, firm actions, and market outcomes. For example, understanding how a data breach affects customer spending or how online reviews influence advertising spending.
• When Experiments Aren’t Feasible: In many marketing scenarios, running a controlled experiment is just not practical. Quasi-experiments offer a way to gain causal insights even then.

How Do Researchers Conduct Convincing Quasi-Experiments?

In quasi-experiments, researchers are studying the effect of something (like a new policy, a marketing campaign, or a product change) without having full control over who gets exposed to it—unlike in real experiments where people are randomly assigned.

🎯 The Main Challenge:

The biggest issue is this:

How can we be sure that the thing we’re studying (the “event” or “action”) actually caused the changes we see?

For example, if sales go up after a new ad campaign, is it really because of the ad? Or could it be due to something else—like a holiday, a price drop, or a competitor going out of business?

🔍 What is “Identification”?

“Identification” is a fancy word researchers use to mean:

Have we done everything we can to prove that the change is really caused by our event, not by something else?

This often involves:

• Comparing similar groups (one that was affected and one that wasn’t).
  
• Checking if the groups were already different before the event.
  
• Controlling for other things that might affect the outcome.
  

Here are some key steps and considerations involved in conducting strong quasi-experimental research:

Asking the Right Research Question:  

Everything starts with asking a good question—and in quasi-experiments, that means focusing on: “Does this specific action (X) actually cause a change in this marketing outcome (Y)?” Not all questions are worth doing a quasi-experiment for. You want a question that Matters for decision-making. (Would the answer help a marketer or manager do something better?). Needs causal insight. (Is it important to know whether X caused Y, not just that they happened together? Here is an example of a bad bad research question: “Do customers who see the ad spend more money?” This just shows a pattern—but maybe those customers were already big spenders. Here is a better question to ask: “Does showing this ad cause customers to spend more money?” Now you’re focused on causality—figuring out if the ad is what made the difference.

Finding the Right Data: 

In quasi-experiments, researchers can’t randomly assign people to treatment and control groups like in lab experiments. So instead, they look for natural changes in the “cause” (X) that affect some people or markets but not others. I call this natural variations. Natural variation means something happened in the real world that:

• Wasn’t planned by the researcher,
• Changed the environment in a way that mimics an experiment,
• And lets you compare what happened before and after, or between affected and unaffected groups.

We can also call it an exogenous change – if you have taken a structural equation model class, you should understand this term. It means that the change came from outside the system you’re studying—not from within the marketing strategy itself. This is important because it helps rule out other possible explanations.

  Examples of Good Exogenous Variation:

• A new government regulation that only affects one region.
  
• A company-wide policy change that wasn’t based on performance.
  
• A snowstorm that stopped deliveries in some cities but not others.
  These kinds of events create “as-if random” conditions that allow you to study cause and effect more convincingly..

Developing an Identification Strategy: 

When researchers use quasi-experiments, they are trying to figure out if something (let’s call it X, like a new policy or ad campaign) causes a change in something else (let’s call it Y, like sales or customer behavior).  But because they didn’t randomly choose who gets X and who doesn’t, they have to be very careful. They must convince others that the group affected by X (the treatment group) and the group not affected (the control group) are similar in every important way—except for X. If the two groups are very different to begin with, then any change in Y might not be because of X. It could be because the groups were different from the start.So, the researcher needs to explain why they believe the only big difference between the two groups is that one experienced X and the other didn’t. If there are some differences, they need to show how they accounted for them (like using statistical controls). This is the only way they can say, “We’re pretty sure X caused the change in Y.”

🛍️ Scenario: A Store Tests a New Ad Campaign

Let’s say Walmart runs a new ad campaign only in Texas, and a researcher wants to know if the campaign caused an increase in sales.

Now, they compare:

• Texas (treatment group – saw the ad)
  
• Oklahoma (control group – didn’t see the ad)

🧠 What the Researcher Must Prove:

The researcher has to explain why they believe Texas and Oklahoma are similar enough that any difference in sales is likely because of the ad—not because Texas is just richer, has more stores, or was already seeing a sales increase before the ad.

If the two states were:

• Already different in customer behavior
  
• Affected by other events (like a local holiday in Texas)

Then it’s hard to say the ad caused the sales boost.

✅ So What Do Researchers Do?

They try to show:

• The two groups had similar sales trends before the ad.
• There were no other big events in Texas at the same time.
• If there are known differences, they use data analysis to adjust for them.

Only then can they argue: “Yes, the sales boost in Texas was likely caused by the ad, not something else.”

Conducting Empirical Analysis: 

This involves using statistical methods to estimate the effect of x on y based on the identified variation. Common methods include:

Difference-in-Differences (DID): Difference-in-Differences (DID) is a method that helps researchers figure out if something actually caused a change. Instead of just comparing two groups at one point in time, DID looks at how each group changes over time—both before and after something happens to one of them. One group experiences a change (like a new policy or ad campaign), and the other doesn’t. Researchers then compare how much the outcome changed in each group. If the group that got the change improved more than the one that didn’t, and both were on similar paths before, it’s likely that the change caused the difference. This only works if both groups would have kept following the same pattern if the change hadn’t happened—that’s called the “parallel trends” assumption.

Regression Discontinuity (RD):This method helps researchers figure out if something caused a change by looking at situations where a decision or treatment is given based on crossing a specific cutoff point—like a rule that says only customers who spend over $50 get a discount. Researchers then compare people who are just below and just above that $50 line, because they’re likely very similar in every other way. If there’s a clear jump in outcomes right at the cutoff, it suggests the treatment (like the discount) caused the change. This only works if the cutoff is kind of random and isn’t tied to other things that would also affect behavior at that point.

Instrumental Variables (IV): This technique is used when it’s hard to tell if X really causes Y, because Y might also be affecting X, or there might be hidden factors influencing both. To solve this, researchers find another variable—called an instrument—that’s related to X but doesn’t directly affect Y. Instead, it only affects Y through its impact on X. By focusing on the part of X that’s driven by this instrument, researchers can better estimate the true effect of X on Y. The key rule is that the instrument can’t have any direct connection to Y except through X—this is called the exclusion restriction.

3. Addressing Challenges to the Research Design: Researchers must consider if there are other reasons besides the “cause” (x) that could explain the change in the outcome (y). They need to think about whether the variation in x is truly exogenous. For instance, could there be other events happening at the same time that are driving the results?.
4. Ensuring Robustness: To increase confidence in their findings, researchers perform various robustness checks. This involves testing if the results hold true under different statistical models, with different sets of control variables, for different time periods, and using slightly different definitions of the treatment and control groups. Placebo tests, where the analysis is repeated on a time period or outcome where no effect should be observed, are also important for checking the validity of the approach.
5. Exploring the Mechanism: Understanding why x causes y is often as important as knowing that it does. Researchers try to identify the underlying behavioral mechanism – whether at the individual, organizational, or market level – that explains the causal relationship. This can involve mediation analysis (looking at intermediate variables) or moderation analysis (examining how the effect differs across different groups or situations).
6. Considering External Validity: Researchers need to think about how generalizable their findings are to other settings, populations, or time periods. The effect observed in a specific quasi-experiment might be local, applying only to the specific context studied. Researchers should clearly discuss the assumptions needed for the results to reflect a broader average treatment effect (ATE) across the entire population of interest.
7. Acknowledging Limitations (“Apologies”): Transparency is key. Researchers should clearly state what remains unproven, the limitations of their study, and the assumptions their identification strategy relies on. This helps the reader understand the boundaries of the claims being made.

What if the Treatment and Control Groups Aren’t Initially Comparable?

Sometimes, the groups affected and not affected by the natural event might be different in important ways even before the event occurred. In such cases, researchers might use additional techniques to try and create more comparable groups:

• Propensity Score Matching: This method attempts to statistically match individuals or groups in the treatment and control conditions based on their observed characteristics (covariates) that might predict who receives the treatment. The goal is to create a control group that looks as similar as possible to the treatment group before the event, based on these observed characteristics. However, this method only addresses differences in observed factors and relies on the assumption that all relevant differences are captured by these observations.
• Synthetic Control Methods: These methods are particularly useful when studying the effect of an intervention on a single entity (like a country or a large company) with a limited number of potential control units observed over a long period. They create a “synthetic” control group by taking a weighted average of several untreated units to best match the treated unit’s outcome trends before the intervention.
• Selection Bias Correction: When the decision of whether to be “treated” is not random and depends on some unobserved factors, researchers might use statistical models like the Heckman correction to account for this selection bias. However, these methods often rely on strong assumptions about the underlying data and require a credible exclusion restriction (a variable that affects the selection into treatment but not the outcome directly) for identification.

In Conclusion

Quasi-experimental methods are powerful tools that allow marketing researchers and practitioners to learn about cause and effect in real-world settings where true experiments are not feasible. By carefully identifying natural sources of variation, employing appropriate analytical techniques, rigorously testing the robustness of their findings, and clearly acknowledging the limitations, researchers can provide valuable insights into how marketing actions influence consumer behavior and market dynamics. As marketing practices and data availability continue to evolve, these methods will become increasingly important for understanding the complex causal relationships that drive success in the marketplace.