Stop Guessing: A Simple Email vs Web Push Experiment For Your SEO SaaS

Most SEO SaaS founders quietly assume email is the workhorse for upgrade nudges, then sprinkle in web push “because we can.” Few actually know which channel moves revenue for which users.

This playbook gives you a concrete way to find out.

The sequence at the top of this page is the whole journey, end to end: a behavioral trigger, a Random Split into email vs web push, then shared measurement so you can compare downstream conversions cleanly.

We’ll walk through how an SEO SaaS like “RankPulse” (rank tracking and content briefs, ~$40k MRR, 3-person team) wires this up in Spreeflo, and how you can adapt it to your own funnel without adding more manual work to your week.

Where this experiment fits in an SEO SaaS funnel

Channel tests matter most at clear “money moments.” For an SEO SaaS, a few obvious ones:

• Trial users who hit their keyword limit and see the upgrade wall

• Self-serve customers who keep visiting the pricing page without upgrading

• Agencies nudged to move from monthly to annual

We’ll anchor this walkthrough on a specific scenario:

A trial user hits a usage limit (“you’ve added 50 keywords”) and sees your upgrade modal. You want to know whether an email or a browser push notification is more effective at turning that intent into a paid subscription.

Instead of blasting everyone with both, you:

1. Capture the behavioral signal when the user sees the upgrade modal

2. Randomly split those users into two groups

3. Send Group A an email, Group B a web push

4. Measure which group upgrades more, and what the side effects are (unsubscribes, push opt-outs, etc.)

Once you build this once, it runs on autopilot for every future user who reaches that moment. That’s the spirit of “founder-led businesses win on leverage, not headcount” in practice.

The journey at a glance

Visually, the journey you see above follows this shape:

• Behavioral trigger: “user is ready for an upgrade opportunity”

• Random Split: 50% to email arm, 50% to web push arm

• Each arm:

• - Tag the contact with their assigned channel

• - Send one upgrade nudge via that channel

• Merge, then a quiet measurement phase:

• - Wait a fixed window for “upgraded” behavior

• - Tag who converted vs who didn’t

No one gets double-nudged. You simply learn, over time, whether email or push produces more upgrades (and at what “cost” in list health).

Let’s go node by node.

Step 1: Define and instrument the upgrade moment

Before you touch the journey builder, you need a reliable event that says:

“This person just hit a paywall where an upgrade would make sense.”

In RankPulse’s case, that’s when the upgrade modal appears after a user adds their 51st keyword. They track this in two places:

• Frontend event for in-app analytics: when the modal renders, the React component calls Spreeflo.track('upgrade_modal_shown', { current_plan: 'trial', feature: 'rank_tracking' }) in the browser, using the Spreeflo SDK.

• Backend event for billing-grade truth: when Stripe confirms an upgrade, a webhook handler POSTs to the Spreeflo events API with event: 'upgraded_plan', the contact’s email, and plan details (per the Spreeflo API docs).

For this experiment you need both:

• upgrade_modal_shown to trigger the journey

• upgraded_plan (or a plan attribute change) to measure conversion later

If you haven’t added Spreeflo’s SDK to your app yet, start with the web tracking and analytics guide. You want visitors identified via Spreeflo.identify(email, {...}) so your events and messages all land on the same contact record.

Step 2: Build a segment of “upgrade-ready and reachable” users

Pure channel tests only work if both arms are actually reachable on that channel. If your push arm includes lots of people who never subscribed to notifications, push will look artificially weak.

You’ll get cleaner data by limiting enrollment to contacts who:

• Are subscribed to marketing email

• Have at least some history with web push (so we know they’ve granted notification permission)

• Have just hit the upgrade moment

Create a saved segment in Spreeflo using the segment builder:

• Email Subscription Status:

• - is “Subscribed”

• Web Push Activity:

• - delivered at least 1 time over all time (this is a proxy for “has a valid push subscription”)

• Custom Events:

• - event name upgrade_modal_shown

• - triggered at least 1 time in the last 1 day

Connector: AND all three rules.

Name it something like Upgrade modal – dual-channel reachable.

This segment now describes who should enter the experiment. The journey handles what happens once they do.

Step 3: Use a Join Segment trigger to start the journey

In the journey builder (see build a journey for a quick refresher), drag in a Join Segment trigger as your starting node.

Configure it:

• Segment: Upgrade modal – dual-channel reachable

• Re-enrollment: Off

Why Join Segment, not a Custom Event trigger?

• Join Segment lets you encode your “reachable on both channels” logic once in the segment, then reuse it.

• With re-enrollment off, each contact only goes through this experiment once, even if they hit the upgrade modal multiple times.

From this trigger, every new contact who becomes email-subscribed, has a valid push subscription, and sees the upgrade modal will immediately enter the journey.

Step 4: Randomly split contacts between email and push

Next node after the trigger: Random Split.

Configure it:

• Percentage to Path A: 0.5 (50%)

• Path A label: email_arm

• Path B label: push_arm

The Random Split node in Spreeflo sends each contact down one of two paths based purely on probability, with no regard to attributes or behavior. That’s what makes this a valid A/B-style test.

You now have two arms:

• Arm A: contacts will get an email upgrade nudge

• Arm B: contacts will get a web push upgrade nudge

Before you send anything, you want to tag who landed where.

Step 5: Tag each arm for clean reporting

On the email arm:

1. Add an Add Tag node right after email_arm.

2. Tags: add channel_test, channel_email.

On the push arm:

1. Add another Add Tag node right after push_arm.

2. Tags: add channel_test, channel_push.

You now have two mutually exclusive groups:

• Everyone with channel_test + channel_email received the email variant

• Everyone with channel_test + channel_push received the push variant

These tags are what you’ll use later to build reporting segments like “email arm who upgraded” vs “push arm who upgraded.”

Step 6: Send the upgrade nudge via each channel

Now we actually talk to people.

Email arm: Send Email

After the Add Tag on the email branch, add a Send Email node.

Key choices:

• Template: create a dedicated “Upgrade after limit reached – Email” template using Spreeflo’s email builder.

• From identity: pick the product or founder address you normally use for product emails.

• Send only once: Leave this on. Even if you later reconfigure the journey, this guards against accidental double-sends.

Copy guidance:

• Subject: Make the benefit concrete, not just “Upgrade now.” For example:

• - “Keep tracking all your keywords (and avoid stale data)”

• Body: Reference the exact moment:

• - “You just hit the 50-keyword limit in RankPulse.”

• - Show what they lose if they stop here (frozen rankings, gaps in SERP coverage).

• - Show what they gain on the paid tier (unlimited keywords, more frequent checks, API access, etc.).

• One clear CTA button to the upgrade page.

Push arm: Send Web Push

On the push branch, after its Add Tag, add a Send Web Push node.

Pre-requisites:

• Web push is a Professional-plan feature, and requires the SDK + service worker configured (the web push setup guide walks through this).

• You already have a permission strategy in place (bell icon or dialog); remember, this experiment only runs for contacts who have previously allowed notifications.

Configure the notification:

• Title: Short and direct. “You’ve hit your RankPulse keyword limit”

• Body: One line of value. “Upgrade now to keep tracking every keyword without gaps.”

• Click URL: direct link to your billing or upgrade page.

Toggle “send only once” on here too.

Because web push is more interruptive than email, keep this a single concise nudge. You’re not building a multi-step sequence; you’re comparing channels for this one moment.

Step 7: Merge paths, wait for conversions, and tag outcomes

After each Send node (email and push), point both branches into a Merge node. This keeps the graph valid (only Merge nodes can have multiple incoming edges) and lets you share the same measurement logic for both arms.

From the Merge, add a Wait Condition action.

Configure it:

• Condition: Custom event upgraded_plan triggered at least 1 time in the last 3 days

• Timeout: 3 days

Effect:

• If the contact upgrades within 3 days of your nudge, they’ll leave the Wait Condition as soon as that event arrives.

• If they don’t upgrade, they leave the Wait Condition after 3 days when the timeout expires.

Next, add an If/Else node right after the Wait Condition, using the same condition:

• Condition: Custom event upgraded_plan triggered at least 1 time over all time

• Branches:

• - Then branch: “converted”

• - Else branch: “did_not_convert”

Why check “over all time” here?

Because by the time they exit the Wait Condition (either on event or timeout), anyone who has upgraded will have that event on their record, regardless of exact timing.

Finally, on each branch:

• Converted branch:

• - Add Tag node with tag channel_test_converted.

• Did-not-convert branch:

• - Add Tag node with tag channel_test_not_converted.

At this point, the journey ends. No extra emails, no extra pushes. You’ve simply recorded which contacts saw a nudge, via which channel, and whether they converted within your test window.

Step 8: Read the results: conversions, cost, and opt-outs

Once the journey has run for a while (you’ll want at least a few dozen conversions per arm for the numbers to mean anything), use tags and segments to answer three questions.

1. Which channel drove more upgrades?

Create two analysis segments:

• “Email arm, converted”:

• - Tags contains channel_test

• - Tags contains channel_email

• - Tags contains channel_test_converted

• “Push arm, converted”:

• - Tags contains channel_test

• - Tags contains channel_push

• - Tags contains channel_test_converted

Compare:

• Count of contacts in each segment (absolute conversions)

• Conversion rate:

• - Email conversion rate = email_converted / total channel_email

• - Push conversion rate = push_converted / total channel_push

You can get the denominators with two more simple segments:

• “Email arm, all”: channel_test + channel_email

• “Push arm, all: channel_test + channel_push`

2. What did it cost you per conversion?

Because Spreeflo pricing is contact-based rather than per-send, your “cost per conversion” here is mostly about your internal sense of effort and any external media you tie in.

For a lean comparison, just estimate:

• Email:

• - Time spent writing/testing the template

• - Any incremental design work

• Push:

• - Setup time for web push (one-off)

• - Time spent on the notification template

On a continuing basis, both are effectively free to run. If push converts significantly better and takes less creative effort to iterate, that’s a strong signal to favor it for similar moments.

3. Did one channel harm list health more?

You don’t want a “high-converting” channel that burns trust.

Check:

• Email unsubscribes: in Spreeflo’s email analytics for the “Upgrade after limit reached – Email” template, look at unsubscribe and spam-complaint rates.

• Push fatigue: use a segment like “channel_push + channel_test + web push dismisses at least X times in last 30 days” to see if push-arm users are dismissing more often or if you see a rise in “blocked” status in browser analytics.

If push converts slightly better but drives much higher opt-outs, you might choose to reserve it for only the highest-intent events. If email underperforms but keeps your list cleaner, that’s useful context too.

What to do with the insight

Once you have data, you can adapt the journey in a few ways:

• If email wins clearly:

• - Keep this experiment running, but start designing a second test inside the email arm: subject A vs B, or short vs long copy.

• - Consider using push as a backup channel only for non-openers, via a separate journey keyed off a Check Email Activity process.

• If push wins clearly:

• - Flip this journey into a “push-first” strategy and test email only for those who don’t have a push subscription.

• - Look for other “micro-moments” in your app where a timely notification would help (e.g., technical audit completed, new keyword dropped off page one).

• If it’s a tie:

• - That’s an answer too. It may mean channel preference is user-specific. The next test might be audience split by behavior (e.g., daily active users vs occasional logins) rather than random.

The point isn’t to crown one channel forever. It’s to remove guesswork at specific points in your funnel and keep iterating with real numbers.

Why this experiment is classic founder leverage

You built:

• One segment to describe “upgrade-ready and reachable on both channels”

• One journey with a few nodes: Join Segment, Random Split, tags, two send actions, Merge, Wait Condition, If/Else

From now on, every user who hits that upgrade moment becomes part of an ongoing experiment you don’t have to think about. You get a steady stream of learning:

• How responsive your audience is to web push vs email

• What that responsiveness costs you in unsubscribes or fatigue

• Where to focus your future creative energy

That’s the real advantage for a small SEO SaaS team. You don’t need a growth squad to run controlled tests. You need a clear question, a simple journey like the one above, and a tool that makes it easy to wire behavior into messaging.

Set this up once in Spreeflo, let it run for a few weeks, and you’ll know far more about how your customers actually prefer to be nudged than most of your competitors. Then you can decide, with confidence, where your next hour of marketing work should go.