The Anatomy of a Strategic Beta Program

In a previous post, we wrote about the main goals of a Beta Program (and what the goal shouldn’t be). Through that lens, I’d like to expand on Beta Programs and talk a bit about how to strategically plan out the program so that it’s of maximum value.

As a quick recap to the previous post mentioned above, I like to think about 3 main goals with a limited release Beta Program:

1. Generate powerful marketing buzz (i.e., generate early client success stories and metrics to maximize GA marketing and adoption)
2. Enable sales and support (i.e., get your CS and Sales teams involved to prep for value proposition, support and pricing conversations)
3. Optimize first impressions (i.e., clean up small bugs and performance issues that can make a big first impression difference once released to the masses)

Now that we have the what, we can talk about the how. The worst thing you can do is waste a very valuable beta program opportunity by not putting a plan in place that will optimize results. Here’s a process that I’ve used successfully with companies in the past to ensure that the beta program is well coordinated and drives high value.

Recruitment

Timing: ~3 – 4 weeks before program start

Put together a list of high potential beta clients depending on the feature in question. Think about the types of customers / users who will benefit most from this innovation and the type of feedback you’ll need in order to achieve your goals. Work with your CSMs to approach the target list to get them signed up and bought in. *If applicable, I’d recommend staying away from having direct competitors in the same beta program.

The email “ask” to the prospective beta client could go something like this…

“We’ve been working hard to innovate our <feature area> capabilities and we’re excited to offer a free, limited beta program starting soon around a new offering. You have been pre-selected as one of our customers that we feel could greatly benefit from this new capability because of the nature of your business and product catalog. You’ll be one of the first ones to start benefiting from this new innovation and have a voice in optimizing the feature the rest of the way. For us, we get to work with you to get an early read on value, impact and of course, your valuable feedback. If you are interested in this opportunity, please let us know by <date> and we will be in touch with next steps!”

There might be some tweaks that you want to make, but that’s the general idea. Note the use of “limited” and “pre-selected” terms to make the client feel more compelled to opt in.

For those that opt-in, send out an invite email for a short kickoff meeting and a link to the pre-beta survey (if applicable, see below) that needs to be filled out before the kickoff meeting.

Internal Prep

Timing: ~2 weeks before program start

The first step of course is to ensure that the Product and Engineering teams are feeling good about the state of the beta version and set up to monitor key performance indicators.

It’s also crucial that beta clients have streamlined ways to provide feedback.

If possible, set up a mechanism within the app to facilitate “in the moment” feedback. It’s also good to provide feedback options outside the app such as a dedicated beta feedback email address and / or a beta participant Slack channel. Depending on the industry you’re in (and propensity for knowledge sharing), a Slack channel may also be appealing to beta clients for discussing experiences, ideas for improvement, etc.

You’ll also want to prep CSMs and Support so that they’re prepared to field questions and feedback from clients if it comes through them.

Finally, don’t forget to equip the Sales team to field questions about pricing (only for features that are an extra cost).

Pre-Beta Survey

Timing: ~1 week before program start

This is an optional step based on whether qualitative feedback makes sense to use as a “before and after” comparison for assessing initial value impact. For instance, you might be trying to improve difficult-to-quantify “ease of use” with the new innovation. Or, you may be trying to drive ultimate value in an area not quantifiable within your software. Note that this is not a replacement for quantitative success measures that can be collected automatically through behavioral analytics, conversion tracking, etc.

With this in mind, craft a survey that aims to give you a current benchmark on the value areas that you’re trying to impact. For instance, if one of the proposed value areas is “make the client dashboard easier to navigate efficiently” then you might ask a pre-beta question like

“On a scale of 1-10 (10 being easiest), how would you rate the ease of navigation in the current client dashboard?”

It’s also a good idea to ask a few segmentation type questions up front to help segment out the results based on client profile afterwards.

Filling out the survey should be a prerequisite for the client to become a beta client, which also tends to have the effect of buy-in and engagement right up front. 

You’ll take a post-beta reading using the same survey to get a “before and after” read of impact based on these qualitative metrics.

Beta Kickoff

Timing: At Program Start

For each beta client, I recommend having a beta kickoff session that introduces the high level innovation, the goals and expectations of the beta program. It’s especially important to call out and get buy-in from the client on being engaged and active during the beta program.

The 30 minute (or less) kickoff call should roughly cover the following areas:

• Quick overview of their business and current challenges (hopefully matching up with the new beta offering)
• Overview of the beta program, goals / expectations and why it’s valuable to both sides
• Target problem(s) that are being addressed with the new innovation
• Quick demo of the beta version
• Specific areas where the Product team is seeking feedback
• How client gets access (have them verify access on the call if possible)
• How client provides feedback
• Soft pre-agreement that the client is willing to partner on a success testimonial to be used in GA go-to-market if they see good upside from the new innovation (if possible)
• Timing for next touchpoint (best to align on a date during the session)

Beta Feedback Period

Timeline: Minimum 4 weeks

The timeline of the actual beta program feedback period is something you’ll need to assess based on your needs. I’ve found that between 4 and 8 weeks is typically sufficient to meet the goals. But, you may have a reason to extend that timeline. The timeline may also need to be extended depending on the amount of beta feedback and updates deemed necessary prior to GA. 

During the beta feedback period, I’d recommend having at least one touch point with each of your beta clients (timing hopefully agreed up on at the kick-off meeting) to get their general feedback / thoughts and field any outstanding questions. 

You’ll also want to continuously monitor and manage beta feedback that’s coming in from the different feedback channels. There will surely be some feedback and bugs that prompt immediate improvements and updates. The rest should be prioritized accordingly.

For beta clients that are seeing great results, start partnering with your Marketing team to start the process of getting client buy-in on a testimonial or success story that can be used during GA launch. For testimonials, a good tactic is to use the feedback and context from your touchpoint to craft a few possible testimonials that the client can simply sign off on when ready.

Post Beta Survey

Timing: Near the end of the program

If a pre-beta survey was administered, craft a similar survey that aims to extract the “after” qualitative stats for comparison to the “before” benchmarks to draw conclusions on value impact. From these stats, you can hopefully extract some great customer value indicators and free form testimonials to use in your full scale go-to-market efforts.

Beta Program Review

Timing: upon program conclusion

The program review step is super important in synthesizing the learnings into an action plan and sharing with the team, and even full company. The review should  cover points like:

• Quick overview of the new innovation
• Goal(s) of the beta program
• Overview of the beta process
• Customers involved
• Success measures
• Results / key learnings / feedback
• Proposed next steps

From the review step it should be pretty clear on whether you’re ready for full release or if it makes more sense to make improvements and extend the beta period a bit longer.