Chapters
Chapters

Chapters

Hide chapters

Real-World Flutter by Tutorials

First Edition · Flutter 3.3 · Dart 2.18 · Android Studio or VS Code

Before You Begin

Section 0: 6 chapters
Show chapters Hide chapters

Real-World Flutter by Tutorials

Section 1: 16 chapters
Show chapters Hide chapters

13. Running Live Experiments With A/B Testing & Feature Flags
Written by Vid Palčar

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

Heads up... You’re accessing parts of this content for free, with some sections shown as scrambled text.

Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now

In the previous chapter, you learned about two Firebase services — Firebase Analytics and Firebase Crashlytics. While browsing through your Firebase console, you might’ve noticed that it offers quite a few other interesting services. So, in this chapter, you’ll look at two of those additional services — Firebase A/B Testing and Firebase Remote Config.

After thousands of hours spent learning good UX and UI practices, studying marketing and working on product development, there will always be situations when you won’t be sure about a specific design decision, call-to-action strategy or advertising text. Even when you think you made the right choice based on your experience, there might be cases when taking a different approach would work better. You might think these differences are irrelevant to your app, but the truth is that an increase of only a few percentage points in user engagement might reflect enormous gains in your sales. The math behind that is quite simple: more sales = more money.

But now, you might be wondering how to determine which solution to a specific problem will work best for your users. That’s where Firebase A/B Testing comes in handy. Publishing a new release of the app for every change required for your experiments would be very inconvenient. Firebase Remote Config has you covered there. With the help of a so-called feature tag, you can make changes to your app on the fly.

With Firebase A/B Testing, you can run experiments in marketing, advertising, politics, product development, design, etc. As mentioned already, you’ll have to use Firebase Remote Config to implement those.

In this chapter, you’ll learn how to:

  • Add a feature flag to a specific feature.
  • Perform changes in the app without rolling out a new version.
  • Set up an experiment in Firebase.
  • Perform and track experiments with Firebase.

While going through this chapter, you’ll work on the starter project from this chapter’s assets folder.

A Deep Dive Into A/B Testing

The introduction already mentioned a few use cases of A/B testing. But before performing your first A/B test, you have to dive a little deeper into the theory behind it.

A/B testing is a simple experiment in which two different versions of a variable — A and B — are compared. The testing group is randomized and usually equally distributed between both samples. More complex cases might have more than just two versions of a specific variable. The use of A/B testing started far before the era of modern computers. Originally, A/B tests were used to figure out which advertising pamphlets worked best. Nowadays, A/B tests are standard procedure in product development as well as preparation of marketing materials — from small businesses to tech giants.

Here’s how A/B testing works in the example of a simple call to action for a shoe store app. You start by preparing two versions of the call-to-action button text. The first one says: “Only five pairs left”, and the second one says “Limited number of pairs left”. After that, you have to clearly define the metrics you want to track for this experiment. You’ll figure out why this is very important in that stage. For the sake of this example, say that you want to find out which text attracts more users to buy the product — this becomes the metric you want to track. Finally, you randomly distribute the two versions of the app amongst your users — 50% will use the first version of the app with the first text, and the other 50% will see the second one with the second text.

After performing the test for two weeks, you’ve noticed that 7% of the users with the first version of the app ended up buying the shoes. On the other hand, only 5% of users with the second version of the app made a purchase. The result makes quite a bit of sense. Users of the first version stress more about shoes going out of stock than the users of the second version; therefore, they’re more motivated to buy. Based on that information, you clearly see that the first call-to-action text works better, and you can distribute it to all the users.

On the other hand, you might want to perform the same experiment and focus on slightly different metrics. If you want to measure the use time of your app, the result might be very different. Because the users of the second app version don’t feel such an urgency to buy the product, they might spend a little bit more time scrolling through other products.

The example above was a very simple one, with only a slight difference between the two samples. But don’t let this example fool you. In slightly more complex A/B tests, you could test the performance of two or even more completely different UIs and features, and measure multiple different metrics.

The last thing you need to understand before continuing with this chapter is the aspect of segmentation of the responses in a specific test. Even though, in the previous example, the first version of the variable — the call-to-action text — worked the best, this might not be the case for some subgroups of your testing group. For example, for a specific age group or gender, the second version of the variable might work significantly better. You must take this into account when you target your audience — you should distribute the second version of the call-to-action text to this specific subgroup once the test is finished.

Logic Behind Remote Config and Feature Flags

As already mentioned, A/B testing and Firebase Remote Config go hand in hand. The second is normally used only as a tool to correctly distribute the different variables among the users when performing A/B tests. This will also be a case in the example for this chapter, although it’s only fair to mention other use cases of Remote Config.

Installing Firebase Remote Config

Time to get your hands dirty. To add Firebase Remote Config, you’ll use the firebase_remote_config package. Locate pubspec.yaml in the packages/monitoring package:

Creating New Parameters in Firebase Remote Config Console

Now that you’ve successfully added Firebase Remote Config into your app, you can start adding the parameters to change the app’s appearance.

Implementing UI Changes Based on Your Remote Config

First, navigate to the app’s main.dart file located in the app-level lib folder. Find // TODO: add loading feature flags from remote config, and replace it with:

Performing A/B Tests

By implementing Firebase Remote Config, you have control over the appearance and behavior of your app in the sense of predefined parameters. But now, your app has multiple different appearances, and you may be wondering which is the best for the end user.

Key Points

  • A/B tests are crucial for making your app more suitable for its end users.
  • Firebase Remote Config allows you to perform changes in the app without needing to release a new version.
  • By performing different A/B tests on your app with the help of Remote Config, you may be able to make better decisions about your app’s appearance and behavior.
12. Supporting the Development Lifecycle With Firebase
14. Automated Testing
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2026 Kodeco Inc.

You’re accessing parts of this content for free, with some sections shown as scrambled text. Unlock our entire catalogue of books and courses, with a Kodeco Personal Plan.

Unlock now