Skip to content

A/B Testing Deployment with CNO Deploy

This deployment model releases two versions of the application simultaneously in production environments:

  • The first, A, is unchanged and is automatically deployed on the market.
  • The second, B, is only available to a defined customer sample.

The first version (A) is called “Control” and will be used by all users except those assigned as testing users. The second version (B) is called “Test”, and it will be used only by testers so that they can evaluate if there are any improvements made in either one version or another based on their feedback.

After monitoring feedback, It is up to you to choose the right moment to release the B version for all users.

Deploying the A/B Testing Strategy from CNOCTL

Run any of these commands to use the A/B Testing strategy:

cnoctl deploy abtesting <WorkloadName> -p <project name> -e <environment name>
Supported Flags:

-e or --environment: The environment name.

-p or --project: The project name.

--header: This refers to the key,value pair. The header format is key=value.

--image: The docker image of your application. It should be in /. When using the image flag, the version flag is required.

--version: the version of your application docker image i.e the tag. When using the version flag, the image flag is required.

--go-to-live:

There will be two versions of the application running.

cnoctl get workload --

The result should look like this:

NAME STRATEGY REPLICAS IMAGE AUTO-DEPLOY LIVE STAGING STAGING-TRAFFIC HEADERS
workload Name ab/testing replicas Number username/workloadName true/false V1.0.0 v1.0.0 0

Deploying the B Version using CNOCTL

To deploy the B version of your application. Run the command:

cnoctl deploy abtesting my-workload --image <image name> --version v2.0.0 -p projectName -e environmentName
NAME STRATEGY REPLICAS IMAGE AUTO-DEPLOY LIVE STAGING STAGING-TRAFFIC HEADERS
workload Name ab/testing replicas Number username/workloadName true/false V1.0.0 v2.0.0 0

Setting the Header using CNOCTL

You can configure your application to collect and receive any type of data you want to be the deciding factor of your B version users, e.g location, browser, browser language,etc. Supply the value to CNO as a key=value pair, and CNO will reroute all traffic that satisfies that criteria to your B version.

E.g

cnoctl deploy abtesting my-workload --header key=value  --project projectName --env environmentName
Have a look at your workload
cnoctl get workload --project projectName --env environmentName
Result:

NAME STRATEGY REPLICAS IMAGE AUTO-DEPLOY LIVE STAGING STAGING-TRAFFIC HEADERS
workload Name ab/testing replicas Number username/workloadName true/false V1.0.0 v2.0.0 0 browser=chrome

Going Live with CNOCTL

When you’re confident in your B version and you wish to officially release it to your entire user base. You can just run the command to replace your A (old)version with your B (new)version.

cnoctl deploy abtesting my-workload --go-to-live -p projectName -e environmentName
Let’s have a look at your workload
cnoctl get workload --project projectName --env environmentName
Results:

NAME STRATEGY REPLICAS IMAGE AUTO-DEPLOY LIVE STAGING STAGING-TRAFFIC HEADERS
workload Name ab/testing replicas Number username/workloadName true/false V2.0.0 v1.0.0 0 browser=chrome

Deploying the A/B Testing Strategy from CNO UI

We can also deploy the and manage the A/B Testing Deployment through the CNO UI.

Go to Deploy > Continuous Deploy

You’ll see all workloads. Tap the Edit button for your workload.

Click “Strategy” and choose A/B testing from the Dropdown.

Deploying the B version

You can also choose the B version of your application by scrolling down and choosing the version you want from the dropdown in “Staging”.

Setting the Header using CNO UI

Click the “Add Value” at the bottom of the form. Fill in the Key, Value fields.

Finally, click "Update" and congrats on deploying your workload with AB Testing Strategy.

Going Live with CNO UI

When you're completely satisfied with the new version of your application and you wish to push it live. Select your image deployment "Go To Live" and you'll be taken to the Go-to-live page of your workload. Simply click the button in front of your staging application link.

You'll be taken to a prompt page, confirm and your staging application will be pushed live while your former live application will be pushed to the staging environment.


Last update: 2022-12-06