Feature Preview Deployments for the Front-end

The front-end for “Employee Referrals” — the department where I work — at Radancy is a single page web application, served as static assets by nginx, which also acts as a reverse proxy.

The problem

Historically, the front-end team at “Employee Referrals” has followed a practice of working on features in long-running git branches, iteratively adding small changes to the branch until the feature is complete, at which point they merge it. While this (somewhat) worked when these front-end developers were part of a dedicated and siloed team, it doesn’t really work well with our new team structure — cross functional teams where front-end and back-end developers work together on product features. The need to wait for merging these long-running branches to the main branch before seeing the corresponding features on our pre-prod environment qa resulted in significant delays. This meant that it took quite a long time before other team members — product owners, testers, other devs — could test out new features and provide feedback. I wanted to get rid of this delay by allowing previews for front-end feature branches.

These feature previews needed to be easy to use with very low friction for everybody involved. This is the workflow I ended up implementing.

The solution

Each team that contributes to the front-end codebase gets a few “deployment slots”, named in a certain way, eg. feature-w-1, feature-w-2, feature-r-1, feature-r-2, and so on. The w and r in the slot name stands for the team name. A developer in these teams needs to prefix their git branch with the name of the feature slot, eg. feature-w-1/make-scrolling-ultra-smooth. Once this branch is somewhat ready to be previewed by other team members, the developer needs to create a pull request with this branch, which will then trigger a deployment of the codebase in the branch.

Team members who need to preview this feature need to have a browser extension installed, one that injects a custom header to all requests made to a certain domain. They can choose a deployment slot by simply changing the value of this header to the name of the deployment slot.

How it works behind the scenes

The normal setup

In the production setup — one without this feature preview functionality — nginx runs as a fargate task on AWS ECS, getting traffic from an Application Load Balancer, which itself acts as an “Origin” for a Cloudfront distribution. The CICD pipeline (on Jenkins) for this repository builds docker images and pushes them to dockerhub, which are then used for the ECS task definitions. Fargate service updates are handled by cloudformation — Jenkins just runs an aws cloudformation update-stack command at the end of the pipeline.

The web application can also be served on a custom domain (owned by a customer) by having it be served by a dedicated Cloudfront distribution that has a CNAME or ALIAS DNS record pointing to it. However, this means that feature preview deployments need to work with custom domains as well.

Infrastructure for feature previews

For the feature preview deployment setup, an ECS fargate service is created for each deployment slot, which is supposed to run an ECS task with a container running the codebase for the git branch corresponding to that slot.

Routing rules are added to the Application Load Balancer on qa, which route based on the value of the header X-1brd-Feature to the target group associated with that deployment slot. The desired count of the service for a deployment slot can be set to zero if it’s not needed.

As mentioned above, our web application — irrespective on whether it’s on a custom domain or our organisation domain — is served by Cloudfront. This HTTP header based approach works with the existing Cloudfront setup as it can simply forward the X-1brd-Feature header to the ALB. The only extra configuration needed on the Cloudfront distributions is to include the header in the cache policies being used.

Continuous delivery for feature previews

Jenkins runs a separate pipeline for pull requests where the change branch matches the regex for the feature names.

// Jenkinsfile

when {
    expression {
        return env.CHANGE_BRANCH =~ /^feature-[wr]-\d{1,2}\//
    }
}

The docker image built in this pipeline is tagged with a string comprised of the branch name, and the build number.

// Jenkinsfile

environment {
    CHANGE_BRANCH_DOCKER_SAFE = env.CHANGE_BRANCH.replaceAll('/', '-')
    RELEASE_VERSION = sh(script: "echo \"$CHANGE_BRANCH_DOCKER_SAFE-$BUILD_NUMBER\"", returnStdout: true).trim()
}

This pipeline then updates the stack that manages the feature deployments ECS cluster, only updating the service that corresponds to the feature deployment in consideration.

Future

The primary idea behind adding feature previews is to increase visibility in the features our teams are building, and enabling feedback loops so bugs/discrepancies are caught early in the software development lifecycle process. As this workflow is new in the whole “Employee Referrals” department, we’ll have to proactively make use of preview deployments to get the most out of them.