Running Shiny in AWS Fargate

In the previous post we outlined the architecture of a dashboard framework to run dashboards based on multiple technologies including Shiny and Flask in production. We will now show how to run a basic Shiny dashboard in AWS Fargate behind an Application Load Balancer in less than 60 lines of CDK code. To define our stack in a reproducible manner we will make use of the Amazon Cloud Development Kit (CDK) with Typescript. Starting from a basic CDK stack we now specify the most important components of our stack:

1. The Application Load Balancer (ALB) to route traffic to our dashboards.
2. The Fargate cluster to run our dashboard tasks in a scalable manner.

The deployed stack will finally run an example Shiny dashboard behind an Application Load Balancer. Note, that the resulting stack will only run one dashboard without encryption. We’ll implement these features as part of the next post. The resulting CDK code can be also downloaded from Github at https://github.com/quantargo/dashboards.

Prerequisites

To run the following code examples make sure to have

1. an AWS Account
2. a locally configured AWS account by running e.g. aws configure with the aws CLI
3. a local Node.js installation (version >= 14.15.0)
4. Typescript: npm -g install typescript
5. CDK (version >= 2.0): npm install -g aws-cdk

Initialize CDK and Deploy first App

To initialize a sample project we first create a project folder and within the folder execute cdk init:

This command creates a new CDK Typescript project and installs all required packages. The following 2 files are relevant for stack development:

• bin/dashboards.ts: Main file which initializes CDK stack class. You can explicitly set the environment env if you use a different account or region for deployment.
• lib/dashboards-stack.ts: CDK Stack class to which all components of our stack will be added.

Specify Application Load Balancer (ALB)

Next, we need to create a Application Load Balancer (ALB) within a new VPC which is responsible for secure connections and routing. We create a new VPC and add an internetFacing load balancer to it. This means that the load balancer will be accessible from the public internet and will therefore be placed into a public subnet. Within the lib/dashboards-stack.ts file we put the following lines:

Specify Dashboard Cluster

Next, we need to add an ECS cluster to our VPC to run our dashboards efficiently:

Add First Fargate Task Definition

We can now add our first Fargate dashboard to the cluster by specifying a task definition. We use the rocker/shiny Docker container as an example running on port 3838. This also requires respective port mappings in the container definition. Additionally, we use half a virtual CPU ( 512)- 1024 would equal a full one-and a memory size of 1024 MiB. By specifying the Fargate service we are already finished with the specification to run our first container in the cluster:

Put Service Behind ALB

Next, we put the Fargate service into an ALB target group so that traffic can be routed through the ALB:

Note, that we added 2 parameters to the ALB target group definition:

• stickinessCookieDuration: Since Shiny sessions are stateful we need to prevent the ALB to switch instances (in case there are more) during a session. The session duration set to one day should be sufficient.
• healthCheck: The health check needs to specify the port (as string) and set to the container port 3838, as well.

Finally, we add an HTTP listener which directly forwards all incoming traffic to our dashboard:

Deploy

Before deployment you should also bootstrap your CDK environment:

Now the stack should be ready for deployment. As an extra step, you can now check if the stack can be successfully synthesized using

Any errors popping up during cdk synth need to be fixed immediately. By continously using cdk synth we make sure that the feedback cycles during development are as short as possible. If cdk synth is successful we can now run

Finally, you should see the successful output message including the DashboardsStack.LoadBalancerDNSName which you can directly access through the browser:

Conclusion

We could show how to run your first basic Shiny dashboard behind an Application Load Balancer in very few lines of CDK Typescript code. In the next post we will cover end-to-end encryption through SSL/TLS and host-based routing to add multiple dashboards to the ALB.

Make code, not war! ✌️

Originally published at https://www.quantargo.com on March 11, 2022.