Ever wondered how restaurants digitize their menus on delivery apps? You might think that restaurant owners manually enter each item on their menu into the app - well, that's how it used to be done. With Woflow, delivery companies and merchants can instead make a simple API request to Woflow’s ML engine to automatically turn their menu into a fully digitized catalog. Menu data is just one example - Woflow also helps companies in other industries, such as e-commerce or POS, turn unstructured data into a structured format in just a few seconds.

‍

For a company like Woflow, robustness of their API is absolutely critical. Porter allows Woflow to deliver these APIs at scale in a reliable way, while vastly simplifying the deployment process.

Starting on Heroku

In the beginning, Woflow set up their servers on Heroku and their ML infrastructure on AWS SageMaker. As Woflow started to scale, however, they soon realized that Heroku is not a feasible long term solution due to vendor lock-in and rapidly increasing costs. The engineers at Woflow had zero visibility into the underlying infrastructure, which meant that there was no way to customize beyond the rigid guardrails that are set up by Heroku. Woflow had started to run into these limitations as their stack evolved. Below are few of the limitations they ran into:

• Installing custom libraries, such as imagemagick, required them to write their own buildpack or rely on Heroku’s unofficial buildpacks that did not work very well.
• It was impossible to install more than a few of these custom libraries because Heroku imposes a maximum image size limit of 500MB.
• Autoscaling on Heroku relied on a metric that cannot be customized (i.e. application response time). This default autoscaling policy often resulted in over-provisioning of resources, which was worsened by the fact that autoscaling was only available on Performance tier dynos that are larger and more expensive.
• Securely connecting the two separate infrastructures on AWS SageMaker and Heroku within the same VPC was possible only by upgrading to Heroku Private Spaces, and even then it was challenging to keep low latency between the two infrastructures.

On top of that, getting locked into Heroku’s ecosystem also meant that Heroku could charge exorbitant costs - Woflow’s bill on Heroku started to increase exponentially as they added more dynos, which amounted to more than 10x the amount that the same computing resources would have cost on AWS.

‍

Having no insight into how their code is run also raised some concerns for the future. For example, delivering an on-premise version of Woflow’s service would have incurred significant amount of overhead as engineers would have had to determine the optimal infrastructure setup from scratch.

Move to AWS ECS

Once Woflow realized that Heroku is not a platform they could stay on for the long term, they decided that it’s better to migrate sooner than later because an early stage of the company is the least risky time to migrate. Moving off Heroku was largely a preemptive decision, but being proactive about the inevitable obstacles in the future turned out to save Woflow a hairy migration later.

The first choice for the new infrastructure was Elastic Container Service (ECS) on AWS. Andrew Harsata, a senior engineer at Woflow who has some familiarity with DevOps, set up Woflow’s production servers on ECS using Terraform. While this setup worked well for Andrew, it introduced a major slowdown for other engineers in the team who had zero experience with Terraform and DevOps in general.

‍

After a few weeks on ECS, engineers at Woflow had started to miss the convenience of Heroku and its user friendly interface. Being the only person with the knowledge and experience to navigate ECS and broadly AWS also meant that the responsibility of managing and troubleshooting deployments fell entirely on Andrew, which resulted in a serious development bottleneck. With the new setup, for example, engineers had to ask Andrew for help to change even a single environment variable. Debugging an application issue in production also became difficult for the engineers who had no visibility into application logs and metrics. Engineers at Woflow had neither the desire nor bandwidth to learn all the concepts and tooling around the infrastructure - simply put, they just wanted to write code. To address this bottleneck, Andrew even planned to develop a set of internal tooling to abstract away the complexity of Terraform and ECS, which would have taken months to build and maintain in-house.

Enter Porter

Woflow needed a way to give engineers the convenience of a PaaS like Heroku without compromising the flexibility of owning the underlying infrastructure. Porter hit the sweet spot by providing them a Heroku-like abstraction layer on top of their own AWS infrastructure that they fully own and control. Porter allows engineers at Woflow to painlessly deploy and troubleshoot their applications without having to wrangle with Terraform or AWS

‍

Just like they used to on Heroku, Woflow engineers now simply push to the repository to deploy to production. In a few clicks, they can update environment variables, view application logs and metrics, and rollback to a specific deployment version from Porter’s user friendly dashboard.

Not only did Porter restore the convenience of Heroku, it also equipped Woflow with features that did not exist on Heroku. On Porter, deployments self-heal upon failure and ensure that a minimum number of replicas are running at all times. Each deployment can be dynamically resized and scales automatically based on the target usage you set for both CPU and memory. It also grants Woflow granular control over the networking layer, allowing them to set their own send/read/connect timeout values and spin up their own load balancers. In terms of reliability, Porter allows Woflow to perform zero downtime updates with custom health check endpoints. And unlike Heroku whose dynos are strictly ephemeral, it is also possible to attach persistent disks to the application to preserve the file system in the case of application crash.

Under the hood, Porter provisions a Kubernetes cluster that is production-grade out of the box. Then it abstracts away all the complexity of Kubernetes so that you can deploy and manage applications on it without any overhead. In essence, Porter allows Woflow to take advantage of the expansive feature set and ecosystem of Kubernetes with the convenience of Heroku.

‍

In terms of application setup, Woflow runs three web servers along with two worker processes on Porter. Their applications connect to a Postgres database that’s set up on AWS RDS, which is connected to Porter via VPC peering. Woflow’s CI/CD is handled by GitHub actions that run a set of Porter CLI commands, along with customized steps that trigger a database migration before prompting a deployment. In addition to these long running workloads, Woflow also runs two categories of jobs on Porter: one-off jobs that execute queries against the database (e.g. database seeding or migration) and cron jobs that periodically kick off third party integrations on a schedule.

Woflow manages three different environments (development, staging, and production) on Porter, each in its own Kubernetes cluster, and has a git branch linked up to each environment that deploys on every push.

Plans for the Future

Woflow wants to focus on the technical aspects of the company that have clear business implications. They want to avoid investing in infrastructure as long as they can, unless it is necessary to do so. That said, they realize that it may become necessary to hire infrastructure engineers as the company continues to grow. By setting them up on Kubernetes and enforcing best practices from the get-go, Porter prepares Woflow for mature DevOps work from day 1.

‍

When infrastructure engineers join the team in the future, they will be able to customize Porter’s underlying Kubernetes cluster without any overhead while preserving the Heroku-level convenience for the rest of Woflow’s engineers.

If you liked this article, share it on Twitter, HackerNews, LinkedIn or Reddit.