Onclusive supercharges PR and Communications teams by leveraging data science and artificial intelligence to provide them with advanced media analytics insights. They serve a wide variety of clientele, including Kellogg’s, Vodafone Germany, the National Museum of Natural History, KPMG, and thousands of other organizations. We spoke with Derek Gavy, Lead Engineer at Onclusive, to learn more about how the company utilizes Porter to aid their application development. 

Starting on Heroku 

‍Initially, Onclusive used Heroku to run their applications. However, these applications had significant scaling requirements, specifically high memory requirements. On Heroku, the containers within which applications consume host resources are called dynos. These dynos are offered at different tier levels, with varying amounts of resource availability. The Performance-M dyno, for example, allows for up to 2.5 GB RAM usage whereas the Performance-L dyno allows for 14 GB RAM usage. Onclusive has many services with higher memory usage of around 8 GB RAM; these applications necessitate the use of Performance-L dynos although they do not need all 14GB of RAM; there are no options that allow for a more granular configuration of containers’ maximum memory capacity, resulting in inefficient resource allocation on Heroku. 

‍

This lack of granularity in scaling also raised concerns regarding cloud spend; having to upgrade to underutilized Performance-L dynos resulted in cost bloat and Derek knew that costs on Heroku would continue to become exponentially more expensive over time as their applications’ resource usage needs increased. There were also growing concerns regarding Heroku’s stability following some of the platform’s outages, and the prototypical PaaS seemed to be stagnating in general. 

‍

“Most of Heroku’s features felt frozen in time.” - Derek Gavey, Lead Engineer at Onclusive

‍

The company also went through several mergers around this time, combining with Kantar Reputation Intelligence and PRgloo to create a new company, retaining the name Onclusive, and later acquiring Digimind and Critical Mention. The scale of their applications had already grown beyond a size that Heroku could effectively handle, and with these mergers, the team expected even more applications, so migrating off of the PaaS and moving to AWS and Kubernetes (K8s) seemed to be the best option for the benefit of consolidation.

A Launching Point for Kubernetes 

Onclusive had decided they needed to graduate from a traditional PaaS and manage the entirety of their own infrastructure on Kubernetes so they could leverage its scaling benefits. They had already spun up some clusters on Elastic Kubernetes Service (EKS) while on Heroku, but Porter seemed like a perfect launching point for them to utilize Kubernetes while still retaining the convenience factor of a PaaS for their application developers. 

‍

Onclusive runs about half of their applications on Porter (the majority of the remaining applications being a byproduct of the mergers, with teams using different services and being on separate technology stacks entirely). The platform increases the company’s development velocity considerably; developers frequently utilize features like Preview Environments to test and merge changes easily; they can quickly spin up staging environments without the help of Onclusive’s infrastructure-savvy engineers. 

‍

The other half of their applications are on clusters that run more simple Machine Learning (ML) workloads for augmenting data. These clusters are managed by Onclusive’s infrastructure team, but are peered with Porter-managed infrastructure. For their more complex applications which are running on Porter-managed clusters, the PaaS is invaluable in allowing Onclusive’s engineering team to work with speed and efficiency. 

‍

“Porter lets us push out new features quickly and with a level of polish that would only be otherwise possible with a massive DevOps team” - Derek Gavey, Lead Engineer at Onclusive

Autoscaling with granularity

One of Onclusive’s major projects on Porter is internally referred to as their “Analyst Application”. This project runs many background services to enhance data and scales dramatically throughout each day and week based on the amount of information intake; container usage varies from 20 on the low end to 100 on the high end, based on load. Compared to Heroku’s dynos, they are able to use far fewer containers on Porter due to the granular configuration available on the platform - down to 1MB RAM and 0.01 vCPU per application. While all of the dynos on Heroku were underutilized, on Porter, all of the containers are right-sized.

‍

While on Heroku, Onclusive utilized HireFire for horizontal autoscaling. This Heroku addon allowed them to autoscale based on sidekiq queue length. On Porter, custom autoscaling based on sidekiq queue length is also available to them through Kubernetes Event-driven Autoscaling (KEDA), which requires fewer adjustments as well; since their containers are right-sized through the more granular configuration and control available with Kubernetes over HireFire, there is also far less concern regarding unnecessary scaling.

‍

“When we were using HireFire on Heroku’s dynos, we had to adjust autoscaling much more frequently. On Porter and AWS, the containers are more robust and can handle more load, resulting in less fluctuation; we rarely have to make adjustments.” - Derek Gavey, Lead Engineer at Onclusive

Future on Porter

‍Porter recently rolled out a newer feature on the platform called Porter Apps, which allows users to group multiple components (front-end and back-end) together and manage them as a whole, deploying and updating all applications at once with a single file. Using Porter Apps, Onclusive is building out a new project which serves as an internal platform that will consolidate the disparate services and tools used by the five teams merged under the Onclusive umbrella. This unified architecture will essentially synthesize five different deployment methods, allowing the company to streamline operations and increase efficiency.