New Add-on (Product analytics for LLM apps)

Langfuse (YC W23) is now available as an add-on. Langfuse is the Mixpanel for LLM applications, helping companies to track and analyze quality, cost and latency across product releases and use cases.

Spot Instances‍

When selecting node groups to deploy onto on AWS, spot instances are now available as an option. This includes GPU instance types. Spot instances can allow for significant cost savings, up to 90% in comparison to on-demand instances. However, by nature, they can be interrupted by AWS with minimal notice, and therefore are not recommended for business-critical applications.

As part of support for fully custom node groups, public subnets are also available.

Slack Notifications Filtering‍

Notifications via Slack can now be filtered on an application level. Users can also assign groups of apps to pipe notifications into specific Slack channels, cutting down noise and allowing for more granular insights.

Enhanced Security‍

All Porter-managed infrastructure can now utilize an advanced service mesh (Linkerd). This allows for zero-trust security via mutual encryption between the services Porter spins up and manages.

The service mesh natively supports gRPC and other advanced protocols, enabling more efficient communication between micro-services.

Customizable GPU Resources

Users can now tailor their GKE clusters with flexible GPU options, choosing both the type and quantity of GPUs to match their specific workload requirements.