HPA Infrastructure
HPA Infrastructure

⁠

Screen Shot 2022-08-22 at 4.29.35 PM.png

⁠

Infrastructure change

We collect metrics from Browserless pods by running a Browserless deployment and a Browserless-metrics-exporter in one container in every pod.

The Browserless-metrics-exporter is web application that scrapes the the /pressure endpoint on the browserless deployment to get metrics. It receives the metrics as a json blog and converts it to metrics in prometheus format .

We enable Prometheus scraper using Sumologic Helm chart and tell it to scrape the /metrics endpoint which exposed by by the Browserless-metrics-exporter that returns metrics in Prometheus format .

Now Prometheus is able to scrape metrics from Browserless deployment at regular intervals.

Next, we deploy a prometheus-adapter using a prometheus adapter Helm chart which lets the scraped metrics be accessible by Kubernetes .

Lastly, the Horizontal Pod Autoscaler is able to query forqueued metric and control the deployment Browserless Pods.

The target average value is 2, which means that whenever browserless_queued is greater than averageValue, the HPA will scale up and viceverca.

Scaling Policies

Right now whenever we want to up-scale, we upscale immediately 500% which results in 5 extra pods being created (one every 15 seconds). And conversely when we want to down-scale, we down-scale gradually slowly 1 pod every minute. This is because we want to be responsive to more workflows, however don't want to immediately downscale and risk having to upscale again. A few more scaling policies can be found

here⁠

⁠

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.

HPA InfrastructureHPA Infrastructure

Infrastructure change

Scaling Policies

HPA Infrastructure
HPA Infrastructure