Crawlee Slow in Kubernetes

[Chetan11-dev]

I deployed my crawlee scraper on Kubernetes using YAML configurations. However, I've noticed that the crawler is slow to launch Chrome. Does anyone have ideas on how to speed this up?
Kubernetes Yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: crawlee-scraper-depl
spec:
  replicas: 1
  selector:
    matchLabels:
      app: crawlee-scraper
  template:
    metadata:
      labels:
        app: crawlee-scraper
    spec:
      volumes:
        - name: dshm
          emptyDir:
            medium: Memory
      containers:
      - name: crawlee-scraper
        image: chetan1111/crawlee-scraper:1.0.4
        volumeMounts:
            - mountPath: /dev/shm
              name: dshm
        resources:
          requests:
            memory: "700Mi"
            cpu: "550m"
          limits:
            memory: "750Mi"
            cpu: "600m"

---

apiVersion: v1
kind: Service
metadata:
  name: crawlee-scraper-srv
spec:
  type: ClusterIP
  selector:
    app: crawlee-scraper
  ports:
    - name: crawlee-scraper
      protocol: TCP
      port: 3000
      targetPort: 3000

[B4nan]

You probably need more resources, there is no magical switch, I am not sure what the 550m cpu actually means, on apify platform you need at least 4gb ram which translates to one CPU core to run a browser crawler.

One hint, crawlee by default uses just a 1/4 of the available ram, on apify platform this is automatically lifted to use everything, but since you deploy elsewhere, maybe that's the case too.

https://crawlee.dev/api/core/interface/ConfigurationOptions#availableMemoryRatio

[Chetan11-dev]

Thanks for the information, it is likely due to less RAM.