Security Attributes for Cluster Operator

[3rk1n]

Hi,

I want to close two security recommendations for Kubernetes Cluster in Microsoft Defender for Cloud.

One of them is "Kubernetes clusters should disable automounting API credentials" and it can be solved by added "automountServiceAccountToken: false" for POD security "spec --> template --> spec"

spec:
  template:
    metadata:
    spec:
      automountServiceAccountToken: false

But when I am adding this values the pods give me CrashLoopBackOff status and that logs in below, and I can not see that spec in Pod level.

How can I solve this?

{"level":"error","ts":"2023-06-19T06:53:56Z","logger":"controller-runtime.client.config","msg":"unable to load in-cluster config","error":"open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory","stacktrace":"sigs.k8s.io/controller-runtime/pkg/client/config.loadConfig.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/client/config/config.go:133\nsigs.k8s.io/controller-runtime/pkg/client/config.loadConfig\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/client/config/config.go:155\nsigs.k8s.io/controller-runtime/pkg/client/config.GetConfigWithContext\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/client/config/config.go:97\nsigs.k8s.io/controller-runtime/pkg/client/config.GetConfig\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/client/config/config.go:77\nsigs.k8s.io/controller-runtime/pkg/client/config.GetConfigOrDie\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/client/config/config.go:175\nmain.main\n\t/workspace/main.go:131\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}
{"level":"error","ts":"2023-06-19T06:53:56Z","logger":"controller-runtime.client.config","msg":"unable to get kubeconfig","error":"invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable","errorCauses":[{"error":"no configuration has been provided, try setting KUBERNETES_MASTER environment variable"}],"stacktrace":"sigs.k8s.io/controller-runtime/pkg/client/config.GetConfigOrDie\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/client/config/config.go:177\nmain.main\n\t/workspace/main.go:131\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:250"}

The second is "Running containers as root user should be avoided" and it can be solved by added "securityContext"

1. for POD security "spec --> template --> spec" add "securityContext:" and under it add "runAsNonRoot: true, runAsGroup: , runAsUser: "

spec:
  template:
    metadata:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsGroup: <Group-ID>
        runAsUser: <User-ID>

2. for CONTAINER security "spec --> template --> spec --> containers" add "securityContext:" and under it add "readOnlyRootFilesystem: true", "allowPrivilegeEscalation: false", "privileged: false", "runAsNonRoot: true" "runAsGroup: ", "runAsUser: " and "capabilities: drop: - ALL"

spec:
  template:
    metadata:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsGroup: <Group-ID>
        runAsUser: <User-ID>
      containers:
      - name: <Name-of-Deployment>
        image: <Name-of-Image>
        securityContext:
          readOnlyRootFilesystem: true
          allowPrivilegeEscalation: false
          privileged: false
          runAsNonRoot: true
          runAsGroup: <Group-ID>
          runAsUser: <User-ID>
          capabilities:
            drop:
            - ALL

How can I add this spec with correct values for runAsGroup: <Group-ID> and runAsUser: <User-ID> ?
Should I add any values for Dockerfile and how can I add?

[Zerpet]

One of them is "Kubernetes clusters should disable automounting API credentials" and it can be solved by added "automountServiceAccountToken: false" for POD security "spec --> template --> spec"

This recommendation doesn't make any sense for this operator. This Operator needs to communicate with Kubernetes API, in order to deploy and manage RabbitMQ automagically for you. In order to communicate with the Kubernetes API, this Operator uses a dedicated Service Account, and it mounts its token to use as credentials. The error you observe is the logical consequence of removing this token i.e. unable to load token -> unable to communicate with Kubernetes API.

You can safely disregard this "warning" and leave auto-mount as true.

The second is "Running containers as root user should be avoided" and it can be solved by added "securityContext"

How can I add this spec with correct values for runAsGroup: and runAsUser: ?
Should I add any values for Dockerfile and how can I add?

You do exactly what the suggestion says 🙂 Add the security context to the cluster operator deployment spec. In case you are looking for inspiration, your Deployment object for the cluster operator should look similar to this:

cluster-operator-deployment.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: rabbitmq-operator
    app.kubernetes.io/name: rabbitmq-cluster-operator
    app.kubernetes.io/part-of: rabbitmq
  name: rabbitmq-cluster-operator
  namespace: rabbitmq-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: rabbitmq-cluster-operator
  template:
    metadata:
      labels:
        app.kubernetes.io/component: rabbitmq-operator
        app.kubernetes.io/name: rabbitmq-cluster-operator
        app.kubernetes.io/part-of: rabbitmq
    spec:
      securityContext:
        runAsNonRoot: true
        runAsGroup: 1001
        runAsUser: 1001
      containers:
      - command:
        - /manager
        env:
        - name: OPERATOR_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: rabbitmqoperator/cluster-operator:2.3.0
        name: operator
        securityContext:
          readOnlyRootFilesystem: true
          allowPrivilegeEscalation: false
          privileged: false
          runAsNonRoot: true
          runAsGroup: 1001
          runAsUser: 1001
          capabilities:
            drop:
            - ALL
        ports:
        - containerPort: 9782
          name: metrics
          protocol: TCP
        resources:
          limits:
            cpu: 200m
            memory: 500Mi
          requests:
            cpu: 200m
            memory: 500Mi
      serviceAccountName: rabbitmq-cluster-operator
      terminationGracePeriodSeconds: 10

The group ID and user ID do not need to exist at all. If you are new to this concept, I'd recommend reading this:

https://kubernetes.io/docs/tasks/configure-pod-container/security-context/