RabbitMQ安装-吊舱具有未绑定的立即PersistentVolumeClaims

问题描述

我正在尝试在Kubernetes中安装RabbitMQ，并按照RabbitMQ网站https://www.rabbitmq.com/blog/2020/08/10/deploying-rabbitmq-to-kubernetes-whats-involved/上的条目进行操作。

请注意我的CentOS 7和Kubernetes 1.18。另外，我什至不确定这是否是部署RabbitMQ的最佳方法，尽管它是我能找到的最佳文档。我确实发现一些内容表明volumeClaimTemplates不支持NFS，所以我想知道这是否是问题所在。

我已使用NFS添加了我的永久卷：

kind: PersistentVolume
apiVersion: v1
metadata:
  name: rabbitmq-nfs-pv
  namespace: ninegold-rabbitmq
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  nfs:
    path: /var/nfsshare
    server: 192.168.1.241
  persistentVolumeReclaimPolicy: Retain

它正确创建了PV。

[admin@centos-controller ~]$ kubectl get pv -n ninegold-rabbitmq
NAME                                      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                                     STORAGECLASS   REASON   AGE
ninegold-platform-custom-config-br        1Gi        RWX            Retain           Bound       ninegold-platform/ninegold-db-pgbr-repo                           22d
ninegold-platform-custom-config-pgadmin   1Gi        RWX            Retain           Bound       ninegold-platform/ninegold-db-pgadmin                             21d
ninegold-platform-custom-config-pgdata    1Gi        RWX            Retain           Bound       ninegold-platform/ninegold-db                                     22d
rabbitmq-nfs-pv                           5Gi        RWO            Retain           Available                                                                     14h

然后我添加我的StatefulSet。

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: rabbitmq
  namespace: ninegold-rabbitmq
spec:
  selector:
    matchLabels:
      app: "rabbitmq"
  # headless service that gives network identity to the RMQ nodes,and enables them to cluster
  serviceName: rabbitmq-headless # serviceName is the name of the service that governs this StatefulSet. This service must exist before the StatefulSet,and is responsible for the network identity of the set. Pods get DNS/hostnames that follow the pattern: pod-specific-string.serviceName.default.svc.cluster.local where "pod-specific-string" is managed by the StatefulSet controller.
  volumeClaimTemplates:
  - metadata:
      name: rabbitmq-data
      namespace: ninegold-rabbitmq
    spec:
      storageClassName: local-storage
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: "5Gi"
  template:
    metadata:
      name: rabbitmq
      namespace: ninegold-rabbitmq
      labels:
        app: rabbitmq
    spec:
      initContainers:
      # Since k8s 1.9.4,config maps mount read-only volumes. Since the Docker image also writes to the config file,# the file must be mounted as read-write. We use init containers to copy from the config map read-only
      # path,to a read-write path
      - name: "rabbitmq-config"
        image: busybox:1.32.0
        volumeMounts:
        - name: rabbitmq-config
          mountPath: /tmp/rabbitmq
        - name: rabbitmq-config-rw
          mountPath: /etc/rabbitmq
        command:
        - sh
        - -c
        # the newline is needed since the Docker image entrypoint scripts appends to the config file
        - cp /tmp/rabbitmq/rabbitmq.conf /etc/rabbitmq/rabbitmq.conf && echo '' >> /etc/rabbitmq/rabbitmq.conf;
          cp /tmp/rabbitmq/enabled_plugins /etc/rabbitmq/enabled_plugins
      volumes:
      - name: rabbitmq-config
        configMap:
          name: rabbitmq-config
          optional: false
          items:
          - key: enabled_plugins
            path: "enabled_plugins"
          - key: rabbitmq.conf
            path: "rabbitmq.conf"
      # read-write volume into which to copy the rabbitmq.conf and enabled_plugins files
      # this is needed since the docker image writes to the rabbitmq.conf file
      # and Kubernetes Config Maps are mounted as read-only since Kubernetes 1.9.4
      - name: rabbitmq-config-rw
        emptyDir: {}
      - name: rabbitmq-data
        persistentVolumeClaim:
          claimName: rabbitmq-data
      serviceAccount: rabbitmq
      # The Docker image runs as the `rabbitmq` user with uid 999 
      # and writes to the `rabbitmq.conf` file
      # The security context is needed since the image needs
      # permission to write to this file. Without the security 
      # context,`rabbitmq.conf` is owned by root and inaccessible
      # by the `rabbitmq` user
      securityContext:
        fsGroup: 999
        runAsUser: 999
        runAsGroup: 999
      containers:
      - name: rabbitmq
        # Community Docker Image
        image: rabbitmq:latest
        volumeMounts:
        # mounting rabbitmq.conf and enabled_plugins
        # this should have writeable access,this might be a problem
        - name: rabbitmq-config-rw
          mountPath: "/etc/rabbitmq"
          # mountPath: "/etc/rabbitmq/conf.d/"
          mountPath: "/var/lib/rabbitmq"
        # rabbitmq data directory
        - name: rabbitmq-data
          mountPath: "/var/lib/rabbitmq/mnesia"
        env:
        - name: RABBITMQ_DEFAULT_PASS
          valueFrom:
            secretKeyRef:
              name: rabbitmq-admin
              key: pass
        - name: RABBITMQ_DEFAULT_USER
          valueFrom:
            secretKeyRef:
              name: rabbitmq-admin
              key: user
        - name: RABBITMQ_ERLANG_COOKIE
          valueFrom:
            secretKeyRef:
              name: erlang-cookie
              key: cookie
        ports:
        - name: amqp
          containerPort: 5672
          protocol: TCP
        - name: management
          containerPort: 15672
          protocol: TCP
        - name: prometheus
          containerPort: 15692
          protocol: TCP
        - name: epmd
          containerPort: 4369
          protocol: TCP
        livenessProbe:
          exec:
            # This is just an example. There is no "one true health check" but rather
            # several rabbitmq-diagnostics commands that can be combined to form increasingly comprehensive
            # and intrusive health checks.
            # Learn more at https://www.rabbitmq.com/monitoring.html#health-checks.
            #
            # Stage 2 check:
            command: ["rabbitmq-diagnostics","status"]
          initialDelaySeconds: 60
          # See https://www.rabbitmq.com/monitoring.html for monitoring frequency recommendations.
          periodSeconds: 60
          timeoutSeconds: 15
        readinessProbe: # probe to know when RMQ is ready to accept traffic
          exec:
            # This is just an example. There is no "one true health check" but rather
            # several rabbitmq-diagnostics commands that can be combined to form increasingly comprehensive
            # and intrusive health checks.
            # Learn more at https://www.rabbitmq.com/monitoring.html#health-checks.
            #
            # Stage 1 check:
            command: ["rabbitmq-diagnostics","ping"]
          initialDelaySeconds: 20
          periodSeconds: 60
          timeoutSeconds: 10

但是我的状态集未绑定，但出现以下错误：

running "VolumeBinding" filter plugin for pod "rabbitmq-0": pod has unbound immediate PersistentVolumeClaims

PVC未正确绑定到PV，但保持挂起状态。

[admin@centos-controller ~]$ kubectl get pvc -n ninegold-rabbitmq
NAME                       STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS    AGE
rabbitmq-data-rabbitmq-0   Pending                                      local-storage   14h

我已经仔细检查了accessModes的容量，我不确定为什么不绑定。我的示例来自https://github.com/rabbitmq/diy-kubernetes-examples/tree/master/gke，我所做的唯一更改就是绑定了我的NFS卷。

任何帮助将不胜感激。

解决方法

在您的YAML中，我发现了一些错误的配置。

local-storage类。

我假设您使用Documentation example创建了local-storage。有以下信息：

本地卷当前不支持动态配置，但是仍应创建StorageClass来延迟卷绑定，直到Pod调度为止。

要使用volumeClaimTemplates时，将使用Dynamic Provisioning。在Medium article中有很好的解释。

StatefulSet中的PV

专门针对卷部分，StatefulSet提供了一个名为volumeClaimTemplates的密钥。这样，您可以动态地从存储类请求PVC。作为新的statefulset应用程序定义的一部分，替换卷... PVC被命名为volumeClaimTemplate name + pod-name + ordinal number。

由于local-storage不支持dynamic provisioning，因此它将不起作用。您需要将NFS storageclass与适当的供应商一起使用，或手动创建PV。

此外，当您为每个吊舱使用volumeClaimTemplates时，它将创建Pv和PVC。此外，PVC和PV的边界为1：1。有关更多详细信息，请检查this SO thread。

错误unbound immediate PersistentVolumeClaims

这意味着dynamic provisioning不能正常工作。如果您勾选kubectl get pv,pvc，将不会看到名称为PV的任何新PVC，volumeClaimTemplate name + pod-name + ordinal number。

claimName: rabbitmq-data

我假设，在此声明中，您要将其安装到由PV创建的volumeClaimTemplates上，但尚未创建。此外，PV的第一个容器的名称为rabbitmq-data-rabbitmq-0，第二个容器的名称为rabbitmq-data-rabbitmq-1。

最后，这篇文章-Kubernetes : NFS and Dynamic NFS provisioning可能会有所帮助。

kubernetes kubernetes-statefulset

RabbitMQ安装-吊舱具有未绑定的立即PersistentVolumeClaims

问题描述

解决方法

相关问答