问题描述
想发布这个是因为它可能对某人有所帮助。我找不到没有 helm 包的 Kubernetes NiFi 设置，所以我为 Kubernetes NiFi 集群设置准备了以下配置 YAML。
这是 Zookeeper Cluster setup in AKS 的链接
解决方法
如果您在配置中的任何地方发现任何问题，或者您想提供任何建议，请发表评论。根据您的使用情况增加磁盘存储配置。
apiVersion: v1
kind: Service
metadata:
name: nifi-hs
labels:
app: nifi
spec:
ports:
- port: 1025
name: nodeport
- port: 8080
name: client
clusterIP: None
selector:
app: nifi
---
apiVersion: v1
kind: Service
metadata:
name: nifi-cs
labels:
app: nifi
annotations:
service.beta.kubernetes.io/azure-dns-label-name: nifi
spec:
ports:
- port: 80
targetPort: 8080
name: client
selector:
app: nifi
type: LoadBalancer
---
apiVersion: policy/v1beta1
kind: PodDisruptionBudget
metadata:
name: nifi-pdb
spec:
selector:
matchLabels:
app: nifi
maxUnavailable: 1
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nifi-sc
selfLink: /apis/storage.k8s.io/v1/storageclasses/nifi-sc
labels:
addonmanager.kubernetes.io/mode: EnsureExists
kubernetes.io/cluster-service: 'true'
provisioner: kubernetes.io/azure-disk
parameters:
cachingmode: ReadOnly
kind: Managed
storageaccounttype: StandardSSD_LRS
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nifi
spec:
selector:
matchLabels:
app: nifi
serviceName: nifi-hs
replicas: 3
updateStrategy:
type: RollingUpdate
podManagementPolicy: OrderedReady
template:
metadata:
labels:
app: nifi
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- k: "app"
operator: In
values:
- nifi
topologyKey: "kubernetes.io/hostname"
containers:
- name: nifi
image: "apache/nifi:1.13.0"
env:
- name: NIFI_CLUSTER_IS_NODE
value: "true"
- name: HOSTNAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NIFI_CLUSTER_ADDRESS
value: $(HOSTNAME).nifi-hs
- name: NIFI_CLUSTER_NODE_PROTOCOL_PORT
value: "1025"
- name: NIFI_WEB_HTTP_HOST
value: $(HOSTNAME).nifi-hs.ns1.svc.cluster.local
#- name: NIFI_WEB_HTTP_PORT
# value: "80"
- name: NIFI_CLUSTER_NODE_PROTOCOL_MAX_THREADS
value: "100"
- name: NIFI_ZK_CONNECT_STRING
value: "zk-cs:2181"
- name: NIFI_ELECTION_MAX_CANDIDATES
value: "3"
ports:
- containerPort: 8080
name: client
- containerPort: 1025
name: nodeport
volumeMounts:
- name: nifi-database
mountPath: "/opt/nifi/nifi-current/database_repository"
- name: nifi-flowfile
mountPath: "/opt/nifi/nifi-current/flowfile_repository"
- name: nifi-content
mountPath: "/opt/nifi/nifi-current/content_repository"
- name: nifi-provenance
mountPath: "/opt/nifi/nifi-current/provenance_repository"
- name: nifi-state
mountPath: "/opt/nifi/nifi-current/state"
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
volumeClaimTemplates:
- metadata:
name: nifi-database
spec:
storageClassName: "nifi-sc"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
- metadata:
name: nifi-flowfile
spec:
storageClassName: "nifi-sc"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
- metadata:
name: nifi-content
spec:
storageClassName: "nifi-sc"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
- metadata:
name: nifi-provenance
spec:
storageClassName: "nifi-sc"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
- metadata:
name: nifi-state
spec:
storageClassName: "nifi-sc"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
affinity block 中的 labelselector 少了几个词。以下是 statefulset 的更新工作 yaml 块。
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nifi
spec:
selector:
matchLabels:
app: nifi
serviceName: nifi-hs
replicas: 3
updateStrategy:
type: RollingUpdate
podManagementPolicy: OrderedReady
template:
metadata:
labels:
app: nifi
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: "app"
operator: In
values:
- nifi
topologyKey: "kubernetes.io/hostname"
containers:
- name: nifi
image: "apache/nifi:1.13.0"
env:
- name: NIFI_CLUSTER_IS_NODE
value: "true"
- name: HOSTNAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NIFI_CLUSTER_ADDRESS
value: $(HOSTNAME).nifi-hs
- name: NIFI_CLUSTER_NODE_PROTOCOL_PORT
value: "1025"
- name: NIFI_WEB_HTTP_HOST
value: $(HOSTNAME).nifi-hs.ns1.svc.cluster.local
#- name: NIFI_WEB_HTTP_PORT
# value: "80"
- name: NIFI_CLUSTER_NODE_PROTOCOL_MAX_THREADS
value: "100"
- name: NIFI_ZK_CONNECT_STRING
value: "zk-cs:2181"
- name: NIFI_ELECTION_MAX_CANDIDATES
value: "3"
ports:
- containerPort: 8080
name: client
- containerPort: 1025
name: nodeport
volumeMounts:
- name: nifi-database
mountPath: "/opt/nifi/nifi-current/database_repository"
- name: nifi-flowfile
mountPath: "/opt/nifi/nifi-current/flowfile_repository"
- name: nifi-content
mountPath: "/opt/nifi/nifi-current/content_repository"
- name: nifi-provenance
mountPath: "/opt/nifi/nifi-current/provenance_repository"
- name: nifi-state
mountPath: "/opt/nifi/nifi-current/state"
securityContext:
runAsUser: 1000
runAsGroup: 1000
fsGroup: 1000
volumeClaimTemplates:
- metadata:
name: nifi-database
spec:
storageClassName: "nifi-sc"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
- metadata:
name: nifi-flowfile
spec:
storageClassName: "nifi-sc"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
- metadata:
name: nifi-content
spec:
storageClassName: "nifi-sc"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
- metadata:
name: nifi-provenance
spec:
storageClassName: "nifi-sc"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi
- metadata:
name: nifi-state
spec:
storageClassName: "nifi-sc"
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 5Gi