问题描述
我们正在尝试在我们的 OCP (4.5.36) 集群中使用 Red Hat Data Grid (RHDG)/Infinispan。我们安装了最新的官方 RHDG Operator,并定义了一个缓存类型集群。 (这显然是一个 k8s StatefulSet。)
然后,我配置了一个 WebSphere Liberty 容器/部署以尝试将该 Infinispan 集群用于其会话,如 https://github.com/WASdev/ci.docker#session-caching 中所述。
Infinispan 集群和 Liberty Deployment 都在同一个项目/命名空间中。
但是,Liberty 容器无法连接,Infinispan 容器报告了自己的几个警告。
Liberty 容器“客户端”日志:
SELECT *,- 100.0 * (value - LEAD(value) OVER (Partition by Country ORDER BY t.year) / value) AS Growth
FROM tbl AS t
ORDER BY t.country
Inifinispan 容器日志的相关部分是什么:
INFINISPAN_SERVICE_NAME(original): session-infinispan
INFINISPAN_SERVICE_NAME(normalized): SESSION_INFINISPAN
INFINISPAN_HOST: 172.30.137.86
INFINISPAN_PORT: 11222
INFINISPAN_USER: developer
INFINISPAN_PASS: <redacted>
Launching defaultServer (WebSphere Application Server 21.0.0.3/wlp-1.0.50.cl210320210309-1101) on Eclipse OpenJ9 VM,version 1.8.0_282-b08 (en_US)
[AUDIT ] CWWKE0001I: The server defaultServer has been launched.
[AUDIT ] CWWKE0100I: This product is licensed for development,and limited production use. The full license terms can be viewed here: https://public.dhe.ibm.com/ibmdl/export/pub/software/websphere/wasdev/license/base_ilan/ilan/21.0.0.3/lafiles/en.html
[AUDIT ] CWWKG0093A: Processing configuration drop-ins resource: /opt/ibm/wlp/usr/servers/defaultServer/configDropins/defaults/keystore.xml
[AUDIT ] CWWKG0093A: Processing configuration drop-ins resource: /opt/ibm/wlp/usr/servers/defaultServer/configDropins/overrides/infinispan-client-sessioncache.xml
[AUDIT ] CWWKZ0058I: Monitoring dropins for applications.
[AUDIT ] CWWKT0016I: Web application available (default_host): http://payment-engine-6dcc5b6d5-jclx2:9080/payment/
[ERROR ] ISPN004007: Exception encountered. Retry 10 out of 10
org.infinispan.client.hotrod.exceptions.TransportException:: ISPN004071: Connection to 172.30.137.86/172.30.137.86:11222 was closed while waiting for response.
[ERROR ] SESN0307E: An exception occurred when initializing the cache. The exception is: org.infinispan.client.hotrod.exceptions.TransportException:: org.infinispan.client.hotrod.exceptions.TransportException:: ISPN004071: Connection to 172.30.137.86/172.30.137.86:11222 was closed while waiting for response.
at org.infinispan.client.hotrod.impl.transport.netty.ActivationHandler.exceptionCaught(ActivationHandler.java:53)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:300)
...
(实际上,有几个 Infinispan 启动警告,主要是关于已弃用的功能。但这是唯一一个带有堆栈跟踪的,所以我得出结论,它可能是“罪魁祸首”)
此外,这是 Infinispan 服务,因此您可以看到 IP 和端口与 Liberty 容器正在使用的匹配:
解决方法
在 the Infinispan chat service 上解决此问题后,似乎 SSL/TLS 设置不正确或不完整。
我曾尝试删除 Infinispan 集群中的加密,但我没有充分重新启动组件,或者事后您无法更改它。但是,删除集群并在禁用它的情况下重新创建,使 Liberty 通信能够正常工作。
以下 CR YAML 有效:
apiVersion: infinispan.org/v1
kind: Infinispan
metadata:
name: session-infinispan
spec:
replicas: 1
service:
type: Cache
security:
endpointEncryption:
type: None
现在继续探索 Liberty 设置中缺少的内容以正确使用 SSL。 Infinispan 聊天对话说官方图片中的这个 Liberty XML 设置:
<server>
<featureManager>
<feature>sessionCache-1.0</feature>
</featureManager>
<httpSessionCache libraryRef="InfinispanLib">
<properties infinispan.client.hotrod.server_list="${INFINISPAN_HOST}:${INFINISPAN_PORT}"/>
<properties infinispan.client.hotrod.marshaller="org.infinispan.commons.marshall.JavaSerializationMarshaller"/>
<properties infinispan.client.hotrod.java_serial_whitelist=".*"/>
<properties infinispan.client.hotrod.auth_username="${INFINISPAN_USER}"/>
<properties infinispan.client.hotrod.auth_password="${INFINISPAN_PASS}"/>
<properties infinispan.client.hotrod.auth_realm="default"/>
<properties infinispan.client.hotrod.sasl_mechanism="DIGEST-MD5"/>
<properties infinispan.client.hotrod.auth_server_name="infinispan"/>
</httpSessionCache>
<httpSessionCache enableBetaSupportForInfinispan="true"/> <!-- TODO remove once no longer gated -->
<library id="InfinispanLib">
<fileset dir="${shared.resource.dir}/infinispan" includes="*.jar"/>
</library>
</server>
Needs the following properties added:
# Encryption
infinispan.client.hotrod.sni_host_name=$SERVICE_HOSTNAME
# Path to the TLS certificate.
# Clients automatically generate trust stores from certificates.
infinispan.client.hotrod.trust_store_path=/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt