Elasticsearch TLS错误：空的客户端证书链。

Question

Elasticsearch TLS错误：空的客户端证书链。

elasticsearchtls1.2azure-aksazure-keyvaultelasticsearch-x-pack

3

AKS Kubernetes版本1.20.7

ES版本：7.12.0

我们遇到了“Empty client certificate chain”（请参见下面的日志跟踪）
我们启用了TLS（以下是设置）
我们正在使用自签名证书，而不是使用elasticsearch-certutil工具
我们的证书具有“扩展密钥用途”，明确启用了客户端认证和服务器认证

非常感谢您的帮助。

XPack设置

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /usr/share/Elasticsearch/config/certs/elasticsearch.pfx
xpack.security.transport.ssl.truststore.path: /usr/share/Elasticsearch/config/certs/elasticsearch.pfx
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.client_authentication: optional
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.keystore.path: /usr/share/Elasticsearch/config/certs/elasticsearch.pfx
xpack.security.http.ssl.truststore.path: /usr/share/Elasticsearch/config/certs/elasticsearch.pfx

日志追踪

ompletionEvent(javax.net.ssl.SSLHandshakeException: Empty client certificate chain)" }
{"type": "server", "timestamp": "2021-10-27T05:32:39,760Z", "level": "TRACE", "component": "o.e.t.n.ESLoggingHandler", "cluster.name": "test-rp-search", "node.name": "test-rp-search-master-0", "message": "[id: 0x7272c902, L:/10.155.12.55:9300 ! R:/10.155.12.34:42246] EXCEPTION: io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Empty client certificate chain",
"stack trace": ["io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Empty client certificate chain",
"at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:471) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]",
"at java.lang.Thread.run(Thread.java:832) [?:?]",
"Caused by: javax.net.ssl.SSLHandshakeException: Empty client certificate chain",
"at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?]",
"at sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:356) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:312) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:303) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1194) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1181) ~[?:?]",
"at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396) ~[?:?]",
"at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:480) ~[?:?]",
"at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1267) ~[?:?]",
"at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1254) ~[?:?]",
"at java.security.AccessController.doPrivileged(AccessController.java:691) ~[?:?]",
"at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1199) ~[?:?]",
"at io.netty.handler.ssl.SslHandler.runAllDelegatedTasks(SslHandler.java:1542) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1556) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1440) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1267) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1314) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]",
"... 16 more"] }
{"type": "server", "timestamp": "2021-10-27T05:32:39,762Z", "level": "WARN", "component": "o.e.t.TcpTransport", "cluster.name": "test-rp-search", "node.name": "test-rp-search-master-0", "message": "exception caught on transport layer [Netty4TcpChannel{localAddress=/10.155.12.55:9300, remoteAddress=/10.155.12.34:42246, profile=default}], closing connection",
"stacktrace": ["io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Empty client certificate chain",
"at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:471) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]",
"at java.lang.Thread.run(Thread.java:832) [?:?]",
"Caused by: javax.net.ssl.SSLHandshakeException: Empty client certificate chain",
"at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?]",
"at sun.security.ssl.Alert.createSSLException(Alert.java:117) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:356) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:312) ~[?:?]",
"at sun.security.ssl.TransportContext.fatal(TransportContext.java:303) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1194) ~[?:?]",
"at sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1181) ~[?:?]",
"at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396) ~[?:?]",
"at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:480) ~[?:?]",
"at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1267) ~[?:?]",
"at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1254) ~[?:?]",
"at java.security.AccessController.doPrivileged(AccessController.java:691) ~[?:?]",
"at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1199) ~[?:?]",
"at io.netty.handler.ssl.SslHandler.runAllDelegatedTasks(SslHandler.java:1542) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1556) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1440) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1267) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1314) ~[netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440) ~[netty-codec-4.1.49.Final.jar:4.1.49.Final]",
"... 16 more"] }

- kjdfkdsf

1个回答

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- kjdfkdsf · Accepted Answer

我们已经解决了这个非常棘手的问题。以下是原因...

Azure KeyVault实际上存在的问题如下：

- 准备用完整链设置的自签名PFX文件。 - PFX必须受到密码保护（尽管Elasticsearch示例没有明确说明），以便获得完整的链。 - 运行以下命令验证完整链 keytool -list -keystone yourFullChainCert.pfx -storetype pkcs12 -v - 使用门户上传受密码保护的PFX（含完整链）到Azure KeyVault。 - 下载PFX文件并再次运行keytool。您将无法验证该链。

解决方案是一个恶劣的变通方法（恶劣是因为我需要构建自己的镜像）

镜像构建

- 用aes256算法加密传输/ssl PFX私钥。 - 创建私有Elasticsearch映像。 - 在容器映像中，在/usr/share/elasticsearch/config/certs目录中放置所有PEM文件： - 您的传输PFX根CA证书PEM，例如rootCA.cert.pem - 您的传输PFX可选颁发CA证书PEM，例如issuingCA.cert.pem - 用aes256加密的传输PFX私钥PEM，例如tls.aes.key.pem - 您的传输PFX证书PEM，例如tls.cert.pem - 用aes256加密的SSL PFX私钥PEM，例如ssl.aes.key.pem - 您的SSL PFX证书PEM，例如ssl.cert.pem - 将AES256加密密钥放在Azure KeyVault中。 - 使用CSI驱动程序同步密钥，请参考以下代码片段certysnc.yaml文件: - 将certysnc.yaml文件放置在Elasticseach helm charts模板目录中。

apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
  name: {{ .Values.secretProviderClassName }}
  namespace: {{ .Release. Namespace }} 
spec:
  provider: azure
  secretObjects:
  - secretName: elasticsearch-tls-ssl-cert-key 
    type: Opaque
    data:
    - objectName: es-tls-ssl-cert-key
      key: xpack.security.transport.ssl.secure_key_passphrase
    - objectName: es-tls-ssl-cert-key
      key: xpack.security.http.ssl.secure_key_passphrase
  - secretName: elastic-credentials
    type: Opaque
    data:
    - objectName: elastic-username
      key: username
    - objectName: elastic-user-password
      key: password
  parameters:
    usePodIdentity: "true"
    userAssignedIdentityID: {{ .Values.userAssignedIdentityID }}
    keyvaultName: {{ .Values.keyvaultName }}
    cloudName: "AzurePublicCloud"
    objects:  |
      array:
        - |
          objectName: elastic-user-password
          objectType: secret
          objectVersion: ""
        - |
          objectName: elastic-username
          objectType: secret
          objectVersion: ""
        - |
          objectName: es-tls-ssl-cert-key
          objectType: secret
          objectVersion: ""
    tenantId: {{ .Values.tenantId }}
    resourceGroup: {{ .Values.resourceGroup }}
    subscriptionId: {{ .Values.subscriptionId }}