使用TcpDiscoveryKubernetesIpFinder在kubernetes集群中无法发现Ignite

6
我正在尝试使用 TcpDiscoveryKubernetesIpFinder 使在k8s中部署的ignite可被发现。我已经按照apache ignite文档中推荐的所有部署配置进行了设置以使其可被发现。Ignite版本为v2.6。当我尝试从集群(和命名空间)内的另一个服务访问ignite时,它会失败并显示以下错误。

. . instance-14292nccv10-74997cfdff-kqdqh] 由于以下原因引起: java.io.IOException: 服务器返回HTTP响应代码: 403,URL为: https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/my-namespace/endpoints/ignite-service [instance-14292nccv10-74997cfdff-kqdqh] 在 sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1894) ~[na:1.8.0_151] [instance-14292nccv10-74997cfdff-kqdqh] 中 sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492) ~[na:1.8.0_151] [instance-14292nccv10-74997cfdff-kqdqh] 在 sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:263) ~[na:1.8.0_151] [instance-14292nccv10-74997cfdff-kqdqh] . .

我用以下配置使我的点火程序可以被发现:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: ignite-service
  namespace: my-namespace
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: ignite-service
  namespace: my-namespace
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - endpoints
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: ignite-service
roleRef:
  kind: ClusterRole
  name: ignite-service
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: ignite-service
  namespace: my-namespace
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: ignite-service-volume-claim-blr3
  namespace: my-namespace
spec:
  storageClassName: ssd
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Secret
metadata:
  name: ignite-files
  namespace: my-namespace
data:
  ignite-config.xml: PGJlYW5zIHhtbG5zID0gImh0dHA6Ly93d3cuc3ByaW5nZnJhbWV3b3JrLm9yZy9zY2hlbWEvYmVhbnMiCiAgICAgICB4bWxuczp4c2kgPSAiaHR0cDovL3d3dy53My5vcmcvMjAwMS9YTUxTY2hlbWEtaW5zdGFuY2UiCiAgICAgICB4bWxuczp1dGlsID0gImh0dHA6Ly93d3cuc3ByaW5nZnJhbWV3b3JrLm9yZy9zY2hlbWEvdXRpbCIKICAgICAgIHhzaTpzY2hlbWFMb2NhdGlvbiA9ICIKICAgICAgIGh0dHA6Ly93d3cuc3ByaW5nZnJhbWV3b3JrLm9yZy9zY2hlbWEvYmVhbnMKICAgICAgIGh0dHA6Ly93d3cuc3ByaW5nZnJhbWV3b3JrLm9yZy9zY2hlbWEvYmVhbnMvc3ByaW5nLWJlYW5zLnhzZAogICAgICAgaHR0cDovL3d3dy5zcHJpbmdmcmFtZXdvcmsub3JnL3NjaGVtYS91dGlsCiAgICAgICBodHRwOi8vd3d3LnNwcmluZ2ZyYW1ld29yay5vcmcvc2NoZW1hL3V0aWwvc3ByaW5nLXV0aWwueHNkIj4KCiAgICA8YmVhbiBjbGFzcyA9ICJvcmcuYXBhY2hlLmlnbml0ZS5jb25maWd1cmF0aW9uLklnbml0ZUNvbmZpZ3VyYXRpb24iPgogICAgICAgIDxwcm9wZXJ0eSBuYW1lID0gImRpc2NvdmVyeVNwaSI+CiAgICAgICAgICAgIDxiZWFuIGNsYXNzID0gIm9yZy5hcGFjaGUuaWduaXRlLnNwaS5kaXNjb3ZlcnkudGNwLlRjcERpc2NvdmVyeVNwaSI+CiAgICAgICAgICAgICAgICA8cHJvcGVydHkgbmFtZSA9ICJpcEZpbmRlciI+CiAgICAgICAgICAgICAgICAgICAgPGJlYW4gY2xhc3MgPSAib3JnLmFwYWNoZS5pZ25pdGUuc3BpLmRpc2NvdmVyeS50Y3AuaXBmaW5kZXIua3ViZXJuZXRlcy5UY3BEaXNjb3ZlcnlLdWJlcm5ldGVzSXBGaW5kZXIiPgogICAgICAgICAgICAgICAgICAgICAgICA8cHJvcGVydHkgbmFtZT0ibmFtZXNwYWNlIiB2YWx1ZT0ibXktbmFtZXNwYWNlIi8+CiAgICAgICAgICAgICAgICAgICAgICAgIDxwcm9wZXJ0eSBuYW1lPSJzZXJ2aWNlTmFtZSIgdmFsdWU9Imlnbml0ZS1zZXJ2aWNlIi8+CiAgICAgICAgICAgICAgICAgICAgPC9iZWFuPgogICAgICAgICAgICAgICAgPC9wcm9wZXJ0eT4KICAgICAgICAgICAgPC9iZWFuPgogICAgICAgIDwvcHJvcGVydHk+CiAgICAgICAgPCEtLSBFbmFibGluZyBBcGFjaGUgSWduaXRlIG5hdGl2ZSBwZXJzaXN0ZW5jZS4gLS0+CiAgICAgICAgPHByb3BlcnR5IG5hbWUgPSAiZGF0YVN0b3JhZ2VDb25maWd1cmF0aW9uIj4KICAgICAgICAgICAgPGJlYW4gY2xhc3MgPSAib3JnLmFwYWNoZS5pZ25pdGUuY29uZmlndXJhdGlvbi5EYXRhU3RvcmFnZUNvbmZpZ3VyYXRpb24iPgogICAgICAgICAgICAgICAgPHByb3BlcnR5IG5hbWUgPSAiZGVmYXVsdERhdGFSZWdpb25Db25maWd1cmF0aW9uIj4KICAgICAgICAgICAgICAgICAgICA8YmVhbiBjbGFzcyA9ICJvcmcuYXBhY2hlLmlnbml0ZS5jb25maWd1cmF0aW9uLkRhdGFSZWdpb25Db25maWd1cmF0aW9uIj4KICAgICAgICAgICAgICAgICAgICAgICAgPHByb3BlcnR5IG5hbWUgPSAicGVyc2lzdGVuY2VFbmFibGVkIiB2YWx1ZSA9ICJ0cnVlIi8+CiAgICAgICAgICAgICAgICAgICAgPC9iZWFuPgogICAgICAgICAgICAgICAgPC9wcm9wZXJ0eT4KICAgICAgICAgICAgICAgIDxwcm9wZXJ0eSBuYW1lID0gInN0b3JhZ2VQYXRoIiB2YWx1ZSA9ICIvZGF0YS9pZ25pdGUvc3RvcmFnZSIvPgogICAgICAgICAgICAgICAgPHByb3BlcnR5IG5hbWUgPSAid2FsUGF0aCIgdmFsdWUgPSAiL2RhdGEvaWduaXRlL2RiL3dhbCIvPgogICAgICAgICAgICAgICAgPHByb3BlcnR5IG5hbWUgPSAid2FsQXJjaGl2ZVBhdGgiIHZhbHVlID0gIi9kYXRhL2lnbml0ZS9kYi93YWwvYXJjaGl2ZSIvPgogICAgICAgICAgICA8L2JlYW4+CiAgICAgICAgPC9wcm9wZXJ0eT4KICAgIDwvYmVhbj4KPC9iZWFucz4=
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
# Name of Ignite Service used by Kubernetes IP finder.
# The name must be equal to TcpDiscoveryKubernetesIpFinder.serviceName.
  name: ignite-service
  namespace: my-namespace
spec:
  clusterIP: None # custom value.
  ports:
    - port: 9042 # custom value.
  selector:
  # Must be equal to one of the labels set in Ignite pods'
  # deployement configuration.
    app: ignite-service
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
# Custom Ignite cluster's name.
 name: ignite-service
 namespace: my-namespace
spec:
# A number of Ignite pods to be started by Kubernetes initially.
 replicas: 1
 template:
  metadata:
    labels:
      # This label has to be added to the selector's section of
      # ignite-service.yaml so that the Kubernetes Ignite lookup service
      # can easily track all Ignite pods available deployed so far.
      app: ignite-service
  spec:
    serviceAccountName: ignite-service
    volumes:
      # Custom name for the storage that holds Ignite's configuration
      # which is example-kube.xml.
      - name: ignite-storage
        persistentVolumeClaim:
         # Must be equal to the PersistentVolumeClaim created before.
         claimName: ignite-service-volume-claim-blr3
      - name: ignite-files
        secret:
          secretName: ignite-files
    containers:
      # Custom Ignite pod name.
    - name: ignite-node
      # Ignite Docker image. Kubernetes IP finder is supported starting from
      # Apache Ignite 2.6.0
      image: apacheignite/ignite:2.6.0
      lifecycle:
            postStart:
              exec:
                command: ['/bin/sh', '/opt/ignite/apache-ignite-fabric/bin/control.sh', '--activate']
      env:
      # Ignite's Docker image parameter. Adding the jar file that
      # contain TcpDiscoveryKubernetesIpFinder implementation.
      - name: OPTION_LIBS
        value: ignite-kubernetes
      # Ignite's Docker image parameter. Passing the Ignite configuration
      # to use for an Ignite pod.
      - name: CONFIG_URI
        value: file:///etc/ignite-files/ignite-config.xml
      - name: ENV
        value: my-namespace
      ports:
      # Ports to open.
      # Might be optional depending on your Kubernetes environment.
      - containerPort: 11211 # REST port number.
      - containerPort: 47100 # communication SPI port number.
      - containerPort: 47500 # discovery SPI port number.
      - containerPort: 49112 # JMX port number.
      - containerPort: 10800 # SQL port number.
      volumeMounts:
      # Mounting the storage with the Ignite configuration.
      - mountPath: "/data/ignite"
        name: ignite-storage
      - name: ignite-files
        mountPath: "/etc/ignite-files"

我在stackoverflow上看到了一些类似的链接,按照提出的解决方案进行了操作,但仍然没有解决问题。如果有任何指针或建议将是非常有帮助的!


1
你能分享一下你的Ignite配置吗? - Arun P Johny
更新了包含Ignite完整部署配置的deployment.yml。 - Kaps
请检查 kubectl -n my-namespace get svckubectl -n my-namespace get ep,它是否列出了名称为 ignite-service 的服务。 - Arun P Johny
两个命令都列出了我的服务。第一个命令列出了集群IP为None的服务。另一个命令以ip:port的格式显示终端节点。 - Kaps
2个回答

6
根据URL,IP查找器试图使用名为ignite的服务,而您是通过ignite-service命名创建该服务的。

您应在IP查找器配置中提供命名空间和服务名称:
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder">
    <property name="namespace" value="my-namespace"/>
    <property name="serviceName" value="ignite-service"/>
</bean>

@ValentinKulichenko,在问题中更新了最新的部署文件配置。 - Kaps
@ValentinKulichenko,你对此有什么想法吗? - Kaps
@ValentinKulichenko,没错,我在启动ignite集群时没有看到任何错误。但是试图连接到ignite的组件失败,并显示错误“Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/my-namespace/endpoints/ignite-service”。该错误表明我们需要为ignite部署添加一些额外的内容(?)。不幸的是,我无法访问该组件的yml文件,但它应该可以工作。您能够连接到Google云中的ignite吗? - Kaps
@Kaps,它是否在同一个Kubernetes集群中运行?客户端部署是否可能存在相同的问题(未指定serviceAccountName)?如果您无法访问yaml文件,则很难确定出错原因。如果您没有看到它,不确定为什么您会认为当前配置应该工作。 - Valentin Kulichenko
@ValentinKulichenko 是的,它们都在同一k8s集群中运行。我怎样才能证明客户端组件部署有问题?客户端组件需要我的Ignite的serviceName和namespace,但是在尝试连接到Ignite集群时出现上述错误。如果客户端和Ignite集群的serviceAccountName不同,会有什么问题? - Kaps
显示剩余5条评论

1

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接