我正在尝试使用etcd配置启动kubernetes api服务器(kubernetes使用go-etcd,该库有一个从配置文件读取所有参数的方法):
{
"cluster": {
"machines": [ "https://my-public-hostname:2379" ]
},
"config": {
"certFile": "/etc/ssl/etcd/client.pem",
"keyFile": "/etc/ssl/etcd/client.key.pem",
"caCertFiles": [
"/etc/ssl/etcd/ca.pem"
],
"timeout": 5,
"consistency": "WEAK"
}
}
但是在kube-apiserver中失败,因为它无法成功访问etcd。我认为这是因为它试图同步集群...但我不知道。
我已经使用内部IP创建了一个(etcd)集群,用于广告和客户端地址,除了监听客户端URLs设置为0.0.0.0/0。此外,整个集群都在负载均衡器后面,该负载均衡器可通过my-public-hostname
访问。
在容器内(因为我正在使用hyperkube
),除非我设置“--no-sync”参数,否则etcdctl
将不起作用。如果我不使用该参数使用etcdctl,则会像kube-apiserver一样出现错误。但是我无法检查kubernetes中执行集群同步的代码片段...
有什么想法吗?
提前感谢您。
编辑:
这似乎是与kubernetes(https://github.com/coreos/go-etcd)中当前etcd客户端相关的错误,这不是最新的(https://github.com/coreos/etcd/client)。我进行了实证测试,“etcd/client”有效,但“go-etcd”则不行,您可以在此处检查此测试:https://github.com/glerchundi/etcd-go-clients-test。
值得注意的是,正在进行将go-etcd迁移到kubernetes中的etcd/client:
https://github.com/kubernetes/kubernetes/issues/11962
可以有kubernetes团队中的任何人确认这一点吗?
附录1
我正在尝试在CoreOS中运行kubernetes,并且:flannel
可用,locksmithd
可用,fleet
可用(它们使用相同的etcd客户端凭据访问),因此可能与kubernetes如何访问etcd端点有关。
附录2(这些命令在hyperkube容器内执行,具体来说是这一个:gcr.io/google_containers/hyperkube:v1.0.6
)
没有--no-sync的etcdctl会失败输出:
root@98b2524464f1:/# etcdctl --cert-file="/etc/ssl/etcd/client.pem" --key-file="/etc/ssl/etcd/client.key.pem" --ca-file="/etc/ssl/etcd/ca.pem" --peers="http//my-public-hostname:2379" ls /
Error: 501: All the given peers are not reachable (failed to propose on members [https://10.1.0.1:2379 https://10.1.0.0:2379 https://10.1.0.2:2379] twice [last error: Get https://10.1.0.0:2379/v2/keys/?quorum=false&recursive=false&sorted=false: dial tcp 10.1.0.0:2379: i/o timeout]) [0]
同时 kube-apiserver 也需要这个:
root@98b2524464f1:/# /hyperkube \
apiserver \
--bind-address=0.0.0.0 \
--etcd_config=/etc/kubernetes/ssl/etcd.json \
--allow-privileged=true \
--service-cluster-ip-range=10.3.0.0/24 \
--secure_port=443 \
--advertise-address=10.0.0.2 \
--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota \
--tls-cert-file=/etc/kubernetes/ssl/apiserver.pem \
--tls-private-key-file=/etc/kubernetes/ssl/apiserver.key.pem \
--client-ca-file=/etc/kubernetes/ssl/ca.pem \
--service-account-key-file=/etc/kubernetes/ssl/apiserver.key.pem
F1002 09:47:29.348527 384 controller.go:80] Unable to perform initial IP allocation check: unable to refresh the service IP block: 501: All the given peers are not reachable (failed to propose on members [https://my-public-hostname:2379] twice [last error: Get https://my-public-hostname:2379/v2/keys/registry/ranges/serviceips?quorum=false&recursive=false&sorted=false: dial tcp: i/o timeout]) [0]
附录3
etcd #0:
etcd2:
name: etcd0
initial-cluster-state: new
initial-cluster: etcd0=http://10.1.0.0:2380,etcd1=http://10.1.0.1:2380,etcd2=http://10.1.0.2:2380
data-dir: /var/lib/etcd2
advertise-client-urls: https://10.1.0.0:2379
initial-advertise-peer-urls: http://10.1.0.0:2380
listen-client-urls: https://0.0.0.0:2379
listen-peer-urls: http://10.1.0.0:2380
client-cert-auth: true
trusted-ca-file: /etc/ssl/etcd/certs/ca-chain.cert.pem
cert-file: /etc/ssl/etcd/certs/etcd-server.cert.pem
key-file: /etc/ssl/etcd/private/etcd-server.key.pem
etcd #1:
etcd2:
name: etcd1
initial-cluster-state: new
initial-cluster: etcd0=http://10.1.0.0:2380,etcd1=http://10.1.0.1:2380,etcd2=http://10.1.0.2:2380
data-dir: /var/lib/etcd2
advertise-client-urls: https://10.1.0.1:2379
initial-advertise-peer-urls: http://10.1.0.1:2380
listen-client-urls: https://0.0.0.0:2379
listen-peer-urls: http://10.1.0.1:2380
client-cert-auth: true
trusted-ca-file: /etc/ssl/etcd/certs/ca-chain.cert.pem
cert-file: /etc/ssl/etcd/certs/etcd-server.cert.pem
key-file: /etc/ssl/etcd/private/etcd-server.key.pem
etcd #2:
etcd2:
name: etcd2
initial-cluster-state: new
initial-cluster: etcd0=http://10.1.0.0:2380,etcd1=http://10.1.0.1:2380,etcd2=http://10.1.0.2:2380
data-dir: /var/lib/etcd2
advertise-client-urls: https://10.1.0.2:2379
initial-advertise-peer-urls: http://10.1.0.2:2380
listen-client-urls: https://0.0.0.0:2379
listen-peer-urls: http://10.1.0.2:2380
client-cert-auth: true
trusted-ca-file: /etc/ssl/etcd/certs/ca-chain.cert.pem
cert-file: /etc/ssl/etcd/certs/etcd-server.cert.pem
key-file: /etc/ssl/etcd/private/etcd-server.key.pem