如何安排一个定时任务来执行 kubectl 命令？

Question

如何安排一个定时任务来执行 kubectl 命令？

21

我希望每隔 5 分钟运行以下 kubectl 命令：

kubectl patch deployment runners -p '{"spec":{"template":{"spec":{"containers":[{"name":"jp-runner","env":[{"name":"START_TIME","value":"'$(date +%s)'"}]}]}}}}' -n jp-test

为此，我创建了以下cronjob：

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "*/5 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: busybox
            args:
            - /bin/sh
            - -c
            - kubectl patch deployment runners -p '{"spec":{"template":{"spec":{"containers":[{"name":"jp-runner","env":[{"name":"START_TIME","value":"'$(date +%s)'"}]}]}}}}' -n jp-test
          restartPolicy: OnFailure

但是它无法启动容器，显示消息：

Back-off restarting failed container

而且还有错误代码127：

State:          Terminated
      Reason:       Error
      Exit Code:    127

根据我的查询，错误代码127表示命令不存在。我如何将kubectl命令作为cron job运行？我有什么遗漏的吗？

注意：我之前发布了一个类似的问题(无停机时间的Kubernetes Pod定期重启)，但那更多地是将主要部署本身作为cron job，这里我正在尝试使用CronJob运行kubectl命令（执行重启）-所以我认为最好单独发布。

kubectl describe cronjob hello -n jp-test:

Name:                       hello
Namespace:                  jp-test
Labels:                     <none>
Annotations:                kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"batch/v1beta1","kind":"CronJob","metadata":{"annotations":{},"name":"hello","namespace":"jp-test"},"spec":{"jobTemplate":{"spec":{"templ...
Schedule:                   */5 * * * *
Concurrency Policy:         Allow
Suspend:                    False
Starting Deadline Seconds:  <unset>
Selector:                   <unset>
Parallelism:                <unset>
Completions:                <unset>
Pod Template:
  Labels:  <none>
  Containers:
   hello:
    Image:      busybox
    Port:       <none>
    Host Port:  <none>
    Args:
      /bin/sh
      -c
      kubectl patch deployment runners -p '{"spec":{"template":{"spec":{"containers":[{"name":"jp-runner","env":[{"name":"START_TIME","value":"'$(date +%s)'"}]}]}}}}' -n jp-test
    Environment:     <none>
    Mounts:          <none>
  Volumes:           <none>
Last Schedule Time:  Wed, 27 Feb 2019 14:10:00 +0100
Active Jobs:         hello-1551273000
Events:
  Type    Reason            Age   From                Message
  ----    ------            ----  ----                -------
  Normal  SuccessfulCreate  6m    cronjob-controller  Created job hello-1551272700
  Normal  SuccessfulCreate  1m    cronjob-controller  Created job hello-1551273000
  Normal  SawCompletedJob   16s   cronjob-controller  Saw completed job: hello-1551272700

kubectl describe job hello -v=5 -n jp-test

Name:           hello-1551276000
Namespace:      jp-test
Selector:       controller-uid=fa009d78-3a97-11e9-ae31-ac1f6b1a0950
Labels:         controller-uid=fa009d78-3a97-11e9-ae31-ac1f6b1a0950
                job-name=hello-1551276000
Annotations:    <none>
Controlled By:  CronJob/hello
Parallelism:    1
Completions:    1
Start Time:     Wed, 27 Feb 2019 15:00:02 +0100
Pods Statuses:  0 Running / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  controller-uid=fa009d78-3a97-11e9-ae31-ac1f6b1a0950
           job-name=hello-1551276000
  Containers:
   hello:
    Image:      busybox
    Port:       <none>
    Host Port:  <none>
    Args:
      /bin/sh
      -c
      kubectl patch deployment runners -p '{"spec":{"template":{"spec":{"containers":[{"name":"jp-runner","env":[{"name":"START_TIME","value":"'$(date +%s)'"}]}]}}}}' -n jp-test
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:
  Type     Reason                Age              From            Message
  ----     ------                ----             ----            -------
  Normal   SuccessfulCreate      7m               job-controller  Created pod: hello-1551276000-lz4dp
  Normal   SuccessfulDelete      1m               job-controller  Deleted pod: hello-1551276000-lz4dp
  Warning  BackoffLimitExceeded  1m (x2 over 1m)  job-controller  Job has reached the specified backoff limit

Name:           hello-1551276300
Namespace:      jp-test
Selector:       controller-uid=ad52e87a-3a98-11e9-ae31-ac1f6b1a0950
Labels:         controller-uid=ad52e87a-3a98-11e9-ae31-ac1f6b1a0950
                job-name=hello-1551276300
Annotations:    <none>
Controlled By:  CronJob/hello
Parallelism:    1
Completions:    1
Start Time:     Wed, 27 Feb 2019 15:05:02 +0100
Pods Statuses:  1 Running / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  controller-uid=ad52e87a-3a98-11e9-ae31-ac1f6b1a0950
           job-name=hello-1551276300
  Containers:
   hello:
    Image:      busybox
    Port:       <none>
    Host Port:  <none>
    Args:
      /bin/sh
      -c
      kubectl patch deployment runners -p '{"spec":{"template":{"spec":{"containers":[{"name":"jp-runner","env":[{"name":"START_TIME","value":"'$(date +%s)'"}]}]}}}}' -n jp-test
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:
  Type    Reason            Age   From            Message
  ----    ------            ----  ----            -------
  Normal  SuccessfulCreate  2m    job-controller  Created pod: hello-1551276300-8d5df

- Chillax

嗨，kubectl需要使用pod内的服务帐户对apiserver进行身份验证。这可能是错误的原因。 - Suresh Vishnoi

你能提供 kubectl describe job <cron_job_name> -v=5 吗？ - Crou

@Crou 我已经更新了我的问题并添加了细节。 - Chillax

@Chillax 我请求的是一个“工作”，而不是“cronjob”，我想看到由cronjob生成的工作的详细信息，如果您可以的话。 - Crou

可能是Kubernetes - 如何在容器内运行kubectl命令？的重复问题。 - Shai Katz

显示剩余2条评论

2个回答

-2

您需要使CronJob的容器下载集群配置，以便您可以对其运行kubectl命令。以下是一个示例：

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: drupal-cron
spec:
  schedule: "*/5 * * * *"
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: drupal-cron
              image: juampynr/digital-ocean-cronjob:latest
              env:
                - name: DIGITALOCEAN_ACCESS_TOKEN
                  valueFrom:
                    secretKeyRef:
                      name: api
                      key: key
              command: ["/bin/bash","-c"]
              args:
                - doctl kubernetes cluster kubeconfig save drupster;
                  POD_NAME=$(kubectl get pods -l tier=frontend -o=jsonpath='{.items[0].metadata.name}');
                  kubectl exec $POD_NAME -c drupal -- vendor/bin/drush core:cron;
          restartPolicy: OnFailure

我在另一个帖子中发布了一个回答，描述了我如何完成这个任务：https://dev59.com/n1gR5IYBdhLWcg3wjdwF#62321138

- Juampy NR

网页内容由stack overflow 提供, 点击上面的

可以查看英文原文，
原文链接

- Crou · Accepted Answer

简而言之，BusyBox没有安装kubectl。

您可以使用命令kubectl run -i --tty busybox --image=busybox -- sh来检查它，这将运行一个交互式的BusyBox pod。

我建议使用bitnami/kubectl:latest。

还要记住，您需要设置适当的RBAC，因为否则您会收到Error from server (Forbidden): services is forbidden的错误信息。

您可以尝试类似以下的方法：

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: jp-test
  name: jp-runner
rules:
- apiGroups:
  - extensions
  - apps
  resources:
  - deployments
  verbs:
  - 'patch'

---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jp-runner
  namespace: jp-test
subjects:
- kind: ServiceAccount
  name: sa-jp-runner
  namespace: jp-test
roleRef:
  kind: Role
  name: jp-runner
  apiGroup: ""

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-jp-runner
  namespace: jp-test

---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "*/5 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: sa-jp-runner
          containers:
          - name: hello
            image: bitnami/kubectl:latest
            command:
            - /bin/sh
            - -c
            - kubectl patch deployment runners -p '{"spec":{"template":{"spec":{"containers":[{"name":"jp-runner","env":[{"name":"START_TIME","value":"'$(date +%s)'"}]}]}}}}' -n jp-test
          restartPolicy: OnFailure