在Luigi Python中，“资源”指什么？

Question

在Web界面和https://github.com/spotify/luigi/blob/master/luigi/task.py中，可以看到Task可以有“资源”。在一个Task类中还有一个名为process_resources()的占位符函数，它只返回空字典，这个字典就是资源。那么这个神秘的资源是什么呢？

- Peter Smit

---

我不知道我有一个以我的名字命名的Python模块！ - Luigi

Answer 1

我没有测试过这个，但它看起来是一个任意值，可以被调度器用来决定是否基于配置中的值来限制作业。从文档中得知：

This section can contain arbitrary keys. Each of these specifies the amount of a global resource that the scheduler can allow workers to use. The scheduler will prevent running jobs with resources specified from exceeding the counts in this section. Unspecified resources are assumed to have limit 1. Example resources section for a configuration with 2 hive resources and 1 mysql resource:

[resources]
hive: 2
mysql: 1

Note that it was not necessary to specify the 1 for mysql here, but it is good practice to do so when you have a fixed set of resources.

- danpelota