尝试将“hello world”写入Airflow日志(Airflow 1.10.3)。根据这里和这里给出的解决方案,我应该只需要import logging
和logging.info('hello world')
。但是这对我似乎不起作用。
import logging
from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.python_operator import PythonOperator
default_args = {
'owner': 'benten',
'depends_on_past': False,
'start_date': datetime(2019, 7, 25),
'email_on_failure': False,
'retries': 1,
'retry_delay': timedelta(minutes=1),
}
def logging_is_fun():
logging.debug("hellow world")
logging.info("hello world")
logging.critical("hello world")
return None
with DAG('fun_logs', schedule_interval='45 * * * *', default_args=default_args) as dag:
log_task = PythonOperator(python_callable=logging_is_fun, task_id='log_test_task')
我手动触发了DAG,任务成功执行。但是当我检查日志时,只看到以下内容:
*** Reading local file: /home/ubuntu/airflow/logs/fun_logs/log_test_task/2019-08-31T19:22:49.653712+00:00/1.log
我的那些"hello world"语句去哪了? 我不指望能看到所有的语句,因为我的日志级别设置。但我希望能看到关键信息。
我的airflow.cfg文件中包含以下内容(据我所知,所有设置均为默认设置):
# The folder where airflow should store its log files
# This path must be absolute
base_log_folder = /home/ubuntu/airflow/logs
# Airflow can store logs remotely in AWS S3, Google Cloud Storage or Elastic Search.
# Users must supply an Airflow connection id that provides access to the storage
# location. If remote_logging is set to true, see UPDATING.md for additional
# configuration requirements.
remote_logging = False
remote_log_conn_id =
remote_base_log_folder =
encrypt_s3_logs = False
# Logging level
logging_level = WARN
fab_logging_level = WARN
# Logging class
# Specify the class that will specify the logging configuration
# This class has to be on the python classpath
# logging_config_class = my.path.default_local_settings.LOGGING_CONFIG
logging_config_class =
# Log format
log_format = [%%(asctime)s] {%%(filename)s:%%(lineno)d} %%(levelname)s - %%(message)s
simple_log_format = %%(asctime)s %%(levelname)s - %%(message)s
# Log filename format
log_filename_template = {{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ try_number }}.log
log_processor_filename_template = {{ filename }}.log
dag_processor_manager_log_location = /home/ubuntu/airflow/logs/dag_processor_manager/dag_processor_manager.log