如何将sys.stdout复制到日志文件中?

168

编辑:由于看起来要么没有解决方案,要么我正在做一些如此非标准的事情以至于没有人知道 - 我将修订我的问题,还要问:在 Python 应用程序进行大量系统调用时,实现记录日志的最佳方法是什么?

我的应用程序有两种模式。在交互模式下,我希望所有输出都能显示在屏幕上并写入日志文件,包括任何系统调用的输出。在守护进程模式下,所有输出都写入日志中。使用 os.dup2() 守护进程模式非常好用。但是在交互模式下,我找不到“tee”所有输出到日志的方法,而无需修改每个系统调用。


换句话说,我想要一个类似命令行工具 'tee' 的功能,用于捕获 Python 应用程序生成的所有输出,包括系统调用输出

澄清一下:

为了重定向所有输出,我做了以下操作,效果非常好:

# open our log file
so = se = open("%s.log" % self.name, 'w', 0)

# re-open stdout without buffering
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

# redirect stdout and stderr to the log file opened above
os.dup2(so.fileno(), sys.stdout.fileno())
os.dup2(se.fileno(), sys.stderr.fileno())

这样做的好处是不需要在代码中特别指定打印调用。此外,代码还运行了一些shell命令,因此不必单独处理它们的输出也很方便。

简单来说,我想做同样的事情,只是要“复制”而不是重定向。

一开始我认为只需简单地反转dup2就可以了,为什么不行?以下是我的测试:

import os, sys

### my broken solution:
so = se = open("a.log", 'w', 0)
sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

os.dup2(sys.stdout.fileno(), so.fileno())
os.dup2(sys.stderr.fileno(), se.fileno())
###

print("foo bar")

os.spawnve("P_WAIT", "/bin/ls", ["/bin/ls"], {})
os.execve("/bin/ls", ["/bin/ls"], os.environ)

文件"a.log"应该与屏幕上显示的内容完全相同。


如果您查看man页面(http://www.manpagez.com/man/2/dup2/),dup2的第二个参数始终关闭(如果已经打开)。因此,在您的“错误解决方案”中,它正在关闭so和se,然后重新分配它们的文件描述符给sys.stdout。 - Jacob Gabrielson
1
关于你的编辑:这并不罕见,我以前也做过类似的事情(在其他编程语言中)。虽然Unix允许为同一个文件句柄设置多个“别名”,但它不会将文件句柄“分割”(复制到其他多个文件句柄)。所以你必须自己实现“tee”(或者直接使用“tee”,请参考我的简单回答)。 - Jacob Gabrielson
我认为JohnT的答案比实际被接受的更好。你可能想要更改已接受的答案。 - Phong
我正在做一些非常非标准的事情 - 你真的是这样,人们只是将他们的日志发送到stderr并从命令行处理。 - khachik
18个回答

5

我已经使用Jacob Gabrielson的被接受的解决方案大约1年了,但现在我的一个用户想在Windows上使用它。看着其他提议的回答,我认为大多数这些答案都没有成功地捕获生成进程的输出(正如原帖子中加粗的部分所示); 我认为唯一的方法是使用os.dup2()。我想我已经找到了如何回答原作者的确切问题,而且不使用Unix特定的工具tee:我现在可以捕获我的Python程序的所有输出,包括任何生成的shell命令。这适用于Windows、Mac和Linux。代码如下:

import os, sys, threading, platform

class StreamCapture:
    def __init__(self,stream,writer,echo=True,monkeypatch=None):
        self.active = True
        self.writer = writer
        self.stream = stream
        self.fd = stream.fileno()
        self.echo = echo
        (r,w) = os.pipe()
        self.pipe_read_fd = r
        self.pipe_write_fd = w
        self.dup_fd = os.dup(self.fd)
        os.dup2(w,self.fd)
        self.monkeypatch = monkeypatch if monkeypatch is not None else platform.system()=='Windows'
        if self.monkeypatch:
            self.oldwrite = stream.write
            stream.write = lambda z: os.write(self.fd,z.encode() if type(z)==str else z)
        t = threading.Thread(target=self.printer)
        self.thread = t
        t.start()
    def printer(self):
        while True:
            data = os.read(self.pipe_read_fd,100000)
            if(len(data)==0):
                self.writer.close()
                os.close(self.dup_fd)
                os.close(self.pipe_read_fd)
                return
            self.writer.write(data)
            if self.echo:
                os.write(self.dup_fd,data)
    def close(self):
        if not self.active:
            return
        self.active = False
        self.stream.flush()
        if self.monkeypatch:
            self.stream.write = self.oldwrite
        os.dup2(self.dup_fd,self.fd)
        os.close(self.pipe_write_fd)
    def __enter__(self):
        return self
    def __exit__(self,a,b,c):
        self.close()

你可以这样使用它(注意,除了Jacob Gabrielson的解决方案之外,其他解决方案都无法捕捉到这个困难的情况):
print("This does not get saved to the log file")
with StreamCapture(sys.stdout,open('logfile.txt','wb')):
        os.write(sys.stdout.fileno(),b"Hello, captured world!\n")
        os.system('echo Hello from the shell')     # Hard case
        print("More capturing")
print("This also does not get saved to the log file")

这不是一个简短而甜美的答案,但我尽力让它简明扼要,也尽可能让它变得简单。它之所以复杂,有以下几个原因:
  1. 由于我不能使用tee,我必须在我的Python进程中执行tee的任务。我不清楚是否有可移植的方法来执行fork()并与os.pipe()进行通信(这里指出在Windows中很难共享文件描述符),所以我决定使用threading
  2. 在Windows中,当sys.stdoutsys.stderr的底层fileno()被通过os.dup2()重定向到os.pipe()时,Python解释器会立即崩溃,在第一次print(...)命令之后。
  3. 仅在Windows中,为了解决解释器崩溃的问题,我通过设置一个新函数,该函数仅调用os.write(...),来对sys.stdout.write = ...进行修补。默认情况下,我只在检测到Windows时执行此操作。因为我进行了修补,我希望这将触及所有缓存引用到sys.stdout的内容。我选择了这种修补方法,而不是分配一个全新的流,比如sys.stdout=...,因为我担心旧sys.stdout的副本可能仍然被缓存在解释器的各个部分中,但我猜想sys.stdout.write不太可能被直接缓存。
  4. 如果你使处理管道输出的线程成为守护进程,那么当主线程完成时,该线程会立即被杀死,但这并不能保证所有输出都已写入日志文件。实际上,有必要使这些辅助线程成为守护进程,并让它们在管道关闭时自动终止。
我其实不完全确定我是否正确地处理了所有角落情况——与敏感操作系统功能交互的多线程代码很难编写。不过,它迄今为止通过了我的测试。由于它有点复杂,我创建了一个PyPI软件包:
pip install streamcapture

Github的地址在这里


1
我在单元测试中使用它来断言外部非Python库的输出,它完美地工作,谢谢。 - Remy F
不客气! - Sébastien Loisel

4

使用logging模块的另一种解决方案:

import logging
import sys

log = logging.getLogger('stdxxx')

class StreamLogger(object):

    def __init__(self, stream, prefix=''):
        self.stream = stream
        self.prefix = prefix
        self.data = ''

    def write(self, data):
        self.stream.write(data)
        self.stream.flush()

        self.data += data
        tmp = str(self.data)
        if '\x0a' in tmp or '\x0d' in tmp:
            tmp = tmp.rstrip('\x0a\x0d')
            log.info('%s%s' % (self.prefix, tmp))
            self.data = ''


logging.basicConfig(level=logging.INFO,
                    filename='text.log',
                    filemode='a')

sys.stdout = StreamLogger(sys.stdout, '[stdout] ')

print 'test for stdout'

3

以上的回答似乎都没有真正回答所提出的问题。我知道这是一个旧帖子,但我认为这个问题比每个人想象的要简单得多:

class tee_err(object):

 def __init__(self):
    self.errout = sys.stderr

    sys.stderr = self

    self.log = 'logfile.log'
    log = open(self.log,'w')
    log.close()

 def write(self, line):

    log = open(self.log,'a')
    log.write(line)
    log.close()   

    self.errout.write(line)

现在这将会把所有内容都重复输出到正常的sys.stderr处理程序和你的文件里。针对sys.stdout,请创建另一个名为tee_out的类。


3
根据@user5359531在@John T的回答下的评论请求,这里提供链接讨论修订版参考帖子的副本:
Issue of redirecting the stdout to both file and screen
Gabriel Genellina gagsl-py2 at yahoo.com.ar
Mon May 28 12:45:51 CEST 2007

    Previous message: Issue of redirecting the stdout to both file and screen
    Next message: Formal interfaces with Python
    Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

En Mon, 28 May 2007 06:17:39 -0300, 人言落日是天涯,望极天涯不见家
<kelvin.you at gmail.com> escribió:

> I wanna print the log to both the screen and file, so I simulatered a
> 'tee'
>
> class Tee(file):
>
>     def __init__(self, name, mode):
>         file.__init__(self, name, mode)
>         self.stdout = sys.stdout
>         sys.stdout = self
>
>     def __del__(self):
>         sys.stdout = self.stdout
>         self.close()
>
>     def write(self, data):
>         file.write(self, data)
>         self.stdout.write(data)
>
> Tee('logfile', 'w')
> print >>sys.stdout, 'abcdefg'
>
> I found that it only output to the file, nothing to screen. Why?
> It seems the 'write' function was not called when I *print* something.

You create a Tee instance and it is immediately garbage collected. I'd
restore sys.stdout on Tee.close, not __del__ (you forgot to call the
inherited __del__ method, btw).
Mmm, doesn't work. I think there is an optimization somewhere: if it looks
like a real file object, it uses the original file write method, not yours.
The trick would be to use an object that does NOT inherit from file:

import sys
class TeeNoFile(object):
     def __init__(self, name, mode):
         self.file = open(name, mode)
         self.stdout = sys.stdout
         sys.stdout = self
     def close(self):
         if self.stdout is not None:
             sys.stdout = self.stdout
             self.stdout = None
         if self.file is not None:
             self.file.close()
             self.file = None
     def write(self, data):
         self.file.write(data)
         self.stdout.write(data)
     def flush(self):
         self.file.flush()
         self.stdout.flush()
     def __del__(self):
         self.close()

tee=TeeNoFile('logfile', 'w')
print 'abcdefg'
print 'another line'
tee.close()
print 'screen only'
del tee # should do nothing

--
Gabriel Genellina

1
我正在编写一个脚本来运行命令行脚本。(因为在某些情况下,没有任何可替代的Linux命令 - 例如rsync的情况。)
我真正想要的是在尽可能多地使用默认的Python记录机制的情况下,仍然捕获任何未预料到的错误。
这段代码似乎可以解决问题。它可能不是特别优雅或高效(虽然它没有使用string + = string,所以至少没有这种具体的潜在瓶颈)。我发布它以防它给其他人带来有用的思路。
import logging
import os, sys
import datetime

# Get name of module, use as application name
try:
  ME=os.path.split(__file__)[-1].split('.')[0]
except:
  ME='pyExec_'

LOG_IDENTIFIER="uuu___( o O )___uuu "
LOG_IDR_LENGTH=len(LOG_IDENTIFIER)

class PyExec(object):

  # Use this to capture all possible error / output to log
  class SuperTee(object):
      # Original reference: http://mail.python.org/pipermail/python-list/2007-May/442737.html
      def __init__(self, name, mode):
          self.fl = open(name, mode)
          self.fl.write('\n')
          self.stdout = sys.stdout
          self.stdout.write('\n')
          self.stderr = sys.stderr

          sys.stdout = self
          sys.stderr = self

      def __del__(self):
          self.fl.write('\n')
          self.fl.flush()
          sys.stderr = self.stderr
          sys.stdout = self.stdout
          self.fl.close()

      def write(self, data):
          # If the data to write includes the log identifier prefix, then it is already formatted
          if data[0:LOG_IDR_LENGTH]==LOG_IDENTIFIER:
            self.fl.write("%s\n" % data[LOG_IDR_LENGTH:])
            self.stdout.write(data[LOG_IDR_LENGTH:])

          # Otherwise, we can give it a timestamp
          else:

            timestamp=str(datetime.datetime.now())
            if 'Traceback' == data[0:9]:
              data='%s: %s' % (timestamp, data)
              self.fl.write(data)
            else:
              self.fl.write(data)

            self.stdout.write(data)


  def __init__(self, aName, aCmd, logFileName='', outFileName=''):

    # Using name for 'logger' (context?), which is separate from the module or the function
    baseFormatter=logging.Formatter("%(asctime)s \t %(levelname)s \t %(name)s:%(module)s:%(lineno)d \t %(message)s")
    errorFormatter=logging.Formatter(LOG_IDENTIFIER + "%(asctime)s \t %(levelname)s \t %(name)s:%(module)s:%(lineno)d \t %(message)s")

    if logFileName:
      # open passed filename as append
      fl=logging.FileHandler("%s.log" % aName)
    else:
      # otherwise, use log filename as a one-time use file
      fl=logging.FileHandler("%s.log" % aName, 'w')

    fl.setLevel(logging.DEBUG)
    fl.setFormatter(baseFormatter)

    # This will capture stdout and CRITICAL and beyond errors

    if outFileName:
      teeFile=PyExec.SuperTee("%s_out.log" % aName)
    else:
      teeFile=PyExec.SuperTee("%s_out.log" % aName, 'w')

    fl_out=logging.StreamHandler( teeFile )
    fl_out.setLevel(logging.CRITICAL)
    fl_out.setFormatter(errorFormatter)

    # Set up logging
    self.log=logging.getLogger('pyExec_main')
    log=self.log

    log.addHandler(fl)
    log.addHandler(fl_out)

    print "Test print statement."

    log.setLevel(logging.DEBUG)

    log.info("Starting %s", ME)
    log.critical("Critical.")

    # Caught exception
    try:
      raise Exception('Exception test.')
    except Exception,e:
      log.exception(str(e))

    # Uncaught exception
    a=2/0


PyExec('test_pyExec',None)

显然,如果你不像我一样容易受到幽默的影响,那么请用另一个字符串替换LOG_IDENTIFIER,这个字符串你不太可能在日志中看到。


0
这是一个上下文管理器,可以将stdout暂时复制到文件中。在我看来,这是一种改进,因为它可以重置sys.stdout并在发生异常时关闭文件,并且语法表明背景中发生了不可见的更改。对John T的解决方案进行了扩展。
class DuplicateStdout:
    def __init__(self, path):
        self.stdout = sys.stdout
        self.path = path
        self.f = None
    
    def write(self, s):
        self.stdout.write(s)
        self.f.write(s)

    def __enter__(self):
        self.f = open(self.path, "w")
        sys.stdout = self
    
    def __exit__(self, *args):
        sys.stdout = self.stdout
        self.f.close()

示例用法:

with DuplicateStdout("foo.log"):
    print("Hey") # also in foo.log

print("There") # not in foo.log

0

你也可以根据shx2的答案,使用class multifile,同时添加stderr

class Log(object):

    def __init__(self, path_log, mode="w", encoding="utf-8"):
        h = open(path_log, mode, encoding=encoding)
        sys.stdout = multifile([ sys.stdout, h ])
        sys.stderr = multifile([ sys.stderr, h ])

    def __enter__(self):
        """ Necessary if called by with (or with... as) """
        return self     # only necessary if "as"

    def __exit__(self, type, value, tb):
        """ Necessary if call by with """
        pass

    def __del__(self):
        if sys is not None:
            # restoring
            sys.stdout = sys.__stdout__
            sys.stderr = sys.__stderr__

log = Log("test.txt")
print("line 1")
print("line 2", file=sys.stderr)
del log
print("line 3 only on screen")

唯一的缺点在于Python 2.x中当出现异常时,消息似乎会丢失。 - Eric H.

-1

我为sys.stderr编写了一个完整的替代品,并且只是复制了代码,将stderr重命名为stdout,以便也可以用来替换sys.stdout

为此,我创建了与当前stderrstdout相同的对象类型,并将所有方法转发到原始系统的stderrstdout

import os
import sys
import logging

class StdErrReplament(object):
    """
        How to redirect stdout and stderr to logger in Python
        https://dev59.com/fWIk5IYBdhLWcg3wRcQo

        Set a Read-Only Attribute in Python?
        https://dev59.com/s2Af5IYBdhLWcg3wlDeb
    """
    is_active = False

    @classmethod
    def lock(cls, logger):
        """
            Attach this singleton logger to the `sys.stderr` permanently.
        """
        global _stderr_singleton
        global _stderr_default
        global _stderr_default_class_type

        # On Sublime Text, `sys.__stderr__` is set to None, because they already replaced `sys.stderr`
        # by some `_LogWriter()` class, then just save the current one over there.
        if not sys.__stderr__:
            sys.__stderr__ = sys.stderr

        try:
            _stderr_default
            _stderr_default_class_type

        except NameError:
            _stderr_default = sys.stderr
            _stderr_default_class_type = type( _stderr_default )

        # Recreate the sys.stderr logger when it was reset by `unlock()`
        if not cls.is_active:
            cls.is_active = True
            _stderr_write = _stderr_default.write

            logger_call = logger.debug
            clean_formatter = logger.clean_formatter

            global _sys_stderr_write
            global _sys_stderr_write_hidden

            if sys.version_info <= (3,2):
                logger.file_handler.terminator = '\n'

            # Always recreate/override the internal write function used by `_sys_stderr_write`
            def _sys_stderr_write_hidden(*args, **kwargs):
                """
                    Suppress newline in Python logging module
                    https://dev59.com/TWw05IYBdhLWcg3wkig-
                """

                try:
                    _stderr_write( *args, **kwargs )
                    file_handler = logger.file_handler

                    formatter = file_handler.formatter
                    terminator = file_handler.terminator

                    file_handler.formatter = clean_formatter
                    file_handler.terminator = ""

                    kwargs['extra'] = {'_duplicated_from_file': True}
                    logger_call( *args, **kwargs )

                    file_handler.formatter = formatter
                    file_handler.terminator = terminator

                except Exception:
                    logger.exception( "Could not write to the file_handler: %s(%s)", file_handler, logger )
                    cls.unlock()

            # Only create one `_sys_stderr_write` function pointer ever
            try:
                _sys_stderr_write

            except NameError:

                def _sys_stderr_write(*args, **kwargs):
                    """
                        Hides the actual function pointer. This allow the external function pointer to
                        be cached while the internal written can be exchanged between the standard
                        `sys.stderr.write` and our custom wrapper around it.
                    """
                    _sys_stderr_write_hidden( *args, **kwargs )

        try:
            # Only create one singleton instance ever
            _stderr_singleton

        except NameError:

            class StdErrReplamentHidden(_stderr_default_class_type):
                """
                    Which special methods bypasses __getattribute__ in Python?
                    https://dev59.com/OGcs5IYBdhLWcg3waTRD
                """

                if hasattr( _stderr_default, "__abstractmethods__" ):
                    __abstractmethods__ = _stderr_default.__abstractmethods__

                if hasattr( _stderr_default, "__base__" ):
                    __base__ = _stderr_default.__base__

                if hasattr( _stderr_default, "__bases__" ):
                    __bases__ = _stderr_default.__bases__

                if hasattr( _stderr_default, "__basicsize__" ):
                    __basicsize__ = _stderr_default.__basicsize__

                if hasattr( _stderr_default, "__call__" ):
                    __call__ = _stderr_default.__call__

                if hasattr( _stderr_default, "__class__" ):
                    __class__ = _stderr_default.__class__

                if hasattr( _stderr_default, "__delattr__" ):
                    __delattr__ = _stderr_default.__delattr__

                if hasattr( _stderr_default, "__dict__" ):
                    __dict__ = _stderr_default.__dict__

                if hasattr( _stderr_default, "__dictoffset__" ):
                    __dictoffset__ = _stderr_default.__dictoffset__

                if hasattr( _stderr_default, "__dir__" ):
                    __dir__ = _stderr_default.__dir__

                if hasattr( _stderr_default, "__doc__" ):
                    __doc__ = _stderr_default.__doc__

                if hasattr( _stderr_default, "__eq__" ):
                    __eq__ = _stderr_default.__eq__

                if hasattr( _stderr_default, "__flags__" ):
                    __flags__ = _stderr_default.__flags__

                if hasattr( _stderr_default, "__format__" ):
                    __format__ = _stderr_default.__format__

                if hasattr( _stderr_default, "__ge__" ):
                    __ge__ = _stderr_default.__ge__

                if hasattr( _stderr_default, "__getattribute__" ):
                    __getattribute__ = _stderr_default.__getattribute__

                if hasattr( _stderr_default, "__gt__" ):
                    __gt__ = _stderr_default.__gt__

                if hasattr( _stderr_default, "__hash__" ):
                    __hash__ = _stderr_default.__hash__

                if hasattr( _stderr_default, "__init__" ):
                    __init__ = _stderr_default.__init__

                if hasattr( _stderr_default, "__init_subclass__" ):
                    __init_subclass__ = _stderr_default.__init_subclass__

                if hasattr( _stderr_default, "__instancecheck__" ):
                    __instancecheck__ = _stderr_default.__instancecheck__

                if hasattr( _stderr_default, "__itemsize__" ):
                    __itemsize__ = _stderr_default.__itemsize__

                if hasattr( _stderr_default, "__le__" ):
                    __le__ = _stderr_default.__le__

                if hasattr( _stderr_default, "__lt__" ):
                    __lt__ = _stderr_default.__lt__

                if hasattr( _stderr_default, "__module__" ):
                    __module__ = _stderr_default.__module__

                if hasattr( _stderr_default, "__mro__" ):
                    __mro__ = _stderr_default.__mro__

                if hasattr( _stderr_default, "__name__" ):
                    __name__ = _stderr_default.__name__

                if hasattr( _stderr_default, "__ne__" ):
                    __ne__ = _stderr_default.__ne__

                if hasattr( _stderr_default, "__new__" ):
                    __new__ = _stderr_default.__new__

                if hasattr( _stderr_default, "__prepare__" ):
                    __prepare__ = _stderr_default.__prepare__

                if hasattr( _stderr_default, "__qualname__" ):
                    __qualname__ = _stderr_default.__qualname__

                if hasattr( _stderr_default, "__reduce__" ):
                    __reduce__ = _stderr_default.__reduce__

                if hasattr( _stderr_default, "__reduce_ex__" ):
                    __reduce_ex__ = _stderr_default.__reduce_ex__

                if hasattr( _stderr_default, "__repr__" ):
                    __repr__ = _stderr_default.__repr__

                if hasattr( _stderr_default, "__setattr__" ):
                    __setattr__ = _stderr_default.__setattr__

                if hasattr( _stderr_default, "__sizeof__" ):
                    __sizeof__ = _stderr_default.__sizeof__

                if hasattr( _stderr_default, "__str__" ):
                    __str__ = _stderr_default.__str__

                if hasattr( _stderr_default, "__subclasscheck__" ):
                    __subclasscheck__ = _stderr_default.__subclasscheck__

                if hasattr( _stderr_default, "__subclasses__" ):
                    __subclasses__ = _stderr_default.__subclasses__

                if hasattr( _stderr_default, "__subclasshook__" ):
                    __subclasshook__ = _stderr_default.__subclasshook__

                if hasattr( _stderr_default, "__text_signature__" ):
                    __text_signature__ = _stderr_default.__text_signature__

                if hasattr( _stderr_default, "__weakrefoffset__" ):
                    __weakrefoffset__ = _stderr_default.__weakrefoffset__

                if hasattr( _stderr_default, "mro" ):
                    mro = _stderr_default.mro

                def __init__(self):
                    """
                        Override any super class `type( _stderr_default )` constructor, so we can 
                        instantiate any kind of `sys.stderr` replacement object, in case it was already 
                        replaced by something else like on Sublime Text with `_LogWriter()`.

                        Assures all attributes were statically replaced just above. This should happen in case
                        some new attribute is added to the python language.

                        This also ignores the only two methods which are not equal, `__init__()` and `__getattribute__()`.
                    """
                    different_methods = ("__init__", "__getattribute__")
                    attributes_to_check = set( dir( object ) + dir( type ) )

                    for attribute in attributes_to_check:

                        if attribute not in different_methods \
                                and hasattr( _stderr_default, attribute ):

                            base_class_attribute = super( _stderr_default_class_type, self ).__getattribute__( attribute )
                            target_class_attribute = _stderr_default.__getattribute__( attribute )

                            if base_class_attribute != target_class_attribute:
                                sys.stderr.write( "    The base class attribute `%s` is different from the target class:\n%s\n%s\n\n" % (
                                        attribute, base_class_attribute, target_class_attribute ) )

                def __getattribute__(self, item):

                    if item == 'write':
                        return _sys_stderr_write

                    try:
                        return _stderr_default.__getattribute__( item )

                    except AttributeError:
                        return super( _stderr_default_class_type, _stderr_default ).__getattribute__( item )

            _stderr_singleton = StdErrReplamentHidden()
            sys.stderr = _stderr_singleton

        return cls

    @classmethod
    def unlock(cls):
        """
            Detach this `stderr` writer from `sys.stderr` and allow the next call to `lock()` create
            a new writer for the stderr.
        """

        if cls.is_active:
            global _sys_stderr_write_hidden

            cls.is_active = False
            _sys_stderr_write_hidden = _stderr_default.write



class StdOutReplament(object):
    """
        How to redirect stdout and stderr to logger in Python
        https://dev59.com/fWIk5IYBdhLWcg3wRcQo

        Set a Read-Only Attribute in Python?
        https://dev59.com/s2Af5IYBdhLWcg3wlDeb
    """
    is_active = False

    @classmethod
    def lock(cls, logger):
        """
            Attach this singleton logger to the `sys.stdout` permanently.
        """
        global _stdout_singleton
        global _stdout_default
        global _stdout_default_class_type

        # On Sublime Text, `sys.__stdout__` is set to None, because they already replaced `sys.stdout`
        # by some `_LogWriter()` class, then just save the current one over there.
        if not sys.__stdout__:
            sys.__stdout__ = sys.stdout

        try:
            _stdout_default
            _stdout_default_class_type

        except NameError:
            _stdout_default = sys.stdout
            _stdout_default_class_type = type( _stdout_default )

        # Recreate the sys.stdout logger when it was reset by `unlock()`
        if not cls.is_active:
            cls.is_active = True
            _stdout_write = _stdout_default.write

            logger_call = logger.debug
            clean_formatter = logger.clean_formatter

            global _sys_stdout_write
            global _sys_stdout_write_hidden

            if sys.version_info <= (3,2):
                logger.file_handler.terminator = '\n'

            # Always recreate/override the internal write function used by `_sys_stdout_write`
            def _sys_stdout_write_hidden(*args, **kwargs):
                """
                    Suppress newline in Python logging module
                    https://dev59.com/TWw05IYBdhLWcg3wkig-
                """

                try:
                    _stdout_write( *args, **kwargs )
                    file_handler = logger.file_handler

                    formatter = file_handler.formatter
                    terminator = file_handler.terminator

                    file_handler.formatter = clean_formatter
                    file_handler.terminator = ""

                    kwargs['extra'] = {'_duplicated_from_file': True}
                    logger_call( *args, **kwargs )

                    file_handler.formatter = formatter
                    file_handler.terminator = terminator

                except Exception:
                    logger.exception( "Could not write to the file_handler: %s(%s)", file_handler, logger )
                    cls.unlock()

            # Only create one `_sys_stdout_write` function pointer ever
            try:
                _sys_stdout_write

            except NameError:

                def _sys_stdout_write(*args, **kwargs):
                    """
                        Hides the actual function pointer. This allow the external function pointer to
                        be cached while the internal written can be exchanged between the standard
                        `sys.stdout.write` and our custom wrapper around it.
                    """
                    _sys_stdout_write_hidden( *args, **kwargs )

        try:
            # Only create one singleton instance ever
            _stdout_singleton

        except NameError:

            class StdOutReplamentHidden(_stdout_default_class_type):
                """
                    Which special methods bypasses __getattribute__ in Python?
                    https://dev59.com/OGcs5IYBdhLWcg3waTRD
                """

                if hasattr( _stdout_default, "__abstractmethods__" ):
                    __abstractmethods__ = _stdout_default.__abstractmethods__

                if hasattr( _stdout_default, "__base__" ):
                    __base__ = _stdout_default.__base__

                if hasattr( _stdout_default, "__bases__" ):
                    __bases__ = _stdout_default.__bases__

                if hasattr( _stdout_default, "__basicsize__" ):
                    __basicsize__ = _stdout_default.__basicsize__

                if hasattr( _stdout_default, "__call__" ):
                    __call__ = _stdout_default.__call__

                if hasattr( _stdout_default, "__class__" ):
                    __class__ = _stdout_default.__class__

                if hasattr( _stdout_default, "__delattr__" ):
                    __delattr__ = _stdout_default.__delattr__

                if hasattr( _stdout_default, "__dict__" ):
                    __dict__ = _stdout_default.__dict__

                if hasattr( _stdout_default, "__dictoffset__" ):
                    __dictoffset__ = _stdout_default.__dictoffset__

                if hasattr( _stdout_default, "__dir__" ):
                    __dir__ = _stdout_default.__dir__

                if hasattr( _stdout_default, "__doc__" ):
                    __doc__ = _stdout_default.__doc__

                if hasattr( _stdout_default, "__eq__" ):
                    __eq__ = _stdout_default.__eq__

                if hasattr( _stdout_default, "__flags__" ):
                    __flags__ = _stdout_default.__flags__

                if hasattr( _stdout_default, "__format__" ):
                    __format__ = _stdout_default.__format__

                if hasattr( _stdout_default, "__ge__" ):
                    __ge__ = _stdout_default.__ge__

                if hasattr( _stdout_default, "__getattribute__" ):
                    __getattribute__ = _stdout_default.__getattribute__

                if hasattr( _stdout_default, "__gt__" ):
                    __gt__ = _stdout_default.__gt__

                if hasattr( _stdout_default, "__hash__" ):
                    __hash__ = _stdout_default.__hash__

                if hasattr( _stdout_default, "__init__" ):
                    __init__ = _stdout_default.__init__

                if hasattr( _stdout_default, "__init_subclass__" ):
                    __init_subclass__ = _stdout_default.__init_subclass__

                if hasattr( _stdout_default, "__instancecheck__" ):
                    __instancecheck__ = _stdout_default.__instancecheck__

                if hasattr( _stdout_default, "__itemsize__" ):
                    __itemsize__ = _stdout_default.__itemsize__

                if hasattr( _stdout_default, "__le__" ):
                    __le__ = _stdout_default.__le__

                if hasattr( _stdout_default, "__lt__" ):
                    __lt__ = _stdout_default.__lt__

                if hasattr( _stdout_default, "__module__" ):
                    __module__ = _stdout_default.__module__

                if hasattr( _stdout_default, "__mro__" ):
                    __mro__ = _stdout_default.__mro__

                if hasattr( _stdout_default, "__name__" ):
                    __name__ = _stdout_default.__name__

                if hasattr( _stdout_default, "__ne__" ):
                    __ne__ = _stdout_default.__ne__

                if hasattr( _stdout_default, "__new__" ):
                    __new__ = _stdout_default.__new__

                if hasattr( _stdout_default, "__prepare__" ):
                    __prepare__ = _stdout_default.__prepare__

                if hasattr( _stdout_default, "__qualname__" ):
                    __qualname__ = _stdout_default.__qualname__

                if hasattr( _stdout_default, "__reduce__" ):
                    __reduce__ = _stdout_default.__reduce__

                if hasattr( _stdout_default, "__reduce_ex__" ):
                    __reduce_ex__ = _stdout_default.__reduce_ex__

                if hasattr( _stdout_default, "__repr__" ):
                    __repr__ = _stdout_default.__repr__

                if hasattr( _stdout_default, "__setattr__" ):
                    __setattr__ = _stdout_default.__setattr__

                if hasattr( _stdout_default, "__sizeof__" ):
                    __sizeof__ = _stdout_default.__sizeof__

                if hasattr( _stdout_default, "__str__" ):
                    __str__ = _stdout_default.__str__

                if hasattr( _stdout_default, "__subclasscheck__" ):
                    __subclasscheck__ = _stdout_default.__subclasscheck__

                if hasattr( _stdout_default, "__subclasses__" ):
                    __subclasses__ = _stdout_default.__subclasses__

                if hasattr( _stdout_default, "__subclasshook__" ):
                    __subclasshook__ = _stdout_default.__subclasshook__

                if hasattr( _stdout_default, "__text_signature__" ):
                    __text_signature__ = _stdout_default.__text_signature__

                if hasattr( _stdout_default, "__weakrefoffset__" ):
                    __weakrefoffset__ = _stdout_default.__weakrefoffset__

                if hasattr( _stdout_default, "mro" ):
                    mro = _stdout_default.mro

                def __init__(self):
                    """
                        Override any super class `type( _stdout_default )` constructor, so we can 
                        instantiate any kind of `sys.stdout` replacement object, in case it was already 
                        replaced by something else like on Sublime Text with `_LogWriter()`.

                        Assures all attributes were statically replaced just above. This should happen in case
                        some new attribute is added to the python language.

                        This also ignores the only two methods which are not equal, `__init__()` and `__getattribute__()`.
                    """
                    different_methods = ("__init__", "__getattribute__")
                    attributes_to_check = set( dir( object ) + dir( type ) )

                    for attribute in attributes_to_check:

                        if attribute not in different_methods \
                                and hasattr( _stdout_default, attribute ):

                            base_class_attribute = super( _stdout_default_class_type, self ).__getattribute__( attribute )
                            target_class_attribute = _stdout_default.__getattribute__( attribute )

                            if base_class_attribute != target_class_attribute:
                                sys.stdout.write( "    The base class attribute `%s` is different from the target class:\n%s\n%s\n\n" % (
                                        attribute, base_class_attribute, target_class_attribute ) )

                def __getattribute__(self, item):

                    if item == 'write':
                        return _sys_stdout_write

                    try:
                        return _stdout_default.__getattribute__( item )

                    except AttributeError:
                        return super( _stdout_default_class_type, _stdout_default ).__getattribute__( item )

            _stdout_singleton = StdOutReplamentHidden()
            sys.stdout = _stdout_singleton

        return cls

    @classmethod
    def unlock(cls):
        """
            Detach this `stdout` writer from `sys.stdout` and allow the next call to `lock()` create
            a new writer for the stdout.
        """

        if cls.is_active:
            global _sys_stdout_write_hidden

            cls.is_active = False
            _sys_stdout_write_hidden = _stdout_default.write

要使用这个功能,你只需要调用StdErrReplament::lock(logger)StdOutReplament::lock(logger),并传入你想要用来发送输出文本的日志记录器。例如:
import os
import sys
import logging

current_folder = os.path.dirname( os.path.realpath( __file__ ) )
log_file_path = os.path.join( current_folder, "my_log_file.txt" )

file_handler = logging.FileHandler( log_file_path, 'a' )
file_handler.formatter = logging.Formatter( "%(asctime)s %(name)s %(levelname)s - %(message)s", "%Y-%m-%d %H:%M:%S" )

log = logging.getLogger( __name__ )
log.setLevel( "DEBUG" )
log.addHandler( file_handler )

log.file_handler = file_handler
log.clean_formatter = logging.Formatter( "", "" )

StdOutReplament.lock( log )
StdErrReplament.lock( log )

log.debug( "I am doing usual logging debug..." )
sys.stderr.write( "Tests 1...\n" )
sys.stdout.write( "Tests 2...\n" )

运行这段代码,你将在屏幕上看到:

enter image description here

关于文件内容:

enter image description here

如果您想在屏幕上查看log.debug调用的内容,您需要向记录器添加一个流处理程序。在这种情况下,它应该像这样:

import os
import sys
import logging

class ContextFilter(logging.Filter):
    """ This filter avoids duplicated information to be displayed to the StreamHandler log. """
    def filter(self, record):
        return not "_duplicated_from_file" in record.__dict__

current_folder = os.path.dirname( os.path.realpath( __file__ ) )
log_file_path = os.path.join( current_folder, "my_log_file.txt" )

stream_handler = logging.StreamHandler()
file_handler = logging.FileHandler( log_file_path, 'a' )

formatter = logging.Formatter( "%(asctime)s %(name)s %(levelname)s - %(message)s", "%Y-%m-%d %H:%M:%S" )
file_handler.formatter = formatter
stream_handler.formatter = formatter
stream_handler.addFilter( ContextFilter() )

log = logging.getLogger( __name__ )
log.setLevel( "DEBUG" )
log.addHandler( file_handler )
log.addHandler( stream_handler )

log.file_handler = file_handler
log.stream_handler = stream_handler
log.clean_formatter = logging.Formatter( "", "" )

StdOutReplament.lock( log )
StdErrReplament.lock( log )

log.debug( "I am doing usual logging debug..." )
sys.stderr.write( "Tests 1...\n" )
sys.stdout.write( "Tests 2...\n" )

当运行时,它会输出如下内容:

enter image description here

虽然仍然会将其保存到文件my_log_file.txt中:

enter image description here

当使用StdErrReplament:unlock()禁用它时,它只会恢复stderr流的标准行为,因为附加的记录器不能被分离,因为其他人可能有对其旧版本的引用。这就是为什么它是一个全局单例,永远不会死亡。因此,在使用imp或其他东西重新加载此模块的情况下,它将永远不会重新捕获当前的sys.stderr,因为它已经注入并在内部保存了它。


7
复制一个流的时候,出现了令人惊讶的意外复杂度。 - Attila Lendvai

网页内容由stack overflow 提供, 点击上面的
可以查看英文原文,
原文链接