如何使用Python对源代码进行更改？

Question

我正在使用Clang编译器的Python绑定(cindex)重构C++代码。使用它，我分析AST并准备更改。最终，我得到了类似以下操作列表：

DELETE line 10
INSERT line 5 column 32: <<  "tofu"
REPLACE from line 31 colum 6 to line 33 column 82 with: std::cout << "Thanks SO"
...

我的问题是如何将这些变成实际的文件更改。

用Python直接做似乎很麻烦：补丁需要按正确顺序应用并检查一致性。这看起来相当困难和容易出错。

我也找不到一个好的库来帮助（clang确实有一个叫做Rewriter的东西，但它没有用Python封装。如果可能的话，我真的想避免使用C++进行重构）。

也许一个想法是生成补丁并用git应用它们，也许？但即使这样也似乎有点繁琐。

有什么想法吗？

- static_rtti

---

你展示的是一系列补丁；为什么不能按顺序应用它们呢？[也许你正在对AST进行多个重叠的补丁操作？] 每次进行更改时修改AST，然后输出修订后的AST是否更有用？这样就不会出现无序的补丁问题了。 - Ira Baxter

你必须小心顺序，因为更改可能会使行号无效。我同意修改AST是最好的选择，不幸的是cindex不支持它。 - static_rtti

如果你手头的工具无法胜任工作，你可以选择另一个工具。请查看我的个人简介，了解一个名为DMS的工具，它可以解析C++，构建和修改C++ AST（我们这样做是为了避免在组合更改时出现愚蠢的补丁问题），然后重新生成有效的源代码。虽然它不是用Python编写的，也不是用C++编写的；DMS是一系列DSL的数组，它们相互协作，让您指定转换。 - Ira Baxter

Answer 1

所以我自己开发了一个。代码几乎肯定有漏洞，不太美观，但我希望发布它，希望它能帮助某人，直到找到更好的解决方案。

class PatchRecord(object):
    """ Record patches, validate them, order them, and apply them """

    def __init__(self):
        # output of readlines for each patched file
        self.lines = {}
        # list of patches for each patched file
        self.patches = {}

    class Patch(object):
        """ Abstract base class for editing operations """

        def __init__(self, filename, start, end):
            self.filename = filename
            self.start = start
            self.end = end

        def __repr__(self):
            return "{op}: {filename} {start}/{end} {what}".format(
                op=self.__class__.__name__.upper(),
                filename=self.filename,
                start=format_place(self.start),
                end=format_place(self.end),
                what=getattr(self, "what", ""))

        def apply(self, lines):
            print "Warning: applying no-op patch"

    class Delete(Patch):

        def __init__(self, filename, extent):
            super(PatchRecord.Delete, self).__init__(
                filename, extent.start, extent.end)
            print "DELETE: {file} {extent}".format(file=self.filename,
                                                   extent=format_extent(extent))

        def apply(self, lines):
            lines[self.start.line - 1:self.end.line] = [
                lines[self.start.line - 1][:self.start.column - 1] +
                lines[self.end.line - 1][self.end.column:]]

    class Insert(Patch):

        def __init__(self, filename, start, what):
            super(PatchRecord.Insert, self).__init__(filename, start, start)
            self.what = what
            print "INSERT {where} {what}".format(what=what, where=format_place(self.start))

        def apply(self, lines):
            line = lines[self.start.line - 1]
            lines[self.start.line - 1] = "%s%s%s" % (
                line[:self.start.column],
                self.what,
                line[self.start.column:])

    class Replace(Patch):

        def __init__(self, filename, extent, what):
            super(PatchRecord.Replace, self).__init__(
                filename, extent.start, extent.end)
            self.what = what
            print "REPLACE: {where} {what}".format(what=what,
                                                   where=format_extent(extent))

        def apply(self, lines):
            lines[self.start.line - 1:self.end.line] = [
                lines[self.start.line - 1][:self.start.column - 1] +
                self.what +
                lines[self.end.line - 1][self.end.column - 1:]]

    # Convenience functions for creating patches
    def delete(self, filename, extent):
        self.patches[filename] = self.patches.get(
            filename, []) + [self.Delete(filename, extent)]

    def insert(self, filename, where, what):
        self.patches[filename] = self.patches.get(
            filename, []) + [self.Insert(filename, where, what)]

    def replace(self, filename, extent, what):
        self.patches[filename] = self.patches.get(
            filename, []) + [self.Replace(filename, extent, what)]

    def _pos_to_tuple(self, position):
        """ Convert a source location to a tuple for use as a sorting key """
        return (position.line, position.column)

    def sort(self, filename):
        """ Sort patches by extent start """
        self.patches[filename].sort(key=lambda p: self._pos_to_tuple(p.start))

    def validate(self, filename):
        """Try to insure patches are consistent"""
        print "Checking patches for %s" % filename
        self.sort(filename)
        previous = self.patches[filename][0]
        for p in self.patches[filename][1:]:
            assert(self._pos_to_tuple(p.start) >
                   self._pos_to_tuple(previous.start))

    def _apply(self, filename):
        self.sort(filename)
        lines = self._getlines(filename)
        for p in reversed(self.patches[filename]):
            print p
            p.apply(lines)

    def _getlines(self, filename):
        """ Get source file lines for editing """
        if not filename in self.lines:
            with open(filename) as f:
                self.lines[filename] = f.readlines()
        return self.lines[filename]

    def apply(self):
        for filename in self.patches:
            self.validate(filename)
            self._apply(filename)
            # with open(filename+".patched","w") as output:
            with open(filename, "w") as output:
                output.write("".join(self._getlines(filename)))

只需创建一个PatchRecord对象，使用create、replace和delete方法添加更改，当您准备好时使用apply应用它们。

- static_rtti