我尝试将目标文件夹中的所有源文件转换为UTF-8（无BOM）编码。
我使用以下PowerShell脚本：
$MyPath = "D:\my projects\etc\"
Get-ChildItem $MyPath\* -Include *.h, *.cpp, *.c | Foreach-Object {
$content = Get-Content $_.FullName
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($_.FullName, $content, $Utf8NoBomEncoding)
}
cmd /c pause | out-null
如果文件不是UTF-8编码，那么它可以正常工作。但是，如果某个文件已经是UTF-8无BOM格式，所有的国际字符就会被转换成未知符号（例如，如果我再次运行脚本）。如何修改脚本以解决这个问题？
[System.Text.Encoding] ::Default中。使用-Encoding Utf8读取每个文件，并测试结果字符串是否包含Unicode替换字符(U+FFFD) 。如果包含，则说明该文件不是UTF-8格式，因为这个特殊字符用来表示遇到了在UTF-8中无效的字节序列。
如果文件不是有效的UTF-8格式，则再次读取文件，不要使用-Encoding参数，这会使Windows PowerShell将文件解释为Windows-1251编码，因为这是你系统区域设置所暗示的编码（代码页）。
$MyPath = "D:\my projects\etc"
Get-ChildItem $MyPath\* -Include *.h, *.cpp, *.c | Foreach-Object {
# Note:
# * the use of -Encoding Utf8 to first try to read the file as UTF-8.
# * the use of -Raw to read the entire file as a *single string*.
$content = Get-Content -Raw -Encoding Utf8 $_.FullName
# If the replacement char. is found in the content, the implication
# is that the file is NOT UTF-8, so read it again *without -Encoding*,
# which interprets the files as "ANSI" encoded (Windows-1251, in your case).
if ($content.Contains([char] 0xfffd)) {
$content = Get-Content -Raw $_.FullName
}
# Note the use of WriteAllText() in lieu of WriteAllLines()
# and that no explicit encoding object is passed, given that
# .NET *defaults* to BOM-less UTF-8.
# CAVEAT: There's a slight risk of data loss if writing back to the input
# file is interrupted.
[System.IO.File]::WriteAllText($_.FullName, $content)
}
使用 [IO.File]::ReadAllText() 方法并搭配一个UTF-8编码对象可以更快地完成操作（PSv5+语法）。当遇到无效的UTF-8字节时，该编码对象会抛出异常：
$utf8EncodingThatThrows = [Text.UTF8Encoding]::new($false, $true)
# ...
try {
$content = [IO.File]::ReadAllText($_.FullName, $utf8EncodingThatThrows)
} catch [Text.DecoderFallbackException] {
$content = [IO.File]::ReadAllText($_.FullName, [Text.Encoding]::Default)
}
# ...
将上述解决方案适应于 PowerShell Core / .NET Core：
PowerShell Core 默认使用（无 BOM 的）UTF-8 编码，因此仅省略 -Encoding 参数不能用于读取 ANSI 编码的文件。
同样，在 .NET Core 中，[System.Text.Encoding]::Default 总是报告 UTF-8 编码。
因此，您必须手动确定活动系统区域设置的 ANSI 代码页并获取相应的编码对象：
$ansiEncoding = [Text.Encoding]::GetEncoding(
[int] (Get-ItemPropertyValue HKLM:\SYSTEM\CurrentControlSet\Control\Nls\CodePage ACP)
)
Get-Content -Encoding(Get-Content -Raw -Encoding $ansiEncoding $_.FullName)或.NET方法([IO.File]::ReadAllText($_.FullName, $ansiEncoding))。
因此，如果您的一些UTF-8编码的文件（已经）没有BOM，则必须显式地指示Get-Content将它们视为UTF-8，使用-Encoding Utf8 - 否则，如果它们包含7位ASCII范围之外的字符，它们将被错误解释：
$MyPath = "D:\my projects\etc"
Get-ChildItem $MyPath\* -Include *.h, *.cpp, *.c | Foreach-Object {
# Note:
# * the use of -Encoding Utf8 to ensure the correct interpretation of the input file
# * the use of -Raw to read the entire file as a *single string*.
$content = Get-Content -Raw -Encoding Utf8 $_.FullName
# Note the use of WriteAllText() in lieu of WriteAllLines()
# and that no explicit encoding object is passed, given that
# .NET *defaults* to BOM-less UTF-8.
# CAVEAT: There's a slight risk of data loss if writing back to the input
# file is interrupted.
[System.IO.File]::WriteAllText($_.FullName, $content)
}
$hasUtf8Bom = "$(Get-Content -Encoding Byte -First 3 $_.FullName)" -eq '239 187 191'（Windows PowerShell）或$hasUtf8Bom = "$(Get-Content -AsByteStream -First 3 $_.FullName)" -eq '239 187 191'（PowerShell Core）。# PowerShell Core only.
$MyPath = "D:\my projects\etc"
Get-ChildItem $MyPath\* -Include *.h, *.cpp, *.c | Foreach-Object {
# * Read the file at hand (UTF8 files both with and without BOM are
# read correctly).
# * Simply rewrite it with the *default* encoding, which in
# PowerShell Core is BOM-less UTF-8.
# Note the (...) around the Get-Content call, which is necessary in order
# to write back to the *same* file in the same pipeline.
# CAVEAT: There's a slight risk of data loss if writing back to the input
# file is interrupted.
(Get-Content -Raw $_.FullName) | Set-Content -NoNewline $_.FullName
}
上述解决方案可以工作，但是Get-Content和Set-Content相对较慢，因此使用.NET类型来读取和重写文件将会更加高效。
与上述一样，在以下解决方案中不需要显式地指定任何编码方式（即使在Windows PowerShell中），因为.NET自身从一开始就默认为无BOM UTF-8（同时仍然识别UTF-8 BOM 如果存在）：
$MyPath = "D:\my projects\etc"
Get-ChildItem $MyPath\* -Include *.h, *.cpp, *.c | Foreach-Object {
# CAVEAT: There's a slight risk of data loss if writing back to the input
# file is interrupted.
[System.IO.File]::WriteAllText(
$_.FullName,
[System.IO.File]::ReadAllText($_.FullName)
)
}
正确检查 BOM 是否存在，例如使用以下模板（在关于 BOM 的注释处应用操作）：
$ps1scripts = Get-ChildItem .\*.ps1 -Recurse # change to match your circumstances
foreach ( $ps1script in $ps1scripts ) {
$first3 = $ps1script | Get-Content -Encoding byte -TotalCount 3
$first3Hex = '{0:X2}{1:X2}{2:X2}' -f $first3[0],$first3[1],$first3[2]
$first2Hex = '{0:x2}{1:x2}' -f $first3[0],$first3[1]
if ( $first3Hex -eq 'EFBBBF' ) {
# UTF-8 BOM
} elseif ( $first2Hex -eq 'fffe' ) {
# UCS-2LE BOM
} elseif ( $first2Hex -eq 'feff' ) {
# UCS-2BE BOM
} else {
# unknown (no BOM)
}
}
$MyPath = "D:\my projects\etc\"
$ps1scripts = Get-ChildItem $MyPath\* -Include *.h, *.cpp, *.c
Get-Content就无法将文件内容识别为UTF-8格式，因此会将文件读取为ANSI格式，从而错误地解释特殊字符。您首先要通过删除BOM来解决什么问题？ - Ansgar Wiechers