我扩展了其他问题,并在LINQPad下运行了测试代码,试图更好地展示CRC32(循环冗余校验)问题是否确实存在。(由于代码只是基于原始代码的轻微修改,因此测试设置/方法可能存在缺陷或存在另一个奇怪的问题/PEBCAK。)
结果很奇怪,因为损坏的数据并不总是会引发(任何!)异常。请注意,仅有时候 CRC32 检查似乎确实在“工作”。导致索引超出范围/坏头/坏尾的损坏字节可以忽略,因为我们可以假设这些正在杀死解压缩之前的CRC32检查(即使IndexOutOfRangeException可能应该被InvalidDataException包装),所以,
CRC32检查为什么比它应该的可靠性要低得多?(为什么下面会有“无效数据(无异常)”的情况?)
由于GZip页脚包含 未压缩数据的CRC32和长度,因此错误检测率应该“显着更高”——也就是说,我不会期望下面出现单个失败的情况,更不用说许多未检测到的损坏流了。(当然,尽早检测到损坏的流很好:但在某些情况下,最终的安全检查和校验和似乎被彻底忽略。)
格式为CorruptByteIndex+FailedDetections: Message
:
0+0: System.IO.InvalidDataException:The magic number in GZip header is not correct. Make sure you are passing in a GZip stream. 1+0: System.IO.InvalidDataException:The magic number in GZip header is not correct. Make sure you are passing in a GZip stream. 2+0: System.IO.InvalidDataException:The compression mode specified in GZip header is unknown. 3+0: Good data (No Exception) 4+0: Good data (No Exception) 5+0: Good data (No Exception) 6+0: Good data (No Exception) 7+0: Good data (No Exception) 8+0: Good data (No Exception) 9+0: Good data (No Exception) 10+0: System.IO.InvalidDataException:Unknown block type. Stream might be corrupted. 11+1: Invalid data (No Exception) 12+1: System.IO.InvalidDataException:Found invalid data while decoding. 13+1: System.IO.InvalidDataException:Found invalid data while decoding. 14+1: System.IO.InvalidDataException:Found invalid data while decoding. 15+1: System.IO.InvalidDataException:Found invalid data while decoding. 16+1: System.IO.InvalidDataException:Found invalid data while decoding. 17+2: Invalid data (No Exception) 18+2: System.IO.InvalidDataException:Found invalid data while decoding. 19+2: System.IndexOutOfRangeException:Index was outside the bounds of the array. 20+2: System.IndexOutOfRangeException:Index was outside the bounds of the array. 21+3: Invalid data (No Exception) 22+3: System.IndexOutOfRangeException:Index was outside the bounds of the array. 23+3: System.IndexOutOfRangeException:Index was outside the bounds of the array. 24+4: Invalid data (No Exception) 25+4: System.IndexOutOfRangeException:Index was outside the bounds of the array. 26+4: System.IndexOutOfRangeException:Index was outside the bounds of the array. 27+4: System.IndexOutOfRangeException:Index was outside the bounds of the array. 28+4: System.IndexOutOfRangeException:Index was outside the bounds of the array. 29+5: Invalid data (No Exception) 30+5: System.IndexOutOfRangeException:Index was outside the bounds of the array. 31+6: Invalid data (No Exception) 32+7: Invalid data (No Exception) 33+7: System.IndexOutOfRangeException:Index was outside the bounds of the array. 34+7: System.IndexOutOfRangeException:Index was outside the bounds of the array. 35+7: System.IndexOutOfRangeException:Index was outside the bounds of the array. 36+8: Invalid data (No Exception) 37+8: System.IndexOutOfRangeException:Index was outside the bounds of the array. 38+8: System.IndexOutOfRangeException:Index was outside the bounds of the array. 39+9: Invalid data (No Exception) 40+9: System.IndexOutOfRangeException:Index was outside the bounds of the array. 41+9: System.IndexOutOfRangeException:Index was outside the bounds of the array. 42+10: Invalid data (No Exception) 43+10: System.IO.InvalidDataException:Found invalid data while decoding. 44+10: System.IndexOutOfRangeException:Index was outside the bounds of the array. 45+10: System.IO.InvalidDataException:Found invalid data while decoding. 46+11: Invalid data (No Exception) 47+11: System.IndexOutOfRangeException:Index was outside the bounds of the array. 48+11: System.IndexOutOfRangeException:Index was outside the bounds of the array. 49+11: System.IndexOutOfRangeException:Index was outside the bounds of the array. 50+12: Invalid data (No Exception) 51+12: System.IndexOutOfRangeException:Index was outside the bounds of the array. 52+12: System.IndexOutOfRangeException:Index was outside the bounds of the array. 53+13: Invalid data (No Exception) 54+13: System.IndexOutOfRangeException:Index was outside the bounds of the array. 55+14: Invalid data (No Exception) 56+14: System.IndexOutOfRangeException:Index was outside the bounds of the array. 57+15: Invalid data (No Exception) 58+15: System.IndexOutOfRangeException:Index was outside the bounds of the array. 59+15: System.IndexOutOfRangeException:Index was outside the bounds of the array. 60+16: Invalid data (No Exception) 61+17: Invalid data (No Exception) 62+18: Invalid data (No Exception) 63+19: Invalid data (No Exception) 64+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 65+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 66+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 67+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 68+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 69+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 70+19: System.IO.InvalidDataException:Found invalid data while decoding. 71+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 72+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 73+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 74+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 75+19: System.IO.InvalidDataException:Found invalid data while decoding. 76+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 77+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 78+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 79+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 80+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 81+19: System.IO.InvalidDataException:Found invalid data while decoding. 82+19: System.IndexOutOfRangeException:Index was outside the bounds of the array. 83+20: Invalid data (No Exception) 84+21: Invalid data (No Exception) 85+22: Invalid data (No Exception) 86+22: System.IndexOutOfRangeException:Index was outside the bounds of the array. 87+23: Invalid data (No Exception) 88+24: Invalid data (No Exception) 89+25: Invalid data (No Exception) 90+25: System.IndexOutOfRangeException:Index was outside the bounds of the array. 91+26: Invalid data (No Exception) 92+26: System.IO.InvalidDataException:Found invalid data while decoding. 93+26: System.IndexOutOfRangeException:Index was outside the bounds of the array. 94+27: Invalid data (No Exception) 95+27: System.IndexOutOfRangeException:Index was outside the bounds of the array. 96+27: System.IndexOutOfRangeException:Index was outside the bounds of the array. 97+28: Invalid data (No Exception) 98+28: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 99+28: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 100+29: Invalid data (No Exception) 101+30: Invalid data (No Exception) 102+31: Invalid data (No Exception) 103+32: Invalid data (No Exception) 104+32: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 105+33: Invalid data (No Exception) 106+34: Invalid data (No Exception) 107+35: Invalid data (No Exception) 108+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 109+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 110+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 111+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 112+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 113+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 114+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 115+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 116+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 117+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 118+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 119+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 120+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 121+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 122+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 123+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 124+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 125+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 126+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 127+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 128+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 129+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 130+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 131+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 132+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 133+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 134+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 135+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 136+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 137+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 138+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 139+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 140+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 141+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 142+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 143+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 144+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 145+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 146+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 147+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 148+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 149+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 150+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 151+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 152+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 153+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 154+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 155+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 156+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 157+35: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 158+36: Invalid data (No Exception) 159+36: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 160+36: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 161+37: Invalid data (No Exception) 162+38: Invalid data (No Exception) 163+39: Invalid data (No Exception) 164+40: Invalid data (No Exception) 165+41: Invalid data (No Exception) 166+41: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 167+41: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 168+41: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 169+41: System.IO.InvalidDataException:The CRC in GZip footer does not match the CRC calculated from the decompressed data. 170+41: System.IO.InvalidDataException:The stream size in GZip footer does not match the real stream size. 171+41: System.IO.InvalidDataException:The stream size in GZip footer does not match the real stream size. 172+41: System.IO.InvalidDataException:The stream size in GZip footer does not match the real stream size. 173+41: System.IO.InvalidDataException:The stream size in GZip footer does not match the real stream size.以下是可在LINQPad中复制并运行的测试代码(适用于.NET 3.5和4,使用“作为C#语句”模式):
string sample = "This is a compression test of microsoft .net gzip compression method and decompression methods";
var encoding = new ASCIIEncoding();
var data = encoding.GetBytes(sample);
string sampleOut = null;
byte[] cmpData;
// Compress
using (var cmpStream = new MemoryStream())
{
using (var hgs = new System.IO.Compression.GZipStream(cmpStream, System.IO.Compression.CompressionMode.Compress))
{
hgs.Write(data, 0, data.Length);
}
cmpData = cmpStream.ToArray();
}
int corruptBytesNotDetected = 0;
// corrupt data byte by byte
for (var byteToCorrupt = 0; byteToCorrupt < cmpData.Length; byteToCorrupt++)
{
var corruptData = new List<byte>(cmpData).ToArray();
// corrupt the data
corruptData[byteToCorrupt]++;
using (var decomStream = new MemoryStream(corruptData))
{
using (var hgs = new System.IO.Compression.GZipStream(decomStream, System.IO.Compression.CompressionMode.Decompress))
{
using (var reader = new StreamReader(hgs))
{
string message;
try
{
sampleOut = reader.ReadToEnd();
// if we get here, the corrupt data was not detected by GZipStream
// ... okay so long as the correct data is extracted
if (!sample.SequenceEqual(sampleOut)) {
corruptBytesNotDetected++;
message = "Invalid data (No Exception)";
} else {
message = "Good data (No Exception)";
}
}
catch(Exception ex)
{
message = (ex.GetType() + ":" + ex.Message);
}
string.Format("{0}+{1}: {2}",
byteToCorrupt, corruptBytesNotDetected, message).Dump();
}
}
}
}
以下是在.NET 3.5中的压缩数据(GZipStream在“压缩”小负载方面非常糟糕,但这是一个“不修复”的问题,因为该流仍然在技术上有效):
1F 8B 08 00 00 00 00 00 04 00 ED BD 07 60 1C 49 96 25 26 2F 6D CA 7B 7F 4A F5 4A D7 E0 74 A1 08 80 60 13 24 D8 90 40 10 EC C1 88 CD E6 92 EC 1D 69 47 23 29 AB 2A 81 CA 65 56 65 5D 66 16 40 CC ED 9D BC F7 DE 7B EF BD F7 DE 7B EF BD F7 BA 3B 9D 4E 27 F7 DF FF 3F 5C 66 64 01 6C F6 CE 4A DA C9 9E 21 80 AA C8 1F 3F 7E 7C 1F 3F 22 DE CC 8B 26 A5 FF 65 E9 B4 5A AC EA BC 69 8A 6A 99 B6 79 D3 A6 D5 79 BA 28 A6 75 D5 54 E7 6D 3A 5E E6 6D 7A F1 83 62 15 B4 5B E4 ED BC 9A A5 D9 72 96 CE F2 FE 17 CD FF 03 5C 51 5E 27 5E 00 00 00
(仅为了好玩,在.NET 4中生成了一个稍微更大/不同的压缩流。)
1F 8B 08 00 00 00 00 00 04 00 EC BD 07 60 1C 49 96 25 26 2F 6D CA 7B 7F 4A F5 4A D7 E0 74 A1 08 80 60 13 24 D8 90 40 10 EC C1 88 CD E6 92 EC 1D 69 47 23 29 AB 2A 81 CA 65 56 65 5D 66 16 40 CC ED 9D BC F7 DE 7B EF BD F7 DE 7B EF BD F7 BA 3B 9D 4E 27 F7 DF FF 3F 5C 66 64 01 6C F6 CE 4A DA C9 9E 21 80 AA C8 1F 3F 7E 7C 1F 3F 22 DE CC 8B 26 A5 FF 65 E9 B4 5A AC EA BC 69 8A 6A 99 B6 79 D3 A6 D5 79 BA 28 A6 75 D5 54 E7 6D 3A 5E E6 6D 7A F1 83 62 15 B4 5B E4 ED BC 9A A5 D9 72 96 CE F2 FE 17 CD FF 13 00 00 FF FF 5C 51 5E 27 5E 00 00 00
附加说明:
在本例中,测试可能会存在一些微小的缺陷。当GZipStream“未能检测到损坏”(没有异常)时,从StreamReader读取的数据为“”(空字符串):在这种情况下,为什么ReadToEnd()
不会引发异常(IOException或其他异常)?
因此,这里是否不是GZipStream而是StreamReader“古怪”的问题,还是GZipStream仍然存在问题(未抛出异常)?是否有一种正确的方法来可靠地处理这种用例?(考虑当前位置的输入流确实为空的情况。)