我正在使用golang编写一个小型的web应用程序,它涉及解析用户上传的文件。我希望自动检测文件是否已经gzip压缩,并相应地创建读取器/扫描器。其中一个难点是我不能将整个文件读入内存,只能在流上操作。以下是我的代码:
func scannerFromFile(reader io.Reader) (*bufio.Scanner, error) {
var scanner *bufio.Scanner
//create a bufio.Reader so we can 'peek' at the first few bytes
bReader := bufio.NewReader(reader)
testBytes, err := bReader.Peek(64) //read a few bytes without consuming
if err != nil {
return nil, err
}
//Detect if the content is gzipped
contentType := http.DetectContentType(testBytes)
//If we detect gzip, then make a gzip reader, then wrap it in a scanner
if strings.Contains(contentType, "x-gzip") {
gzipReader, err := gzip.NewReader(bReader)
if (err != nil) {
return nil, err
}
scanner = bufio.NewScanner(gzipReader)
} else {
//Not gzipped, just make a scanner based on the reader
scanner = bufio.NewScanner(bReader)
}
return scanner, nil
}
这对于普通文本来说很好用,但对于压缩数据来说,它解压出现问题,几千字节后不可避免地得到乱码。有没有更简单的方法?为什么在几千行后解压不正确?
contentType == "application/x-gzip"
而不是strings.Contains。 - Charlie Tumahai