在Node JS中响应缓冲区数据被截断

Question

我正在尝试使用Node JS内置的http和https模块的request函数（我知道有第三方包装器，但我想自己解决问题）。 我遇到的问题是响应中的缓冲数据在末尾部分被截断。 在使用cURL测试时不会出现此问题。

这是我一直在使用的代码：

const { request: httpRequest } = require("http");
const { request: httpsRequest } = require("https");
const parseURLData = (url, init) => {
  const { hostname, protocol, port: somePort, pathname, search } = new URL(url);
  const port = +somePort || (protocol === "https:" ? 443 : 80);
  const options = { hostname, port, path: pathname + search, ...init };
  return [protocol, options];
};
const makeRequest = (url, init = { method: "GET" }) => {
  const [protocol, options] = argumentsToOptions(url, init);
  const request = protocol === "http:" ? httpRequest : httpsRequest;
  return new Promise((resolve, reject) =>
    request(options, (res) => {
      res.on("error", reject);
      resolve(res);
    })
      .on("error", reject)
      .end()
  );
};
// not using `async/await` while testing
makeRequest("https://jsonplaceholder.typicode.com/users/1/")
  .then((res) => 
    new Promise((resolve) =>
      res.on("data", (buffer) => {
        resolve(buffer.toString("utf8")); // part of data is cut off
        // resolve(JSON.parse(buffer.toString()));
      })
    )
  )
  .then(console.log)
  .catch(console.error);

以下是预期输出（来自 cURL ）：

{
  "id": 1,
  "name": "Leanne Graham",
  "username": "Bret",
  "email": "[e-mail]",
  "address": {
    "street": "Kulas Light",
    "suite": "Apt. 556",
    "city": "Gwenborough",
    "zipcode": "92998-3874",
    "geo": {
      "lat": "-37.3159",
      "lng": "81.1496"
    }
  },
  "phone": "1-770-736-8031 x56442",
  "website": "hildegard.org",
  "company": {
    "name": "Romaguera-Crona",
    "catchPhrase": "Multi-layered client-server neural-net",
    "bs": "harness real-time e-markets"
  }
}

以下是实际输出，不知何故每次运行代码结果略有不同：

{
  "id": 1,
  "name": "Leanne Graham",
  "username": "Bret",
  "email": "[e-mail]",
  "address": {
    "street": "Kulas Light",
    "suite": "Apt. 556",
    "city": "Gwenborough",
    "zipcode": "92998-3874",
    "geo": {
      "lat": "-37.3159",
      "lng": "81.1496"
    }
  },
  "p

这个问题的正确解决方案是什么？我应该怎样获取请求中的所有数据？

- Jacob

Answer 1

只有在end事件触发时才应该处理缓冲区，否则可能会处理不完整的缓冲区。

makeRequest("https://jsonplaceholder.typicode.com/users/1/")
  .then((res) => 
    new Promise((resolve) => {
        let totalBuffer = "";

        res.on("data", (buffer) => {
            totalBuffer += buffer.toString("utf8");    
        });

        res.on("end", () => resolve(totalBuffer));
    })
  )
  .then(console.log)
  .catch(console.error);

当文件大小超过1MB时，响应几乎总是被截断为多个部分，因此需要使用“end”事件，该事件指示流已处理所有可用数据。

https://nodejs.org/api/http.html 查找“JSON获取示例”

- Gamedroit