木偶浏览器用户代理列表

问题描述

我想从我的默认位置路径中加载随机的用户代理列表，例如：'agents.json'，而不是直接添加仅一个用户代理。

agents.json

["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/85.0.4183.83 Safari/537.36","Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/78.0.3904.108 Safari/537.36","Mozilla/5.0 (iPad; cpu OS 13_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML,like Gecko) CriOS/79.0.3945.73 Mobile/15E148 Safari/604.1"]

我的代码：

const puppeteer = require('../core/puppeteer');
const referers = require('../core/referers.json');
const referers = require('../core/agents.json');
const viewVideosinBatch = async ({ targetUrls,durationInSeconds,port }) => {
  let browser;
  try {
    browser = await puppeteer.getbrowserInstance(port);
    const randomreferer = referers[Math.floor(Math.random() * referers.length)];
    const randomAgents = agents[Math.floor(Math.random() * agents.length)];
    const page = await browser.newPage();
    browser = await page.setUserAgent({ agents: randomAgents });
    page.setDefaultTimeout(PAGE_DEFAULT_TIMEOUT * 1000);
    page.on('error',handlePageCrash(page));
    page.on('pageerror',handlePageCrash(page));
    page.setExtraHTTPHeaders({ referer: randomreferer });

    await page.setViewport({
      width: 640,height: 480,deviceScaleFactor: 1,});
    const ipAddr = await getCurrentIP(page);
    const targetUrlsForAction = _take(_shuffle(targetUrls),VIEW_ACTION_COUNT);
    await watchVideosinSequence(page,ipAddr,targetUrlsForAction,durationInSeconds);
    await page.close();
  } catch (error) {
    logger.warn('Entire view action in a batch Failed.');
    logger.debug(error);
  } finally {
    await browser.close();
  }
};

我得到的错误：

(node:1) UnhandledPromiseRejectionWarning: Error: Protocol error (Network.setUserAgentOverride): Target closed.
view_4  |     at /app/node_modules/puppeteer/lib/Connection.js:183:56
view_4  |     at new Promise (<anonymous>)
view_4  |     at CDPSession.send (/app/node_modules/puppeteer/lib/Connection.js:182:12)
view_4  |     at next (/app/node_modules/puppeteer-extra-plugin-stealth/evasions/sourceurl/index.js:30:43)
view_4  |     at CDPSession.send (/app/node_modules/puppeteer-extra-plugin-stealth/evasions/sourceurl/index.js:46:18)
view_4  |     at Plugin.onPageCreated (/app/node_modules/puppeteer-extra-plugin-stealth/evasions/user-agent-override/index.js:75:18)
view_4  |     at runMicrotasks (<anonymous>)
view_4  |     at processticksAndRejections (internal/process/task_queues.js:97:5)
view_4  |     at async Plugin._onTargetCreated (/app/node_modules/puppeteer-extra-plugin/dist/index.cjs.js:491:17)
view_4  | (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block,or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection,use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 79)

解决方法

设置UA时，您似乎重新声明了浏览器，但您不应该这样做。同样，您也不需要将其设置为对象：page.setUserAgent接受字符串：page.setUserAgent(userAgent)。

await page.setUserAgent(randomAgent);就可以了。

const browser = await puppeteer.getBrowserInstance(port);
const randomReferer = referers[Math.floor(Math.random() * referers.length)];
const randomAgent = agents[Math.floor(Math.random() * agents.length)]; // made it singular instead of plural
const page = await browser.newPage();
await page.setUserAgent(randomAgent); // like this
page.setDefaultTimeout(PAGE_DEFAULT_TIMEOUT * 1000);
page.on('error',handlePageCrash(page));
page.on('pageerror',handlePageCrash(page));
page.setExtraHTTPHeaders({ referer: randomReferer });

注意：我认为这只是您发帖中的一个错误，但是您还有两个名为referers的const，这个常量应该是agents：

const referers = require('../core/agents.json');