0

I have trying to scrape using pupperter in Nextjs App router. it worked fine when i did it locally but when deployed to vercel it failed.

here is the corresponding code:

import puppeteer, { executablePath } from "puppeteer-core";

export async function scrapeData(id: string): Promise<{html: string, userpfp: string}> {
  let browser;
  try {
    browser = await puppeteer.launch({
      headless: true,
      executablePath: executablePath(),
      args: [
        "--no-sandbox",
        "--disable-setuid-sandbox",
        "--disable-blink-features=AutomationControlled",
      ],
    });
    const page = await browser.newPage();
    const url = `https://www.instagram.com/${id}/`;
    await page.goto(url);
    console.log(url);

    await page.waitForFunction(
      "window.performance.timing.loadEventEnd - window.performance.timing.navigationStart >= 500"
    );

    const html = await page.$eval(
      "head > meta[name='description']",
      (element) => element.content
    );

    const userpfp = await page.$eval(
      "head > meta[property='og:image']",
      (element) => element.content
    ) || "https://upload.wikimedia.org/wikipedia/commons/a/ac/Default_pfp.jpg?20200418092106";
    return {html, userpfp};
  } catch (error) {
    throw new Error("User not found." + error);
  } finally {
    if (browser) {
      await browser.close();
    }
  }
}

trying-to-deploy-puppeteer-on-vercel-using-nodejs doesn't works at all

Error i am getting is

Error: User not found.Error: Failed to launch the browser process!
Improve this question
6
  • 1
    Have you checked these serverless solutions? stackoverflow.com/questions/77855572/… It does look like someone in there had success on Vercel Commented Jan 26 at 10:33
  • When you say "it failed", what error do you see? Commented Jan 26 at 15:43
  • This question is similar to: Trying to deploy puppeteer on vercel using nodejs. If you believe it’s different, please edit the question, make it clear how it’s different and/or how the answers on that question are not helpful for your problem. Commented Jan 26 at 15:43
  • Many related resources our there as well: 1, 2, 3, 4, 5, 6, 7 etc Commented Jan 26 at 15:45
  • It logs Error: User not found.Error: Failed to launch the browser process! Commented Jan 26 at 16:19

1 Answer 1

Reset to default
-1

Use puppeteer instead of puppeteer-core on Vercel

Since puppeteer-core does not include the browser binary, you need to ensure that Puppeteer can access a compatible Chromium binary on Vercel. The simplest approach is to switch from puppeteer-core to puppeteer, as it automatically includes the required Chromium binary.

However, be aware that this may increase the size of your deployment, as Puppeteer includes the Chromium browser.

Install

npm install puppeteer

Update code

import puppeteer from "puppeteer";

export async function scrapeData(id: string): Promise<{html: string, userpfp: string}> {
  let browser;
  try {
    browser = await puppeteer.launch({
      headless: true,
      args: [
        "--no-sandbox",
        "--disable-setuid-sandbox",
        "--disable-blink-features=AutomationControlled",
      ],
    });
    const page = await browser.newPage();
    const url = `https://www.instagram.com/${id}/`;
    await page.goto(url);
    console.log(url);

    await page.waitForFunction(
      "window.performance.timing.loadEventEnd - window.performance.timing.navigationStart >= 500"
    );

    const html = await page.$eval(
      "head > meta[name='description']",
      (element) => element.content
    );

    const userpfp = await page.$eval(
      "head > meta[property='og:image']",
      (element) => element.content
    ) || "https://upload.wikimedia.org/wikipedia/commons/a/ac/Default_pfp.jpg?20200418092106";
    return {html, userpfp};
  } catch (error) {
    throw new Error("User not found." + error);
  } finally {
    if (browser) {
      await browser.close();
    }
  }
}
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

doesn't works either

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.