Puppeteer Being Redirected When Browser Is Not

Attempting to test page https://publicindex.sccourts.org/anderson/publicindex/ When navigating with standard browser to the page, the navigation ends at the requested page (https:/

Solution 1:

I don't have a full resolution but I know where the redirection is happening.

I tested your script locally with below:

const puppeteer = require('puppeteer');
constPuppeteerHar = require('puppeteer-har');

functionrun () {
    let url = 'https://publicindex.sccourts.org/anderson/publicindex/';
    puppeteer.launch({headless: false, devtools: true }).then(async browser => {
        const page = await browser.newPage();
        await page.setRequestInterception(true);
        page.on('request', request => {
            console.log('GOT NEW REQUEST', request.url());
            request.continue();
        });

        page.on('response', response => {
            console.log('GOT NEW RESPONSE', response.status(), response.headers());
        });
        await page.setJavaScriptEnabled(true);
        await page.setViewport({width: 1920, height: 1280});
        await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36');
        const har = newPuppeteerHar(page);
        await har.start({path: 'results.har'});
        const response = await page.goto(url);
        await page.waitForNavigation();
        await har.stop();
        let bodyHTML = await page.content();
    });
};
run();

I edited three parts:

• Removed headless mode and open the devtools automatically
• Intercept all network requests (that I audited)
• Hoisted require import because it hurts my eyes. I always see them call without nesting

Turns out the page https://publicindex.sccourts.org/anderson/publicindex/ make a request to https://publicindex.sccourts.org/

However this request returns a 302 Redirect to https://www.sccourts.org/caseSearch/ location, so the browser acts accordingly

I would try to investigate this weird request if it is legit or not and why it redirects on chrome puppeteer

This post might help, there could be something related on chromium being seen as insecure

I also tried to pass args: ['--disable-web-security', '--allow-running-insecure-content'] to launch() object parameter, but without results

Please let us know how it goes! Har has been fun to discover!