Skip to content
This repository has been archived by the owner on Nov 24, 2018. It is now read-only.

--proxy-server setting on local Chromeless does not work for authenticated proxies #457

Open
wflanagan opened this issue Jul 14, 2018 · 1 comment

Comments

@wflanagan
Copy link

wflanagan commented Jul 14, 2018

When I attempt to use a proxy server on the local configuration, and that proxy has authentication, Chromeless returns an empty body without an error.

I'm using the most recent Chromeless as of 14 July 2018. Here is my script.

I've obviously removed my private proxies usernames, passwords, and host names. If you message me 1:1 I'm happy to give them to you for testing purposes.

FYI, my proxies are provided by a commercial service where I pay for usage. I'm highly confident they are configured correctly. They have run millions of URLs without problems and continue to do so in our production environment.

const chromeLauncher = require('chrome-launcher');
const Chromeless = require('chromeless').Chromeless;

if (process.env.NODE_ENV !== 'production') {
  require('dotenv').config()
}
const argv = require('minimist')(process.argv.slice(2));
const url = argv['u'];

async function run(url) {
  const chrome = await chromeLauncher.launch({
    port: 9222,
    chromeFlags: [
      '--disable-gpu',
      '--headless',
      //'--proxy-server=http://username:password@domain_name.com:80' // doesn't work (with DNS resolution to get IP)
      //'--proxy-server=http://username:password@1.2.3.4:80' // doesn't work (checking DNS resolution as problem)
      //'--proxy-server=http://201.49.66.97:92' // works (get one from https://www.proxynova.com/proxy-server-list/)
    ]
  });

  console.log('Chrome debuggable on port: ' + chrome.port);
  const chromeless = new Chromeless({
    launchChrome: false,
  })

  const html = await chromeless
    .goto(url)
    .wait(15000) // set arbitrarily long to ensure that this isn't a thing that its not finished rendering
    .html()

  const response = { html: html, url: url }
  console.log(JSON.stringify(response))
  await chromeless.end()
}

run(url).catch(console.error.bind(console))

Ideally, I can do this using Lambda as well. But, given its lack of "on demand changing proxy servers", I have opted to run a local Chrome for now.

Chrome Version: Version 67.0.3396.99 (Official Build) (64-bit)
Node Version: v10.5.0
Development Environment: MacOS High Sierra 10.13.5 (MBRt 16gb RAM)

@wflanagan
Copy link
Author

As an update to this, this isn't Chromeless--it's Chrome itself that's the problem.

See https://superuser.com/questions/902620/google-chrome-proxy-settings-with-username-and-password for more information.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant