Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support connecting to browser using connect() instead of launch() #892

Open
Kilowhisky opened this issue Jan 15, 2025 · 1 comment
Open

Comments

@Kilowhisky
Copy link

Is your feature request related to a problem? Please describe.
When trying to automate the process of scraping i've found that trying to run the local browser inside a lambda that handles multiple sessions causes problems with the browser when its built into the app.

Describe the solution you'd like
Playwright lets you setup a remote server and connect to it using web sockets.

see here for example:
https://github.com/pixelfactoryio/playwright-server

I would like to see a configuration that lets me specify that it should connect to one of these servers instead of trying to launch the browser locally.

see here for how to connect() instead of launch() the browser:
https://playwright.dev/python/docs/api/class-browsertype#browser-type-connect

Describe alternatives you've considered
I've played around with scrape_do and browser_base and they both have some drawbacks and sometimes its best to have total control over the browser that is being launched.

Additional context
There seems to be some bugs when trying to run scrapegraph in a long running process that handles multiple requests. It would be just easier if I can place the chrome server on a machine with a GPU and let scrapegraph just control it remotely.

@Kilowhisky
Copy link
Author

Injection point could be right here:

browser = await p.chromium.launch(
headless=self.headless,
proxy=self.proxy,
**self.browser_config,
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant