Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Differences between @request and @browser #216

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Differences between @request and @browser
A table that highlights the key differences between the `@request` and `@browser` decorators available in Botasaurus
drego85 authored Jan 8, 2025
commit d645f81a3668f467978e33f45553f71b96c5aaef
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -281,6 +281,27 @@ def scrape_heading_task(driver: Driver, data):
scrape_heading_task()
```

## Differences between `@request` and `@browser`

Below is a table that highlights the key differences between the `@request` and `@browser` decorators available in Botasaurus:

| **Feature** | **`@request`** | **`@browser`** |
|----------------------------|------------------------------------------------------|------------------------------------------------------|
| **Use Case** | For direct scraping via HTTP requests (static sites or APIs). | For scraping via a browser (dynamic sites or JavaScript-heavy pages). |
| **Speed** | Very fast (direct requests). | Slower (requires rendering the page in a browser). |
| **JavaScript Execution** | Not supported. | Supported (the browser executes the page's JavaScript). |
| **Anti-bot Measures** | Limited (depends on proxy, headers, etc.). | More effective (simulates human-like behavior in a browser). |
| **Page Interaction** | Not possible (no clicks, scrolling, etc.). | Supported (simulate clicks, scrolling, form filling, etc.). |
| **Screenshots** | Not available. | Available (via `driver.screenshot()` or similar tools). |
| **Data Parsing** | Works with returned HTML or JSON sources. | Works with the DOM rendered in the browser. |
| **Suitable for** | - Static pages. <br> - APIs. <br> - Structured data (HTML, JSON). | - Dynamic pages. <br> - Sites with anti-bot protections. |
| **Setup Requirements** | No browser required (uses HTTP requests only). | Requires a browser (e.g., Chrome) and WebDriver. |
| **Proxy/User-Agent Rotation** | Supported manually. | Supported manually but requires configuration in the driver. |
| **Ease of Use** | Simpler (no browser setup required). | More complex (requires WebDriver and browser setup). |

### When to Use `@request` or `@browser`?
- Use **`@request`** for fast and lightweight scraping of static pages or APIs.
- Use **`@browser`** for dynamic pages, sites requiring JavaScript execution, or in the presence of anti-bot measures.

### What are the benefits of a UI Scraper?