Inputs

scrapy-poet integration, if enabled during the initial setup, allows obtaining the following inputs from web-poet and zyte-common-items through Zyte API:

• web_poet.BrowserHtml

• web_poet.BrowserResponse

• web_poet.AnyResponse

  This re-uses either web_poet.BrowserResponse (takes priority) or web_poet.HttpResponse if they’re available.

  If neither is available, it would use web_poet.HttpResponse requested from Zyte API. However, if other item inputs (e.g. zyte_common_items.Product) are present, it would request web_poet.BrowserResponse from Zyte API unless an extraction source is provided.

• zyte_common_items.Article

• zyte_common_items.ArticleList

• zyte_common_items.ArticleNavigation

• zyte_common_items.JobPosting

• zyte_common_items.JobPostingNavigation

• zyte_common_items.Product

• zyte_common_items.ProductList

• zyte_common_items.ProductNavigation

• zyte_common_items.Serp

Additional inputs and input annotations are also provided:

Built-in inputs

class scrapy_zyte_api.Actions(results: list[_ActionResult] | None)

A page input that specifies browser actions and contains their results.

The actions must be specified with an annotation using actions().

results: list[_ActionResult] | None

Results of actions.

class scrapy_zyte_api.CapturedResponse(url: str, status: int, headers: dict[str, str], body: bytes | None)

A network response captured during browser page rendering.

Part of NetworkCapture.

body: bytes | None

Response body. None if httpResponseBody was not set to True on the matching filter.

headers: dict[str, str]

Response headers.

status: int

HTTP status code.

url: str

Response URL.

class scrapy_zyte_api.Geolocation

A page input that forces a given geolocation for all other page inputs.

The target geolocation must be specified with an annotation.

class scrapy_zyte_api.NetworkCapture(results: list[CapturedResponse])

A page input that specifies network capture filters and contains captured responses.

The filters must be specified with an annotation using network_capture().

results: list[CapturedResponse]

Captured responses.

class scrapy_zyte_api.Screenshot(body: bytes)

A container for holding the screenshot of a webpage.

body: bytes

Body.

Built-in input annotations

enum scrapy_zyte_api.ExtractFrom(value)

Annotation to specify the extraction source of an automatic extraction input, such as Product or Article.

See Dependency annotations.

Member Type:

str

Valid values are as follows:

httpResponseBody = <ExtractFrom.httpResponseBody: 'httpResponseBody'>

browserHtml = <ExtractFrom.browserHtml: 'browserHtml'>

scrapy_zyte_api.actions(value: Iterable[Action]) → tuple[Any, ...]

Convert an iterable of Action dicts into a hashable value.

scrapy_zyte_api.custom_attrs(input: dict[str, Any], options: dict[str, Any] | None = None) → tuple[frozenset[Any], frozenset[Any] | None]

scrapy_zyte_api.network_capture(filters: Iterable[NetworkCaptureFilter]) → tuple[Any, ...]

Convert an iterable of NetworkCaptureFilter dicts into a hashable value.