Manual request parameters

To send a Scrapy request through Zyte API with manually-defined Zyte API request parameters, define your parameters in the zyte_api key in Request.meta as a dict.

The only exception is the url parameter, which should not be defined as a Zyte API parameter. The value from Request.url is used automatically.

For example:

import scrapy


class SampleQuotesSpider(scrapy.Spider):
    name = "sample_quotes"

    def start_requests(self):
        yield scrapy.Request(
            url="https://quotes.toscrape.com/",
            meta={
                "zyte_api": {
                    "browserHtml": True,
                }
            },
        )

    def parse(self, response):
        print(response.text)
        # "<html>…</html>"

Note that response headers are necessary for raw response decoding. When defining parameters manually and requesting httpResponseBody, remember to also request httpResponseHeaders:

import scrapy


class SampleQuotesSpider(scrapy.Spider):
    name = "sample_quotes"

    def start_requests(self):
        yield scrapy.Request(
            url="https://quotes.toscrape.com/",
            meta={
                "zyte_api": {
                    "httpResponseBody": True,
                    "httpResponseHeaders": True,
                }
            },
        )

    def parse(self, response):
        print(response.text)
        # "<html>…</html>"

To learn more about Zyte API parameters, see the upstream usage and API reference pages.