Response mapping

Parameters

Zyte API response parameters are mapped into response class attributes where possible:

url becomes response.url.
statusCode becomes response.status.
httpResponseHeaders and experimental.responseCookies become response.headers.
experimental.responseCookies is also mapped into the request cookiejar.
browserHtml and httpResponseBody are mapped into both response.text and response.body.

If none of these parameters were present, e.g. if the only requested output was screenshot, response.text and response.body would be empty.

If a future version of Zyte API supported requesting both outputs on the same request, and both parameters were present, browserHtml would be the one mapped into response.text and response.body.

Both response classes have a response.raw_api_response attribute that contains a dict with the complete, raw response from Zyte API, where you can find all Zyte API response parameters, including those that are not mapped into other response class attributes.

For example, for a request for httpResponseBody and httpResponseHeaders, you would get:

def parse(self, response):
    print(response.url)
    # "https://quotes.toscrape.com/"
    print(response.status)
    # 200
    print(response.headers)
    # {b"Content-Type": [b"text/html"], …}
    print(response.text)
    # "<html>…</html>"
    print(response.body)
    # b"<html>…</html>"
    print(response.raw_api_response)
    # {
    #     "url": "https://quotes.toscrape.com/",
    #     "statusCode": 200,
    #     "httpResponseBody": "PGh0bWw+4oCmPC9odG1sPg==",
    #     "httpResponseHeaders": […],
    # }

For a request for screenshot, on the other hand, the response would look as follows:

def parse(self, response):
    print(response.url)
    # "https://quotes.toscrape.com/"
    print(response.status)
    # 200
    print(response.headers)
    # {}
    print(response.text)
    # ""
    print(response.body)
    # b""
    print(response.raw_api_response)
    # {
    #     "url": "https://quotes.toscrape.com/",
    #     "statusCode": 200,
    #     "screenshot": "iVBORw0KGgoAAAANSUh…",
    # }
    from base64 import b64decode

    print(b64decode(response.raw_api_response["screenshot"]))
    # b'\x89PNG\r\n\x1a\n\x00\x00\x00\r…'

Classes

Zyte API responses are mapped with one of the following classes:

ZyteAPITextResponse is used to map text responses, i.e. responses with browserHtml or responses with both httpResponseBody and httpResponseHeaders with a text body (e.g. plain text, HTML, JSON).
ZyteAPIResponse is used to map any other response.

class scrapy_zyte_api.responses.ZyteAPIResponse(*args: Any, **kwargs: Any)[source]