Response mapping
Parameters
Zyte API response parameters are mapped into response class attributes where possible:
url becomes
response.url
.statusCode becomes
response.status
.httpResponseHeaders and experimental.responseCookies become
response.headers
.experimental.responseCookies is also mapped into the request
cookiejar
.browserHtml and httpResponseBody are mapped into both
response.text
andresponse.body
.If none of these parameters were present, e.g. if the only requested output was screenshot,
response.text
andresponse.body
would be empty.If a future version of Zyte API supported requesting both outputs on the same request, and both parameters were present, browserHtml would be the one mapped into
response.text
andresponse.body
.
Both response classes have a
response.raw_api_response
attribute that contains a dict
with the complete, raw response from
Zyte API, where you can find all Zyte API response parameters, including those
that are not mapped into other response class attributes.
For example, for a request for httpResponseBody and httpResponseHeaders, you would get:
def parse(self, response):
print(response.url)
# "https://quotes.toscrape.com/"
print(response.status)
# 200
print(response.headers)
# {b"Content-Type": [b"text/html"], …}
print(response.text)
# "<html>…</html>"
print(response.body)
# b"<html>…</html>"
print(response.raw_api_response)
# {
# "url": "https://quotes.toscrape.com/",
# "statusCode": 200,
# "httpResponseBody": "PGh0bWw+4oCmPC9odG1sPg==",
# "httpResponseHeaders": […],
# }
For a request for screenshot, on the other hand, the response would look as follows:
def parse(self, response):
print(response.url)
# "https://quotes.toscrape.com/"
print(response.status)
# 200
print(response.headers)
# {}
print(response.text)
# ""
print(response.body)
# b""
print(response.raw_api_response)
# {
# "url": "https://quotes.toscrape.com/",
# "statusCode": 200,
# "screenshot": "iVBORw0KGgoAAAANSUh…",
# }
from base64 import b64decode
print(b64decode(response.raw_api_response["screenshot"]))
# b'\x89PNG\r\n\x1a\n\x00\x00\x00\r…'
Classes
Zyte API responses are mapped with one of the following classes:
ZyteAPITextResponse
is used to map text responses, i.e. responses with browserHtml or responses with both httpResponseBody and httpResponseHeaders with a text body (e.g. plain text, HTML, JSON).ZyteAPIResponse
is used to map any other response.
- class scrapy_zyte_api.responses.ZyteAPIResponse(*args: Any, **kwargs: Any)[source]
Bases:
ZyteAPIMixin
,Response
- url
- status
- headers
- raw_api_response
Contains the raw API response from Zyte API.
For the full list of parameters, see Zyte API reference documentation.