请求使用示例¶使用FormRequest通过HTTP Post发送数据¶如果您想在spider中模拟HTML表单发布并发送几个键值字段,可以返回 return [FormRequest(url="http://www.example.com/post/action",
formdata={'name': 'John Doe', 'age': '27'},
callback=self.after_post)]
使用formRequest.from_response()模拟用户登录¶网站通常通过 import scrapy
def authentication_failed(response):
# TODO: Check the contents of the response and return True if it failed
# or False if it succeeded.
pass
class LoginSpider(scrapy.Spider):
name = 'example.com'
start_urls = ['http://www.example.com/users/login.php']
def parse(self, response):
return scrapy.FormRequest.from_response(
response,
formdata={'username': 'john', 'password': 'secret'},
callback=self.after_login
)
def after_login(self, response):
if authentication_failed(response):
self.logger.error("Login failed")
return
# continue scraping with authenticated session...
JsonRequest¶JsonRequest类扩展了
JsonRequest用法示例¶使用JSON负载发送JSON POST请求: data = {
'name1': 'value1',
'name2': 'value2',
}
yield JsonRequest(url='http://www.example.com/post/action', data=data) |
Archiver|手机版|笨鸟自学网 ( 粤ICP备20019910号 )
GMT+8, 2024-9-8 09:53 , Processed in 0.025174 second(s), 17 queries .