延迟信号处理程序¶一些信号支持返回 举个例子: class SignalSpider(scrapy.Spider):
name = 'signals'
start_urls = ['http://quotes.toscrape.com/page/1/']
@classmethod
def from_crawler(cls, crawler, *args, **kwargs):
spider = super(SignalSpider, cls).from_crawler(crawler, *args, **kwargs)
crawler.signals.connect(spider.item_scraped, signal=signals.item_scraped)
return spider
def item_scraped(self, item):
# Send the scraped item to the server
d = treq.post(
'http://example.com/post',
json.dumps(item).encode('ascii'),
headers={b'Content-Type': [b'application/json']}
)
# The next item will be scraped only after
# deferred (d) is fired
return d
def parse(self, response):
for quote in response.css('div.quote'):
yield {
'text': quote.css('span.text::text').get(),
'author': quote.css('small.author::text').get(),
'tags': quote.css('div.tags a.tag::text').getall(),
}
见 内置信号参考 下面要知道哪些信号支持 内置信号参考¶这是一个零碎的内置信号列表及其含义。 发动机信号¶engine_started¶
注解 此信号可能被触发 之后 这个 engine_stopped¶
项目信号¶注解 最大值时 item_scraped¶
item_dropped¶
|
Archiver|手机版|笨鸟自学网 ( 粤ICP备20019910号 )
GMT+8, 2024-9-8 10:13 , Processed in 0.043436 second(s), 17 queries .