[fix] FutureWarning from lxml

Just in case if content is None, the original code will skip extract_text(), and
just append the None value to 'content'. So just add allow_none=True, and this
will return None without raising a ValueError in extract_text().
This commit is contained in:
capric98 2022-04-15 20:14:09 +08:00 committed by Markus Heiser
parent 6ff48859e3
commit 8c7e6cc983

View file

@ -142,8 +142,7 @@ def response(resp):
title = extract_text(title)[offset:]
content = eval_xpath_getindex(result, './/div[contains(@class, "compText")]', 0, default='')
if content:
content = extract_text(content)
content = extract_text(content, allow_none=True)
# append result
results.append({'url': url, 'title': title, 'content': content})