Commit graph

24 commits

Author SHA1 Message Date
Markus Heiser 2de007138c [fix] prepare for pylint 2.14.0
Remove issue reported by Pylint 2.14.0:

- no-self-use: has been moved to optional extension [1]
- The refactoring checker now also raises 'consider-using-generator' messages
  for max(), min() and sum(). [2]

.pylintrc:
  - <option name>-hint has been removed since long, Pylint 2.14.0 raises an
    error on invalid options
  - bad-continuation and bad-whitespace have been removed [3]

[1] https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/summary.html#removed-checkers
[2] https://pylint.pycqa.org/en/latest/whatsnew/2/2.14/full.html#what-s-new-in-pylint-2-14-0
[2] https://pylint.pycqa.org/en/latest/whatsnew/2/2.6/summary.html#summary-release-highlights

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-06-03 15:41:52 +02:00
Alexandre Flament 2134703b4b [enh] settings.yml: implement general.enable_metrics
* allow not to record metrics (response time, etc...)
* this commit doesn't change the UI. If the metrics are disabled
  /stats and /stats/errors will return empty response.
  in /preferences, the columns response time and reliability will be empty.
2022-01-05 19:03:04 +01:00
Markus Heiser 3d96a9839a [format.python] initial formatting of the python code
This patch was generated by black [1]::

    make format.python

[1] https://github.com/psf/black

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-27 09:26:22 +01:00
Alexandre Flament f9c6393502 [enh] verify that Tor proxy works every time searx starts
based on @MarcAbonce commit on searx
2021-10-12 21:01:02 +02:00
Alexandre Flament b513917ef9 [mod] searx.metrics & searx.search: use the engine loggers
metrics & processors use the engine logger
2021-09-10 21:49:34 +02:00
Alexandre Flament 0b27c8698f [doc] update docs/dev/plugins.rst 2021-09-10 10:58:22 +02:00
Alexandre Flament 660c180170 [mod] plugin: call on_result after each engine from the ResultContainer
Currently, searx.search.Search calls on_result once the engine results have been merged
(ResultContainer.order_results).

on_result plugins can rewrite the results: once the URL(s) are modified, even they can be merged,
it won't be the case since ResultContainer.order_results has already be called.

This commit call on_result inside for each result of each engines.
In addition the on_result function can return False to remove the result.

Note: the on_result function now run on the engine thread instead of the Flask thread.
2021-09-09 11:31:44 +02:00
Markus Heiser 2a3b9a2e26 [pylint] searx: drop no longer needed 'missing-function-docstring'
Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914168470
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-07 13:34:35 +02:00
Alexandre Flament 6fa114c9ba [mod] settings_default: remove searx.search.max_request_timeout global variable 2021-06-01 08:10:15 +02:00
Markus Heiser 6f1446d55f [pylint] searx/search/__init__.py & replace lic-text by SPDX tag
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-21 17:31:22 +02:00
Alexandre Flament 426fadccb3 [mod] remove gc.collect() after each user request 2021-05-21 17:23:18 +02:00
Alexandre Flament 8c1a65d32f [mod] multithreading only in searx.search.* packages
it prepares the new architecture change,
everything about multithreading in moved in the searx.search.* packages

previously the call to the "init" function of the engines was done in searx.engines:
* the network was not set (request not sent using the defined proxy)
* it requires to monkey patch the code to avoid HTTP requests during the tests
2021-05-05 13:12:42 +02:00
Alexandre Flament 7acd7ffc02 [enh] rewrite and enhance metrics 2021-04-21 16:24:46 +02:00
Alexandre Flament aae7830d14 [mod] refactoring: processors
Report to the user suspended engines.

searx.search.processor.abstract:
* manages suspend time (per network).
* reports suspended time to the ResultContainer (method extend_container_if_suspended)
* adds the results to the ResultContainer (method extend_container)
* handles exceptions (method handle_exception)
2021-04-21 16:24:46 +02:00
Alexandre Flament 0b45afd4d7 [fix] checker: various bug fixes
* initialize engine_data (youtube engine)
* don't crash if an engine don't set result['url']
2021-03-25 09:37:37 +01:00
Adam Tauber 44f4a9d49a [enh] add ability to send engine data to subsequent requests 2021-03-06 12:12:35 +01:00
Alexandre Flament 3a9f513521 [enh] checker: background check
See settings.yml for the options
SIGUSR1 signal starts the checker.
The result is available at /stats/checker
2021-01-12 11:47:17 +01:00
Alexandre Flament 8cbc9f2d58 [enh] add checker 2021-01-12 11:47:17 +01:00
Alexandre Flament eda8934f15 [mod] searx.search.EngineRef: remove from_bang parameter
from_bang is True when the user query contains a bang.
In this case the category is also set to 'none'.

from_bang only usage was in searx.webadapter.parse_specific :
if from_bang is True, then the EngineRef category is ignored and force to 'none'.

This commit also removes the searx.webadapter.parse_sepecific function.
2020-12-18 12:29:48 +01:00
Alexandre Flament 995ba2f406 [mod] searx.search.SearchQuery: remove categories parameter
The categories parameter is useless in the constructor:
it is always the categories from the EngineRef.

The categories becomes a property.
2020-12-18 12:29:48 +01:00
Alexandre Flament 5c6a5407a0 [fix] fix of PR #2225 2020-12-17 16:49:48 +01:00
Alexandre Flament 7ec8bc3ea7 [mod] split searx.search into different processors
see searx.search.processors.abstract.EngineProcessor

First the method searx call the get_params method.

If the return value is not None, then the searx call the method search.
2020-12-17 11:39:36 +01:00
Alexandre Flament c0cc01e936 [mod] searx.search: search_multiple_requests is a method of Search class 2020-12-17 11:39:36 +01:00
Alexandre Flament 3b87efb3db [mod] move seax/search.py to searx/search/__init__.py 2020-12-17 11:39:36 +01:00
Renamed from searx/search.py (Browse further)