Depending on the order the unit tests are executed, the searx.search module is
initalized or not, issue reported in [1]::
Traceback (most recent call last):
File "searxng/tests/unit/test_results.py", line 72, in test_result_merge_by_title
self.container.extend('stract', [fake_result(engine='stract', title='short title')])
File "searxng/searx/results.py", line 243, in extend
histogram_observe(standard_result_count, 'engine', engine_name, 'result', 'count')
File "searxng/searx/metrics/__init__.py", line 49, in histogram_observe
histogram_storage.get(*args).observe(duration)
^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'
To ensure that the searx.search module is initialized, the
- searx.engines.load_engines is replace by
- searx.search.initialize
[1] https://github.com/searxng/searxng/pull/3932#discussion_r1822406569
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- for tests which perform the same arrange/act/assert pattern but with different
data, the data portion has been moved to the ``paramaterized.expand`` fields
- for monolithic tests which performed multiple arrange/act/asserts,
they have been broken up into different unit tests.
- when possible, change generic assert statements to more concise
asserts (i.e. ``assertIsNone``)
This work ultimately is focused on creating smaller and more concise tests.
While paramaterized may make adding new configurations for existing tests
easier, that is just a beneficial side effect. The main benefit is that smaller
tests are easier to reason about, meaning they are easier to debug when they
start failing. This improves the developer experience in debugging what went
wrong when refactoring the project.
Total number of tests went from 192 -> 259; or, broke apart larger tests into 69
more concise ones.
Engines are loaded into global name `searx.engines.engines` other applications
such as statistics or the histogram use this global variable to search for
values in their own memories, which can lead to key errors as described in
- https://github.com/searxng/searxng/issues/2988
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Closes: https://github.com/searxng/searxng/issues/2988
In the past, some files were tested with the standard profile, others with a
profile in which most of the messages were switched off ... some files were not
checked at all.
- ``PYLINT_SEARXNG_DISABLE_OPTION`` has been abolished
- the distinction ``# lint: pylint`` is no longer necessary
- the pylint tasks have been reduced from three to two
1. ./searx/engines -> lint engines with additional builtins
2. ./searx ./searxng_extra ./tests -> lint all other python files
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
When the user choose "Auto-detected", the choice remains on the following queries.
The detected language is displayed.
For example "Auto-detected (en)":
* the next query language is going to be auto detected
* for the current query, the detected language is English.
This replace the autodetect_search_language plugin.
- fix the issue of fetching more the 7000 *languages*
- improve the request function and filter by language & country
- implement time_range_support & safesearch
- add more fields to the response from dailymotion (allow_embed, length)
- better clean up of HTML tags in the 'content' field.
This is more or less a complete rework based on the '/videos' API from [1].
This patch cleans up the language list in SearXNG that has been polluted by the
ISO-639-3 2 and 3 letter codes from dailymotion languages which have never been
used.
[1] https://developers.dailymotion.com/tools/
Closes: https://github.com/searxng/searxng/issues/1065
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The ? search operator has been broken for some time and
currently only raises the question why it's still there.
## Context ##
The query "Paris !images" searches for "Paris" in the "images" category.
Once upon a time Searx supported "Paris ?images" to search for "Paris"
in the currently enabled categories and the "images" category.
The feature makes sense ... the ? syntax does not.
We will hopefully introduce a +!images syntax in the future.
Fixes#702.
it prepares the new architecture change,
everything about multithreading in moved in the searx.search.* packages
previously the call to the "init" function of the engines was done in searx.engines:
* the network was not set (request not sent using the defined proxy)
* it requires to monkey patch the code to avoid HTTP requests during the tests
The new url parameter "timeout_limit" set timeout limit defined in second.
Example "timeout_limit=1.5" means the timeout limit is 1.5 seconds.
In addition, the query can start with <[number] to set the timeout limit.
For number between 0 and 99, the unit is the second :
Example: "<30 searx" means the timeout limit is 3 seconds
For number above 100, the unit is the millisecond:
Example: "<850 searx" means the timeout is 850 milliseconds.
In addition, there is a new optional setting: outgoing.max_request_timeout.
If not set, the user timeout can't go above searx configuration (as before: the max timeout of selected engine for a query).
If the value is set, the user can set a timeout between 0 and max_request_timeout using
<[number] or timeout_limit query parameter.
Related to #1077
Updated version of PR #1413 from @isj-privacore
Markus Heiser
Grant Lanham
Richard Lyons
Bnyro
Alexandre Flament
Martin Fischer
Marc Abonce Seguin
Adam Tauber
marc