mirror of
https://github.com/searxng/searxng.git
synced 2025-12-22 19:50:00 +00:00
[refactor] typification of SearXNG (initial) / result items (part 1)
Typification of SearXNG
=======================
This patch introduces the typing of the results. The why and how is described
in the documentation, please generate the documentation ..
$ make docs.clean docs.live
and read the following articles in the "Developer documentation":
- result types --> http://0.0.0.0:8000/dev/result_types/index.html
The result types are available from the `searx.result_types` module. The
following have been implemented so far:
- base result type: `searx.result_type.Result`
--> http://0.0.0.0:8000/dev/result_types/base_result.html
- answer results
--> http://0.0.0.0:8000/dev/result_types/answer.html
including the type for translations (inspired by #3925). For all other
types (which still need to be set up in subsequent PRs), template documentation
has been created for the transition period.
Doc of the fields used in Templates
===================================
The template documentation is the basis for the typing and is the first complete
documentation of the results (needed for engine development). It is the
"working paper" (the plan) with which further typifications can be implemented
in subsequent PRs.
- https://github.com/searxng/searxng/issues/357
Answer Templates
================
With the new (sub) types for `Answer`, the templates for the answers have also
been revised, `Translation` are now displayed with collapsible entries (inspired
by #3925).
!en-de dog
Plugins & Answerer
==================
The implementation for `Plugin` and `Answer` has been revised, see
documentation:
- Plugin: http://0.0.0.0:8000/dev/plugins/index.html
- Answerer: http://0.0.0.0:8000/dev/answerers/index.html
With `AnswerStorage` and `AnswerStorage` to manage those items (in follow up
PRs, `ArticleStorage`, `InfoStorage` and .. will be implemented)
Autocomplete
============
The autocompletion had a bug where the results from `Answer` had not been shown
in the past. To test activate autocompletion and try search terms for which we
have answerers
- statistics: type `min 1 2 3` .. in the completion list you should find an
entry like `[de] min(1, 2, 3) = 1`
- random: type `random uuid` .. in the completion list, the first item is a
random UUID
Extended Types
==============
SearXNG extends e.g. the request and response types of flask and httpx, a module
has been set up for type extensions:
- Extended Types
--> http://0.0.0.0:8000/dev/extended_types.html
Unit-Tests
==========
The unit tests have been completely revised. In the previous implementation,
the runtime (the global variables such as `searx.settings`) was not initialized
before each test, so the runtime environment with which a test ran was always
determined by the tests that ran before it. This was also the reason why we
sometimes had to observe non-deterministic errors in the tests in the past:
- https://github.com/searxng/searxng/issues/2988 is one example for the Runtime
issues, with non-deterministic behavior ..
- https://github.com/searxng/searxng/pull/3650
- https://github.com/searxng/searxng/pull/3654
- https://github.com/searxng/searxng/pull/3642#issuecomment-2226884469
- https://github.com/searxng/searxng/pull/3746#issuecomment-2300965005
Why msgspec.Struct
==================
We have already discussed typing based on e.g. `TypeDict` or `dataclass` in the past:
- https://github.com/searxng/searxng/pull/1562/files
- https://gist.github.com/dalf/972eb05e7a9bee161487132a7de244d2
- https://github.com/searxng/searxng/pull/1412/files
- https://github.com/searxng/searxng/pull/1356
In my opinion, TypeDict is unsuitable because the objects are still dictionaries
and not instances of classes / the `dataclass` are classes but ...
The `msgspec.Struct` combine the advantages of typing, runtime behaviour and
also offer the option of (fast) serializing (incl. type check) the objects.
Currently not possible but conceivable with `msgspec`: Outsourcing the engines
into separate processes, what possibilities this opens up in the future is left
to the imagination!
Internally, we have already defined that it is desirable to decouple the
development of the engines from the development of the SearXNG core / The
serialization of the `Result` objects is a prerequisite for this.
HINT: The threads listed above were the template for this PR, even though the
implementation here is based on msgspec. They should also be an inspiration for
the following PRs of typification, as the models and implementations can provide
a good direction.
Why just one commit?
====================
I tried to create several (thematically separated) commits, but gave up at some
point ... there are too many things to tackle at once / The comprehensibility of
the commits would not be improved by a thematic separation. On the contrary, we
would have to make multiple changes at the same places and the goal of a change
would be vaguely recognizable in the fog of the commits.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
Markus Heiser
committed by
Markus Heiser
Markus Heiser
parent
9079d0cac0
commit
edfbf1e118
@@ -1,8 +1,10 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
import os
|
||||
from os.path import dirname, sep, abspath
|
||||
from pathlib import Path
|
||||
|
||||
# In unit tests the user settings from unit/settings/test_settings.yml are used.
|
||||
os.environ['SEARXNG_SETTINGS_PATH'] = abspath(dirname(__file__) + sep + 'settings' + sep + 'test_settings.yml')
|
||||
# By default, in unit tests the user settings from
|
||||
# unit/settings/test_settings.yml are used.
|
||||
|
||||
os.environ['SEARXNG_SETTINGS_PATH'] = str(Path(__file__).parent / "settings" / "test_settings.yml")
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
@@ -1,27 +1,12 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
|
||||
'''
|
||||
searx is free software: you can redistribute it and/or modify
|
||||
it under the terms of the GNU Affero General Public License as published by
|
||||
the Free Software Foundation, either version 3 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
searx is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
GNU Affero General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU Affero General Public License
|
||||
along with searx. If not, see < http://www.gnu.org/licenses/ >.
|
||||
|
||||
'''
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from searx.engines import command as command_engine
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
class TestCommandEngine(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestCommandEngine(SearxTestCase):
|
||||
|
||||
def test_basic_seq_command_engine(self):
|
||||
ls_engine = command_engine
|
||||
ls_engine.command = ['seq', '{{QUERY}}']
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from collections import defaultdict
|
||||
import mock
|
||||
@@ -12,7 +12,7 @@ from tests import SearxTestCase
|
||||
logger = logger.getChild('engines')
|
||||
|
||||
|
||||
class TestXpathEngine(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestXpathEngine(SearxTestCase):
|
||||
html = """
|
||||
<div>
|
||||
<div class="search_result">
|
||||
@@ -29,6 +29,7 @@ class TestXpathEngine(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
"""
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
xpath.logger = logger.getChild('test_xpath')
|
||||
|
||||
def test_request(self):
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
@@ -1,17 +1,15 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring, protected-access
|
||||
|
||||
from mock import patch
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
import httpx
|
||||
from mock import patch
|
||||
|
||||
from searx.network.network import Network, NETWORKS, initialize
|
||||
from searx.network.network import Network, NETWORKS
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
class TestNetwork(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
def setUp(self):
|
||||
initialize()
|
||||
class TestNetwork(SearxTestCase):
|
||||
# pylint: disable=protected-access
|
||||
|
||||
def test_simple(self):
|
||||
network = Network()
|
||||
@@ -122,10 +120,13 @@ class TestNetwork(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
await network.aclose()
|
||||
|
||||
|
||||
class TestNetworkRequestRetries(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestNetworkRequestRetries(SearxTestCase):
|
||||
|
||||
TEXT = 'Lorem Ipsum'
|
||||
|
||||
def setUp(self):
|
||||
self.init_test_settings()
|
||||
|
||||
@classmethod
|
||||
def get_response_404_then_200(cls):
|
||||
first = True
|
||||
@@ -195,10 +196,13 @@ class TestNetworkRequestRetries(SearxTestCase): # pylint: disable=missing-class
|
||||
await network.aclose()
|
||||
|
||||
|
||||
class TestNetworkStreamRetries(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestNetworkStreamRetries(SearxTestCase):
|
||||
|
||||
TEXT = 'Lorem Ipsum'
|
||||
|
||||
def setUp(self):
|
||||
self.init_test_settings()
|
||||
|
||||
@classmethod
|
||||
def get_response_exception_then_200(cls):
|
||||
first = True
|
||||
|
||||
@@ -1,2 +1,2 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
@@ -1,31 +1,16 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from searx.search import SearchQuery, EngineRef
|
||||
from searx.search.processors import online
|
||||
import searx.search
|
||||
from searx import engines
|
||||
|
||||
from tests import SearxTestCase
|
||||
|
||||
TEST_ENGINE_NAME = 'dummy engine'
|
||||
TEST_ENGINE = {
|
||||
'name': TEST_ENGINE_NAME,
|
||||
'engine': 'dummy',
|
||||
'categories': 'general',
|
||||
'shortcut': 'du',
|
||||
'timeout': 3.0,
|
||||
'tokens': [],
|
||||
}
|
||||
TEST_ENGINE_NAME = "dummy engine" # from the ./settings/test_settings.yml
|
||||
|
||||
|
||||
class TestOnlineProcessor(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
|
||||
def setUp(self):
|
||||
searx.search.initialize([TEST_ENGINE])
|
||||
|
||||
def tearDown(self):
|
||||
searx.search.load_engines([])
|
||||
class TestOnlineProcessor(SearxTestCase):
|
||||
|
||||
def _get_params(self, online_processor, search_query, engine_category):
|
||||
params = online_processor.get_params(search_query, engine_category)
|
||||
|
||||
2
tests/unit/settings/limiter.toml
Normal file
2
tests/unit/settings/limiter.toml
Normal file
@@ -0,0 +1,2 @@
|
||||
[botdetection.ip_limit]
|
||||
link_token = true
|
||||
8
tests/unit/settings/test_result_container.yml
Normal file
8
tests/unit/settings/test_result_container.yml
Normal file
@@ -0,0 +1,8 @@
|
||||
# This SearXNG setup is used in unit tests
|
||||
|
||||
use_default_settings:
|
||||
|
||||
engines:
|
||||
keep_only:
|
||||
- google
|
||||
- duckduckgo
|
||||
@@ -1,10 +1,30 @@
|
||||
# This SearXNG setup is used in unit tests
|
||||
|
||||
use_default_settings: true
|
||||
use_default_settings:
|
||||
|
||||
engines:
|
||||
# remove all engines
|
||||
keep_only: []
|
||||
|
||||
search:
|
||||
|
||||
formats: [html, csv, json, rss]
|
||||
|
||||
server:
|
||||
|
||||
secret_key: "user_secret_key"
|
||||
|
||||
engines:
|
||||
- name: general dummy
|
||||
|
||||
- name: dummy engine
|
||||
engine: demo_offline
|
||||
categories: ["general"]
|
||||
shortcut: "gd"
|
||||
timeout: 3
|
||||
|
||||
- name: dummy private engine
|
||||
engine: demo_offline
|
||||
categories: ["general"]
|
||||
shortcut: "gdp"
|
||||
timeout: 3
|
||||
tokens: ["my-token"]
|
||||
|
||||
16
tests/unit/settings/test_tineye.yml
Normal file
16
tests/unit/settings/test_tineye.yml
Normal file
@@ -0,0 +1,16 @@
|
||||
# This SearXNG setup is used in unit tests
|
||||
|
||||
use_default_settings:
|
||||
|
||||
engines:
|
||||
# remove all engines
|
||||
keep_only: []
|
||||
|
||||
engines:
|
||||
|
||||
- name: tineye
|
||||
engine: tineye
|
||||
categories: ["general"]
|
||||
shortcut: "tin"
|
||||
timeout: 9.0
|
||||
disabled: true
|
||||
@@ -1,17 +1,34 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from mock import Mock
|
||||
from parameterized import parameterized
|
||||
|
||||
from searx.answerers import answerers
|
||||
import searx.plugins
|
||||
import searx.answerers
|
||||
import searx.preferences
|
||||
|
||||
from searx.extended_types import sxng_request
|
||||
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
class AnswererTest(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
@parameterized.expand(answerers)
|
||||
def test_unicode_input(self, answerer):
|
||||
query = Mock()
|
||||
unicode_payload = 'árvíztűrő tükörfúrógép'
|
||||
query.query = '{} {}'.format(answerer.keywords[0], unicode_payload)
|
||||
self.assertIsInstance(answerer.answer(query), list)
|
||||
class AnswererTest(SearxTestCase):
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
|
||||
self.storage = searx.plugins.PluginStorage()
|
||||
engines = {}
|
||||
self.pref = searx.preferences.Preferences(["simple"], ["general"], engines, self.storage)
|
||||
self.pref.parse_dict({"locale": "en"})
|
||||
|
||||
@parameterized.expand(searx.answerers.STORAGE.answerer_list)
|
||||
def test_unicode_input(self, answerer_obj: searx.answerers.Answerer):
|
||||
|
||||
with self.app.test_request_context():
|
||||
sxng_request.preferences = self.pref
|
||||
|
||||
unicode_payload = "árvíztűrő tükörfúrógép"
|
||||
for keyword in answerer_obj.keywords:
|
||||
query = f"{keyword} {unicode_payload}"
|
||||
self.assertIsInstance(answerer_obj.answer(query), list)
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from unittest.mock import MagicMock, Mock
|
||||
|
||||
from searx.engines import mariadb_server
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
class MariadbServerTests(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class MariadbServerTests(SearxTestCase):
|
||||
|
||||
def test_init_no_query_str_raises(self):
|
||||
self.assertRaises(ValueError, lambda: mariadb_server.init({}))
|
||||
|
||||
@@ -1,25 +1,25 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring
|
||||
|
||||
import logging
|
||||
from datetime import datetime
|
||||
from unittest.mock import Mock
|
||||
from requests import HTTPError
|
||||
from parameterized import parameterized
|
||||
|
||||
import searx.search
|
||||
import searx.engines
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
class TinEyeTests(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TinEyeTests(SearxTestCase):
|
||||
|
||||
TEST_SETTINGS = "test_tineye.yml"
|
||||
|
||||
def setUp(self):
|
||||
searx.search.initialize(
|
||||
[{'name': 'tineye', 'engine': 'tineye', 'shortcut': 'tin', 'timeout': 9.0, 'disabled': True}]
|
||||
)
|
||||
|
||||
super().setUp()
|
||||
self.tineye = searx.engines.engines['tineye']
|
||||
self.tineye.logger.setLevel(logging.CRITICAL)
|
||||
self.tineye.logger.setLevel(logging.INFO)
|
||||
|
||||
def tearDown(self):
|
||||
searx.search.load_engines([])
|
||||
@@ -33,11 +33,12 @@ class TinEyeTests(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
@parameterized.expand([(400), (422)])
|
||||
def test_returns_empty_list(self, status_code):
|
||||
response = Mock()
|
||||
response.json.return_value = {}
|
||||
response.json.return_value = {"suggestions": {"key": "Download Error"}}
|
||||
response.status_code = status_code
|
||||
response.raise_for_status.side_effect = HTTPError()
|
||||
results = self.tineye.response(response)
|
||||
self.assertEqual(0, len(results))
|
||||
with self.assertLogs(self.tineye.logger):
|
||||
results = self.tineye.response(response)
|
||||
self.assertEqual(0, len(results))
|
||||
|
||||
def test_logs_format_for_422(self):
|
||||
response = Mock()
|
||||
|
||||
@@ -1,16 +1,11 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from searx import settings, engines
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
class TestEnginesInit(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
settings['outgoing']['using_tor_proxy'] = False
|
||||
settings['outgoing']['extra_proxy_timeout'] = 0
|
||||
engines.load_engines([])
|
||||
class TestEnginesInit(SearxTestCase):
|
||||
|
||||
def test_initialize_engines_default(self):
|
||||
engine_list = [
|
||||
@@ -23,7 +18,7 @@ class TestEnginesInit(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
self.assertIn('engine1', engines.engines)
|
||||
self.assertIn('engine2', engines.engines)
|
||||
|
||||
def test_initialize_engines_exclude_onions(self): # pylint: disable=invalid-name
|
||||
def test_initialize_engines_exclude_onions(self):
|
||||
settings['outgoing']['using_tor_proxy'] = False
|
||||
engine_list = [
|
||||
{'engine': 'dummy', 'name': 'engine1', 'shortcut': 'e1', 'categories': 'general'},
|
||||
@@ -35,7 +30,7 @@ class TestEnginesInit(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
self.assertIn('engine1', engines.engines)
|
||||
self.assertNotIn('onions', engines.categories)
|
||||
|
||||
def test_initialize_engines_include_onions(self): # pylint: disable=invalid-name
|
||||
def test_initialize_engines_include_onions(self):
|
||||
settings['outgoing']['using_tor_proxy'] = True
|
||||
settings['outgoing']['extra_proxy_timeout'] = 100.0
|
||||
engine_list = [
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from parameterized import parameterized
|
||||
from tests import SearxTestCase
|
||||
@@ -7,7 +7,8 @@ import searx.exceptions
|
||||
from searx import get_setting
|
||||
|
||||
|
||||
class TestExceptions(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestExceptions(SearxTestCase):
|
||||
|
||||
@parameterized.expand(
|
||||
[
|
||||
searx.exceptions.SearxEngineAccessDeniedException,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from searx.external_bang import (
|
||||
get_node,
|
||||
@@ -34,7 +34,7 @@ TEST_DB = {
|
||||
}
|
||||
|
||||
|
||||
class TestGetNode(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestGetNode(SearxTestCase):
|
||||
|
||||
DB = { # pylint:disable=invalid-name
|
||||
'trie': {
|
||||
@@ -65,7 +65,8 @@ class TestGetNode(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
self.assertEqual(after, 's')
|
||||
|
||||
|
||||
class TestResolveBangDefinition(SearxTestCase): # pylint:disable=missing-class-docstring
|
||||
class TestResolveBangDefinition(SearxTestCase):
|
||||
|
||||
def test_https(self):
|
||||
url, rank = resolve_bang_definition('//example.com/' + chr(2) + chr(1) + '42', 'query')
|
||||
self.assertEqual(url, 'https://example.com/query')
|
||||
@@ -77,7 +78,8 @@ class TestResolveBangDefinition(SearxTestCase): # pylint:disable=missing-class-
|
||||
self.assertEqual(rank, 0)
|
||||
|
||||
|
||||
class TestGetBangDefinitionAndAutocomplete(SearxTestCase): # pylint:disable=missing-class-docstring
|
||||
class TestGetBangDefinitionAndAutocomplete(SearxTestCase):
|
||||
|
||||
def test_found(self):
|
||||
bang_definition, new_autocomplete = get_bang_definition_and_autocomplete('exam', external_bangs_db=TEST_DB)
|
||||
self.assertEqual(bang_definition, TEST_DB['trie']['exam'][LEAF_KEY])
|
||||
@@ -109,7 +111,8 @@ class TestGetBangDefinitionAndAutocomplete(SearxTestCase): # pylint:disable=mis
|
||||
self.assertEqual(new_autocomplete, [])
|
||||
|
||||
|
||||
class TestExternalBangJson(SearxTestCase): # pylint:disable=missing-class-docstring
|
||||
class TestExternalBangJson(SearxTestCase):
|
||||
|
||||
def test_no_external_bang_query(self):
|
||||
result = get_bang_url(SearchQuery('test', engineref_list=[EngineRef('wikipedia', 'general')]))
|
||||
self.assertIsNone(result)
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
"""Test some code from module :py:obj:`searx.locales`"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -1,57 +1,57 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
import flask
|
||||
from parameterized.parameterized import parameterized
|
||||
from searx import plugins
|
||||
from searx import preferences
|
||||
|
||||
import searx.plugins
|
||||
import searx.preferences
|
||||
|
||||
from searx.extended_types import sxng_request
|
||||
from searx.plugins._core import _default, ModulePlugin
|
||||
from searx.result_types import Answer
|
||||
from searx.utils import load_module
|
||||
|
||||
from tests import SearxTestCase
|
||||
from .test_utils import random_string
|
||||
|
||||
from .test_plugins import get_search_mock
|
||||
from .test_plugins import do_post_search
|
||||
|
||||
|
||||
class PluginCalculator(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class PluginCalculator(SearxTestCase):
|
||||
|
||||
def setUp(self):
|
||||
from searx import webapp # pylint: disable=import-outside-toplevel
|
||||
super().setUp()
|
||||
|
||||
self.webapp = webapp
|
||||
self.store = plugins.PluginStore()
|
||||
plugin = plugins.load_and_initialize_plugin('searx.plugins.calculator', False, (None, {}))
|
||||
self.store.register(plugin)
|
||||
self.preferences = preferences.Preferences(["simple"], ["general"], {}, self.store)
|
||||
self.preferences.parse_dict({"locale": "en"})
|
||||
f = _default / "calculator.py"
|
||||
mod = load_module(f.name, str(f.parent))
|
||||
engines = {}
|
||||
|
||||
self.storage = searx.plugins.PluginStorage()
|
||||
self.storage.register(ModulePlugin(mod))
|
||||
self.storage.init(self.app)
|
||||
self.pref = searx.preferences.Preferences(["simple"], ["general"], engines, self.storage)
|
||||
self.pref.parse_dict({"locale": "en"})
|
||||
|
||||
def test_plugin_store_init(self):
|
||||
self.assertEqual(1, len(self.store.plugins))
|
||||
self.assertEqual(1, len(self.storage))
|
||||
|
||||
def test_single_page_number_true(self):
|
||||
with self.webapp.app.test_request_context():
|
||||
flask.request.preferences = self.preferences
|
||||
search = get_search_mock(query=random_string(10), pageno=2)
|
||||
result = self.store.call(self.store.plugins, 'post_search', flask.request, search)
|
||||
def test_pageno_1_2(self):
|
||||
with self.app.test_request_context():
|
||||
sxng_request.preferences = self.pref
|
||||
query = "1+1"
|
||||
answer = Answer(results=[], answer=f"{query} = {eval(query)}") # pylint: disable=eval-used
|
||||
|
||||
self.assertTrue(result)
|
||||
self.assertNotIn('calculate', search.result_container.answers)
|
||||
search = do_post_search(query, self.storage, pageno=1)
|
||||
self.assertIn(answer, search.result_container.answers)
|
||||
|
||||
def test_long_query_true(self):
|
||||
with self.webapp.app.test_request_context():
|
||||
flask.request.preferences = self.preferences
|
||||
search = get_search_mock(query=random_string(101), pageno=1)
|
||||
result = self.store.call(self.store.plugins, 'post_search', flask.request, search)
|
||||
search = do_post_search(query, self.storage, pageno=2)
|
||||
self.assertEqual(list(search.result_container.answers), [])
|
||||
|
||||
self.assertTrue(result)
|
||||
self.assertNotIn('calculate', search.result_container.answers)
|
||||
|
||||
def test_alpha_true(self):
|
||||
with self.webapp.app.test_request_context():
|
||||
flask.request.preferences = self.preferences
|
||||
search = get_search_mock(query=random_string(10), pageno=1)
|
||||
result = self.store.call(self.store.plugins, 'post_search', flask.request, search)
|
||||
|
||||
self.assertTrue(result)
|
||||
self.assertNotIn('calculate', search.result_container.answers)
|
||||
def test_long_query_ignored(self):
|
||||
with self.app.test_request_context():
|
||||
sxng_request.preferences = self.pref
|
||||
query = f"1+1 {random_string(101)}"
|
||||
search = do_post_search(query, self.storage)
|
||||
self.assertEqual(list(search.result_container.answers), [])
|
||||
|
||||
@parameterized.expand(
|
||||
[
|
||||
@@ -77,27 +77,22 @@ class PluginCalculator(SearxTestCase): # pylint: disable=missing-class-docstrin
|
||||
("1,0^1,0", "1", "de"),
|
||||
]
|
||||
)
|
||||
def test_localized_query(self, operation: str, contains_result: str, lang: str):
|
||||
with self.webapp.app.test_request_context():
|
||||
self.preferences.parse_dict({"locale": lang})
|
||||
flask.request.preferences = self.preferences
|
||||
search = get_search_mock(query=operation, lang=lang, pageno=1)
|
||||
result = self.store.call(self.store.plugins, 'post_search', flask.request, search)
|
||||
def test_localized_query(self, query: str, res: str, lang: str):
|
||||
with self.app.test_request_context():
|
||||
self.pref.parse_dict({"locale": lang})
|
||||
sxng_request.preferences = self.pref
|
||||
answer = Answer(results=[], answer=f"{query} = {res}")
|
||||
|
||||
self.assertTrue(result)
|
||||
self.assertIn('calculate', search.result_container.answers)
|
||||
self.assertIn(contains_result, search.result_container.answers['calculate']['answer'])
|
||||
search = do_post_search(query, self.storage)
|
||||
self.assertIn(answer, search.result_container.answers)
|
||||
|
||||
@parameterized.expand(
|
||||
[
|
||||
"1/0",
|
||||
]
|
||||
)
|
||||
def test_invalid_operations(self, operation):
|
||||
with self.webapp.app.test_request_context():
|
||||
flask.request.preferences = self.preferences
|
||||
search = get_search_mock(query=operation, pageno=1)
|
||||
result = self.store.call(self.store.plugins, 'post_search', flask.request, search)
|
||||
|
||||
self.assertTrue(result)
|
||||
self.assertNotIn('calculate', search.result_container.answers)
|
||||
def test_invalid_operations(self, query):
|
||||
with self.app.test_request_context():
|
||||
sxng_request.preferences = self.pref
|
||||
search = do_post_search(query, self.storage)
|
||||
self.assertEqual(list(search.result_container.answers), [])
|
||||
|
||||
@@ -1,51 +1,69 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring, invalid-name
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from mock import Mock
|
||||
from parameterized.parameterized import parameterized
|
||||
from searx import plugins
|
||||
|
||||
import searx.plugins
|
||||
import searx.preferences
|
||||
|
||||
from searx.extended_types import sxng_request
|
||||
from searx.result_types import Answer
|
||||
|
||||
from tests import SearxTestCase
|
||||
from .test_plugins import do_post_search
|
||||
|
||||
from .test_plugins import get_search_mock
|
||||
query_res = [
|
||||
("md5 test", "md5 hash digest: 098f6bcd4621d373cade4e832627b4f6"),
|
||||
("sha1 test", "sha1 hash digest: a94a8fe5ccb19ba61c4c0873d391e987982fbbd3"),
|
||||
("sha224 test", "sha224 hash digest: 90a3ed9e32b2aaf4c61c410eb925426119e1a9dc53d4286ade99a809"),
|
||||
("sha256 test", "sha256 hash digest: 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"),
|
||||
(
|
||||
"sha384 test",
|
||||
"sha384 hash digest: 768412320f7b0aa5812fce428dc4706b3c"
|
||||
"ae50e02a64caa16a782249bfe8efc4b7ef1ccb126255d196047dfedf1"
|
||||
"7a0a9",
|
||||
),
|
||||
(
|
||||
"sha512 test",
|
||||
"sha512 hash digest: ee26b0dd4af7e749aa1a8ee3c10ae9923f6"
|
||||
"18980772e473f8819a5d4940e0db27ac185f8a0e1d5f84f88bc887fd67b143732c304cc5"
|
||||
"fa9ad8e6f57f50028a8ff",
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
class PluginHashTest(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class PluginHashTest(SearxTestCase):
|
||||
|
||||
def setUp(self):
|
||||
self.store = plugins.PluginStore()
|
||||
plugin = plugins.load_and_initialize_plugin('searx.plugins.hash_plugin', False, (None, {}))
|
||||
self.store.register(plugin)
|
||||
super().setUp()
|
||||
engines = {}
|
||||
|
||||
self.storage = searx.plugins.PluginStorage()
|
||||
self.storage.register_by_fqn("searx.plugins.hash_plugin.SXNGPlugin")
|
||||
self.storage.init(self.app)
|
||||
self.pref = searx.preferences.Preferences(["simple"], ["general"], engines, self.storage)
|
||||
self.pref.parse_dict({"locale": "en"})
|
||||
|
||||
def test_plugin_store_init(self):
|
||||
self.assertEqual(1, len(self.store.plugins))
|
||||
self.assertEqual(1, len(self.storage))
|
||||
|
||||
@parameterized.expand(
|
||||
[
|
||||
('md5 test', 'md5 hash digest: 098f6bcd4621d373cade4e832627b4f6'),
|
||||
('sha1 test', 'sha1 hash digest: a94a8fe5ccb19ba61c4c0873d391e987982fbbd3'),
|
||||
('sha224 test', 'sha224 hash digest: 90a3ed9e32b2aaf4c61c410eb925426119e1a9dc53d4286ade99a809'),
|
||||
('sha256 test', 'sha256 hash digest: 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08'),
|
||||
(
|
||||
'sha384 test',
|
||||
'sha384 hash digest: 768412320f7b0aa5812fce428dc4706b3c'
|
||||
'ae50e02a64caa16a782249bfe8efc4b7ef1ccb126255d196047dfedf1'
|
||||
'7a0a9',
|
||||
),
|
||||
(
|
||||
'sha512 test',
|
||||
'sha512 hash digest: ee26b0dd4af7e749aa1a8ee3c10ae9923f6'
|
||||
'18980772e473f8819a5d4940e0db27ac185f8a0e1d5f84f88bc887fd67b143732c304cc5'
|
||||
'fa9ad8e6f57f50028a8ff',
|
||||
),
|
||||
]
|
||||
)
|
||||
def test_hash_digest_new(self, query: str, hash_str: str):
|
||||
request = Mock(remote_addr='127.0.0.1')
|
||||
search = get_search_mock(query=query, pageno=1)
|
||||
self.store.call(self.store.plugins, 'post_search', request, search)
|
||||
self.assertIn(hash_str, search.result_container.answers['hash']['answer'])
|
||||
@parameterized.expand(query_res)
|
||||
def test_hash_digest_new(self, query: str, res: str):
|
||||
with self.app.test_request_context():
|
||||
sxng_request.preferences = self.pref
|
||||
answer = Answer(results=[], answer=res)
|
||||
|
||||
def test_md5_bytes_no_answer(self):
|
||||
request = Mock(remote_addr='127.0.0.1')
|
||||
search = get_search_mock(query=b'md5 test', pageno=2)
|
||||
self.store.call(self.store.plugins, 'post_search', request, search)
|
||||
self.assertNotIn('hash', search.result_container.answers)
|
||||
search = do_post_search(query, self.storage)
|
||||
self.assertIn(answer, search.result_container.answers)
|
||||
|
||||
def test_pageno_1_2(self):
|
||||
with self.app.test_request_context():
|
||||
sxng_request.preferences = self.pref
|
||||
query, res = query_res[0]
|
||||
answer = Answer(results=[], answer=res)
|
||||
|
||||
search = do_post_search(query, self.storage, pageno=1)
|
||||
self.assertIn(answer, search.result_container.answers)
|
||||
|
||||
search = do_post_search(query, self.storage, pageno=2)
|
||||
self.assertEqual(list(search.result_container.answers), [])
|
||||
|
||||
@@ -1,65 +1,69 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring, invalid-name
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from mock import Mock
|
||||
from parameterized.parameterized import parameterized
|
||||
|
||||
from searx import (
|
||||
plugins,
|
||||
limiter,
|
||||
botdetection,
|
||||
)
|
||||
from flask_babel import gettext
|
||||
|
||||
import searx.plugins
|
||||
import searx.preferences
|
||||
import searx.limiter
|
||||
import searx.botdetection
|
||||
|
||||
from searx.extended_types import sxng_request
|
||||
from searx.result_types import Answer
|
||||
|
||||
from tests import SearxTestCase
|
||||
from .test_plugins import get_search_mock
|
||||
from .test_plugins import do_post_search
|
||||
|
||||
|
||||
class PluginIPSelfInfo(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class PluginIPSelfInfo(SearxTestCase):
|
||||
|
||||
def setUp(self):
|
||||
plugin = plugins.load_and_initialize_plugin('searx.plugins.self_info', False, (None, {}))
|
||||
self.store = plugins.PluginStore()
|
||||
self.store.register(plugin)
|
||||
cfg = limiter.get_cfg()
|
||||
botdetection.init(cfg, None)
|
||||
super().setUp()
|
||||
|
||||
self.storage = searx.plugins.PluginStorage()
|
||||
self.storage.register_by_fqn("searx.plugins.self_info.SXNGPlugin")
|
||||
self.storage.init(self.app)
|
||||
self.pref = searx.preferences.Preferences(["simple"], ["general"], {}, self.storage)
|
||||
self.pref.parse_dict({"locale": "en"})
|
||||
cfg = searx.limiter.get_cfg()
|
||||
searx.botdetection.init(cfg, None)
|
||||
|
||||
def test_plugin_store_init(self):
|
||||
self.assertEqual(1, len(self.store.plugins))
|
||||
self.assertEqual(1, len(self.storage))
|
||||
|
||||
def test_ip_in_answer(self):
|
||||
request = Mock()
|
||||
request.remote_addr = '127.0.0.1'
|
||||
request.headers = {'X-Forwarded-For': '1.2.3.4, 127.0.0.1', 'X-Real-IP': '127.0.0.1'}
|
||||
search = get_search_mock(query='ip', pageno=1)
|
||||
self.store.call(self.store.plugins, 'post_search', request, search)
|
||||
self.assertIn('127.0.0.1', search.result_container.answers["ip"]["answer"])
|
||||
def test_pageno_1_2(self):
|
||||
|
||||
def test_ip_not_in_answer(self):
|
||||
request = Mock()
|
||||
request.remote_addr = '127.0.0.1'
|
||||
request.headers = {'X-Forwarded-For': '1.2.3.4, 127.0.0.1', 'X-Real-IP': '127.0.0.1'}
|
||||
search = get_search_mock(query='ip', pageno=2)
|
||||
self.store.call(self.store.plugins, 'post_search', request, search)
|
||||
self.assertNotIn('ip', search.result_container.answers)
|
||||
with self.app.test_request_context():
|
||||
sxng_request.preferences = self.pref
|
||||
sxng_request.remote_addr = "127.0.0.1"
|
||||
sxng_request.headers = {"X-Forwarded-For": "1.2.3.4, 127.0.0.1", "X-Real-IP": "127.0.0.1"} # type: ignore
|
||||
answer = Answer(results=[], answer=gettext("Your IP is: ") + "127.0.0.1")
|
||||
|
||||
search = do_post_search("ip", self.storage, pageno=1)
|
||||
self.assertIn(answer, search.result_container.answers)
|
||||
|
||||
search = do_post_search("ip", self.storage, pageno=2)
|
||||
self.assertEqual(list(search.result_container.answers), [])
|
||||
|
||||
@parameterized.expand(
|
||||
[
|
||||
'user-agent',
|
||||
'What is my User-Agent?',
|
||||
"user-agent",
|
||||
"USER-AgenT lorem ipsum",
|
||||
]
|
||||
)
|
||||
def test_user_agent_in_answer(self, query: str):
|
||||
request = Mock(user_agent=Mock(string='Mock'))
|
||||
search = get_search_mock(query=query, pageno=1)
|
||||
self.store.call(self.store.plugins, 'post_search', request, search)
|
||||
self.assertIn('Mock', search.result_container.answers["user-agent"]["answer"])
|
||||
|
||||
@parameterized.expand(
|
||||
[
|
||||
'user-agent',
|
||||
'What is my User-Agent?',
|
||||
]
|
||||
)
|
||||
def test_user_agent_not_in_answer(self, query: str):
|
||||
request = Mock(user_agent=Mock(string='Mock'))
|
||||
search = get_search_mock(query=query, pageno=2)
|
||||
self.store.call(self.store.plugins, 'post_search', request, search)
|
||||
self.assertNotIn('user-agent', search.result_container.answers)
|
||||
query = "user-agent"
|
||||
|
||||
with self.app.test_request_context():
|
||||
sxng_request.preferences = self.pref
|
||||
sxng_request.user_agent = "Dummy agent" # type: ignore
|
||||
answer = Answer(results=[], answer=gettext("Your user-agent is: ") + "Dummy agent")
|
||||
|
||||
search = do_post_search(query, self.storage, pageno=1)
|
||||
self.assertIn(answer, search.result_container.answers)
|
||||
|
||||
search = do_post_search(query, self.storage, pageno=2)
|
||||
self.assertEqual(list(search.result_container.answers), [])
|
||||
|
||||
@@ -1,50 +1,106 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
import babel
|
||||
from mock import Mock
|
||||
from searx import plugins
|
||||
|
||||
import searx.plugins
|
||||
import searx.preferences
|
||||
import searx.results
|
||||
|
||||
from searx.result_types import Result
|
||||
from searx.extended_types import sxng_request
|
||||
|
||||
from tests import SearxTestCase
|
||||
|
||||
plg_store = searx.plugins.PluginStorage()
|
||||
plg_store.load_builtins()
|
||||
|
||||
|
||||
def get_search_mock(query, **kwargs):
|
||||
|
||||
lang = kwargs.get("lang", "en-US")
|
||||
kwargs["pageno"] = kwargs.get("pageno", 1)
|
||||
kwargs["locale"] = babel.Locale.parse(lang, sep="-")
|
||||
return Mock(search_query=Mock(query=query, **kwargs), result_container=Mock(answers={}))
|
||||
user_plugins = kwargs.pop("user_plugins", [x.id for x in plg_store])
|
||||
|
||||
return Mock(
|
||||
search_query=Mock(query=query, **kwargs),
|
||||
user_plugins=user_plugins,
|
||||
result_container=searx.results.ResultContainer(),
|
||||
)
|
||||
|
||||
|
||||
class PluginMock: # pylint: disable=missing-class-docstring, too-few-public-methods
|
||||
default_on = False
|
||||
name = 'Default plugin'
|
||||
description = 'Default plugin description'
|
||||
def do_pre_search(query, storage, **kwargs) -> bool:
|
||||
|
||||
search = get_search_mock(query, **kwargs)
|
||||
ret = storage.pre_search(sxng_request, search)
|
||||
return ret
|
||||
|
||||
|
||||
class PluginStoreTest(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
def do_post_search(query, storage, **kwargs) -> Mock:
|
||||
|
||||
search = get_search_mock(query, **kwargs)
|
||||
storage.post_search(sxng_request, search)
|
||||
return search
|
||||
|
||||
|
||||
class PluginMock(searx.plugins.Plugin):
|
||||
|
||||
def __init__(self, _id: str, name: str, default_on: bool):
|
||||
self.id = _id
|
||||
self.default_on = default_on
|
||||
self._name = name
|
||||
super().__init__()
|
||||
|
||||
# pylint: disable= unused-argument
|
||||
def pre_search(self, request, search) -> bool:
|
||||
return True
|
||||
|
||||
def post_search(self, request, search) -> None:
|
||||
return None
|
||||
|
||||
def on_result(self, request, search, result) -> bool:
|
||||
return False
|
||||
|
||||
def info(self):
|
||||
return searx.plugins.PluginInfo(
|
||||
id=self.id,
|
||||
name=self._name,
|
||||
description=f"Dummy plugin: {self.id}",
|
||||
preference_section="general",
|
||||
)
|
||||
|
||||
|
||||
class PluginStorage(SearxTestCase):
|
||||
|
||||
def setUp(self):
|
||||
self.store = plugins.PluginStore()
|
||||
super().setUp()
|
||||
engines = {}
|
||||
|
||||
self.storage = searx.plugins.PluginStorage()
|
||||
self.storage.register(PluginMock("plg001", "first plugin", True))
|
||||
self.storage.register(PluginMock("plg002", "second plugin", True))
|
||||
self.storage.init(self.app)
|
||||
self.pref = searx.preferences.Preferences(["simple"], ["general"], engines, self.storage)
|
||||
self.pref.parse_dict({"locale": "en"})
|
||||
|
||||
def test_init(self):
|
||||
self.assertEqual(0, len(self.store.plugins))
|
||||
self.assertIsInstance(self.store.plugins, list)
|
||||
|
||||
def test_register(self):
|
||||
testplugin = PluginMock()
|
||||
self.store.register(testplugin)
|
||||
self.assertEqual(1, len(self.store.plugins))
|
||||
self.assertEqual(2, len(self.storage))
|
||||
|
||||
def test_call_empty(self):
|
||||
testplugin = PluginMock()
|
||||
self.store.register(testplugin)
|
||||
setattr(testplugin, 'asdf', Mock())
|
||||
request = Mock()
|
||||
self.store.call([], 'asdf', request, Mock())
|
||||
self.assertFalse(getattr(testplugin, 'asdf').called) # pylint: disable=E1101
|
||||
def test_hooks(self):
|
||||
|
||||
def test_call_with_plugin(self):
|
||||
store = plugins.PluginStore()
|
||||
testplugin = PluginMock()
|
||||
store.register(testplugin)
|
||||
setattr(testplugin, 'asdf', Mock())
|
||||
request = Mock()
|
||||
store.call([testplugin], 'asdf', request, Mock())
|
||||
self.assertTrue(getattr(testplugin, 'asdf').called) # pylint: disable=E1101
|
||||
with self.app.test_request_context():
|
||||
sxng_request.preferences = self.pref
|
||||
query = ""
|
||||
|
||||
ret = do_pre_search(query, self.storage, pageno=1)
|
||||
self.assertTrue(ret is True)
|
||||
|
||||
ret = self.storage.on_result(
|
||||
sxng_request,
|
||||
get_search_mock("lorem ipsum", user_plugins=["plg001", "plg002"]),
|
||||
Result(results=[]),
|
||||
)
|
||||
self.assertFalse(ret)
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring, invalid-name
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
import flask
|
||||
from mock import Mock
|
||||
from tests import SearxTestCase
|
||||
|
||||
from searx import favicons
|
||||
from searx.locales import locales_initialize
|
||||
from searx.preferences import (
|
||||
@@ -15,20 +15,19 @@ from searx.preferences import (
|
||||
PluginsSetting,
|
||||
ValidationException,
|
||||
)
|
||||
from searx.plugins import Plugin
|
||||
import searx.plugins
|
||||
from searx.preferences import Preferences
|
||||
|
||||
from tests import SearxTestCase
|
||||
from .test_plugins import PluginMock
|
||||
|
||||
|
||||
locales_initialize()
|
||||
favicons.init()
|
||||
|
||||
|
||||
class PluginStub(Plugin): # pylint: disable=missing-class-docstring, too-few-public-methods
|
||||
def __init__(self, plugin_id, default_on):
|
||||
self.id = plugin_id
|
||||
self.default_on = default_on
|
||||
class TestSettings(SearxTestCase):
|
||||
|
||||
|
||||
class TestSettings(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
# map settings
|
||||
|
||||
def test_map_setting_invalid_default_value(self):
|
||||
@@ -93,6 +92,7 @@ class TestSettings(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
self.assertEqual(setting.get_value(), ['2'])
|
||||
|
||||
# search language settings
|
||||
|
||||
def test_lang_setting_valid_choice(self):
|
||||
setting = SearchLanguageSetting('all', choices=['all', 'de', 'en'])
|
||||
setting.parse('de')
|
||||
@@ -114,23 +114,30 @@ class TestSettings(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
self.assertEqual(setting.get_value(), 'es-ES')
|
||||
|
||||
# plugins settings
|
||||
|
||||
def test_plugins_setting_all_default_enabled(self):
|
||||
plugin1 = PluginStub('plugin1', True)
|
||||
plugin2 = PluginStub('plugin2', True)
|
||||
setting = PluginsSetting(['3'], plugins=[plugin1, plugin2])
|
||||
self.assertEqual(set(setting.get_enabled()), set(['plugin1', 'plugin2']))
|
||||
storage = searx.plugins.PluginStorage()
|
||||
storage.register(PluginMock("plg001", "first plugin", True))
|
||||
storage.register(PluginMock("plg002", "second plugin", True))
|
||||
plgs_settings = PluginsSetting(False, storage)
|
||||
self.assertEqual(set(plgs_settings.get_enabled()), {"plg001", "plg002"})
|
||||
|
||||
def test_plugins_setting_few_default_enabled(self):
|
||||
plugin1 = PluginStub('plugin1', True)
|
||||
plugin2 = PluginStub('plugin2', False)
|
||||
plugin3 = PluginStub('plugin3', True)
|
||||
setting = PluginsSetting('name', plugins=[plugin1, plugin2, plugin3])
|
||||
self.assertEqual(set(setting.get_enabled()), set(['plugin1', 'plugin3']))
|
||||
storage = searx.plugins.PluginStorage()
|
||||
storage.register(PluginMock("plg001", "first plugin", True))
|
||||
storage.register(PluginMock("plg002", "second plugin", False))
|
||||
storage.register(PluginMock("plg003", "third plugin", True))
|
||||
plgs_settings = PluginsSetting(False, storage)
|
||||
self.assertEqual(set(plgs_settings.get_enabled()), set(['plg001', 'plg003']))
|
||||
|
||||
|
||||
class TestPreferences(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestPreferences(SearxTestCase):
|
||||
|
||||
def setUp(self):
|
||||
self.preferences = Preferences(['simple'], ['general'], {}, [])
|
||||
super().setUp()
|
||||
|
||||
storage = searx.plugins.PluginStorage()
|
||||
self.preferences = Preferences(['simple'], ['general'], {}, storage)
|
||||
|
||||
def test_encode(self):
|
||||
url_params = (
|
||||
|
||||
@@ -1,25 +1,13 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from parameterized.parameterized import parameterized
|
||||
import searx.search
|
||||
from searx.query import RawTextQuery
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
TEST_ENGINES = [
|
||||
{
|
||||
'name': 'dummy engine',
|
||||
'engine': 'dummy',
|
||||
'categories': 'general',
|
||||
'shortcut': 'du',
|
||||
'timeout': 3.0,
|
||||
'tokens': [],
|
||||
},
|
||||
]
|
||||
class TestQuery(SearxTestCase):
|
||||
|
||||
|
||||
class TestQuery(SearxTestCase): # pylint:disable=missing-class-docstring
|
||||
def test_simple_query(self):
|
||||
query_text = 'the query'
|
||||
query = RawTextQuery(query_text, [])
|
||||
@@ -59,7 +47,8 @@ class TestQuery(SearxTestCase): # pylint:disable=missing-class-docstring
|
||||
self.assertEqual(query.getFullQuery(), '<8 another text')
|
||||
|
||||
|
||||
class TestLanguageParser(SearxTestCase): # pylint:disable=missing-class-docstring
|
||||
class TestLanguageParser(SearxTestCase):
|
||||
|
||||
def test_language_code(self):
|
||||
language = 'es-ES'
|
||||
query_text = 'the query'
|
||||
@@ -143,7 +132,8 @@ class TestLanguageParser(SearxTestCase): # pylint:disable=missing-class-docstri
|
||||
self.assertEqual(query.autocomplete_list, autocomplete_list)
|
||||
|
||||
|
||||
class TestTimeoutParser(SearxTestCase): # pylint:disable=missing-class-docstring
|
||||
class TestTimeoutParser(SearxTestCase):
|
||||
|
||||
@parameterized.expand(
|
||||
[
|
||||
('<3 the query', 3),
|
||||
@@ -182,7 +172,8 @@ class TestTimeoutParser(SearxTestCase): # pylint:disable=missing-class-docstrin
|
||||
self.assertEqual(query.autocomplete_list, ['<3', '<850'])
|
||||
|
||||
|
||||
class TestExternalBangParser(SearxTestCase): # pylint:disable=missing-class-docstring
|
||||
class TestExternalBangParser(SearxTestCase):
|
||||
|
||||
def test_external_bang(self):
|
||||
query_text = '!!ddg the query'
|
||||
query = RawTextQuery(query_text, [])
|
||||
@@ -212,17 +203,11 @@ class TestExternalBangParser(SearxTestCase): # pylint:disable=missing-class-doc
|
||||
self.assertEqual(query.get_autocomplete_full_query(a), a + ' the query')
|
||||
|
||||
|
||||
class TestBang(SearxTestCase): # pylint:disable=missing-class-docstring
|
||||
class TestBang(SearxTestCase):
|
||||
|
||||
SPECIFIC_BANGS = ['!dummy_engine', '!du', '!general']
|
||||
SPECIFIC_BANGS = ['!dummy_engine', '!gd', '!general']
|
||||
THE_QUERY = 'the query'
|
||||
|
||||
def setUp(self):
|
||||
searx.search.initialize(TEST_ENGINES)
|
||||
|
||||
def tearDown(self):
|
||||
searx.search.load_engines([])
|
||||
|
||||
@parameterized.expand(SPECIFIC_BANGS)
|
||||
def test_bang(self, bang: str):
|
||||
with self.subTest(msg="Check bang", bang=bang):
|
||||
@@ -246,7 +231,7 @@ class TestBang(SearxTestCase): # pylint:disable=missing-class-docstring
|
||||
|
||||
def test_bang_autocomplete(self):
|
||||
query = RawTextQuery('the query !dum', [])
|
||||
self.assertEqual(query.autocomplete_list, ['!dummy_engine'])
|
||||
self.assertEqual(query.autocomplete_list, ['!dummy_engine', '!dummy_private_engine'])
|
||||
|
||||
query = RawTextQuery('!dum the query', [])
|
||||
self.assertEqual(query.autocomplete_list, [])
|
||||
|
||||
@@ -1,75 +1,56 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
|
||||
from searx.result_types import LegacyResult
|
||||
from searx.results import ResultContainer
|
||||
import searx.search
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
def make_test_engine_dict(**kwargs) -> dict:
|
||||
test_engine = {
|
||||
# fmt: off
|
||||
'name': None,
|
||||
'engine': None,
|
||||
'categories': 'general',
|
||||
'shortcut': 'dummy',
|
||||
'timeout': 3.0,
|
||||
'tokens': [],
|
||||
# fmt: on
|
||||
}
|
||||
class ResultContainerTestCase(SearxTestCase):
|
||||
# pylint: disable=use-dict-literal
|
||||
|
||||
test_engine.update(**kwargs)
|
||||
return test_engine
|
||||
|
||||
|
||||
def fake_result(url='https://aa.bb/cc?dd=ee#ff', title='aaa', content='bbb', engine='wikipedia', **kwargs):
|
||||
result = {
|
||||
# fmt: off
|
||||
'url': url,
|
||||
'title': title,
|
||||
'content': content,
|
||||
'engine': engine,
|
||||
# fmt: on
|
||||
}
|
||||
result.update(kwargs)
|
||||
return result
|
||||
|
||||
|
||||
class ResultContainerTestCase(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
|
||||
def setUp(self) -> None:
|
||||
stract_engine = make_test_engine_dict(name="stract", engine="stract", shortcut="stra")
|
||||
duckduckgo_engine = make_test_engine_dict(name="duckduckgo", engine="duckduckgo", shortcut="ddg")
|
||||
mojeek_engine = make_test_engine_dict(name="mojeek", engine="mojeek", shortcut="mjk")
|
||||
searx.search.initialize([stract_engine, duckduckgo_engine, mojeek_engine])
|
||||
self.container = ResultContainer()
|
||||
|
||||
def tearDown(self):
|
||||
searx.search.load_engines([])
|
||||
TEST_SETTINGS = "test_result_container.yml"
|
||||
|
||||
def test_empty(self):
|
||||
self.assertEqual(self.container.get_ordered_results(), [])
|
||||
container = ResultContainer()
|
||||
self.assertEqual(container.get_ordered_results(), [])
|
||||
|
||||
def test_one_result(self):
|
||||
self.container.extend('wikipedia', [fake_result()])
|
||||
result = dict(url="https://example.org", title="title ..", content="Lorem ..")
|
||||
|
||||
self.assertEqual(self.container.results_length(), 1)
|
||||
container = ResultContainer()
|
||||
container.extend("google", [result])
|
||||
container.close()
|
||||
|
||||
self.assertEqual(container.results_length(), 1)
|
||||
self.assertIn(LegacyResult(result), container.get_ordered_results())
|
||||
|
||||
def test_one_suggestion(self):
|
||||
self.container.extend('wikipedia', [fake_result(suggestion=True)])
|
||||
result = dict(suggestion="lorem ipsum ..")
|
||||
|
||||
self.assertEqual(len(self.container.suggestions), 1)
|
||||
self.assertEqual(self.container.results_length(), 0)
|
||||
container = ResultContainer()
|
||||
container.extend("duckduckgo", [result])
|
||||
container.close()
|
||||
|
||||
def test_result_merge(self):
|
||||
self.container.extend('wikipedia', [fake_result()])
|
||||
self.container.extend('wikidata', [fake_result(), fake_result(url='https://example.com/')])
|
||||
self.assertEqual(container.results_length(), 0)
|
||||
self.assertEqual(len(container.suggestions), 1)
|
||||
self.assertIn(result["suggestion"], container.suggestions)
|
||||
|
||||
self.assertEqual(self.container.results_length(), 2)
|
||||
def test_merge_url_result(self):
|
||||
# from the merge of eng1 and eng2 we expect this result
|
||||
result = LegacyResult(
|
||||
url="https://example.org", title="very long title, lorem ipsum", content="Lorem ipsum dolor sit amet .."
|
||||
)
|
||||
eng1 = dict(url=result.url, title="short title", content=result.content, engine="google")
|
||||
eng2 = dict(url="http://example.org", title=result.title, content="lorem ipsum", engine="duckduckgo")
|
||||
|
||||
def test_result_merge_by_title(self):
|
||||
self.container.extend('stract', [fake_result(engine='stract', title='short title')])
|
||||
self.container.extend('duckduckgo', [fake_result(engine='duckduckgo', title='normal title')])
|
||||
self.container.extend('mojeek', [fake_result(engine='mojeek', title='this long long title')])
|
||||
container = ResultContainer()
|
||||
container.extend(None, [eng1, eng2])
|
||||
container.close()
|
||||
|
||||
self.assertEqual(self.container.get_ordered_results()[0].get('title', ''), 'this long long title')
|
||||
result_list = container.get_ordered_results()
|
||||
self.assertEqual(container.results_length(), 1)
|
||||
self.assertIn(result, result_list)
|
||||
self.assertEqual(result_list[0].title, result.title)
|
||||
self.assertEqual(result_list[0].content, result.content)
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring, invalid-name
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from copy import copy
|
||||
import logging
|
||||
|
||||
import searx.search
|
||||
from searx.search import SearchQuery, EngineRef
|
||||
@@ -12,20 +11,11 @@ from tests import SearxTestCase
|
||||
|
||||
SAFESEARCH = 0
|
||||
PAGENO = 1
|
||||
PUBLIC_ENGINE_NAME = 'general dummy'
|
||||
TEST_ENGINES = [
|
||||
{
|
||||
'name': PUBLIC_ENGINE_NAME,
|
||||
'engine': 'dummy',
|
||||
'categories': 'general',
|
||||
'shortcut': 'gd',
|
||||
'timeout': 3.0,
|
||||
'tokens': [],
|
||||
},
|
||||
]
|
||||
PUBLIC_ENGINE_NAME = "dummy engine" # from the ./settings/test_settings.yml
|
||||
|
||||
|
||||
class SearchQueryTestCase(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class SearchQueryTestCase(SearxTestCase):
|
||||
|
||||
def test_repr(self):
|
||||
s = SearchQuery('test', [EngineRef('bing', 'general')], 'all', 0, 1, '1', 5.0, 'g')
|
||||
self.assertEqual(
|
||||
@@ -44,21 +34,7 @@ class SearchQueryTestCase(SearxTestCase): # pylint: disable=missing-class-docst
|
||||
self.assertEqual(s, t)
|
||||
|
||||
|
||||
class SearchTestCase(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
def setUp(self):
|
||||
|
||||
log = logging.getLogger("searx")
|
||||
log_lev = log.level
|
||||
log.setLevel(logging.ERROR)
|
||||
from searx import webapp # pylint: disable=import-outside-toplevel
|
||||
|
||||
log.setLevel(log_lev)
|
||||
|
||||
self.app = webapp.app
|
||||
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
searx.search.initialize(TEST_ENGINES)
|
||||
class SearchTestCase(SearxTestCase):
|
||||
|
||||
def test_timeout_simple(self):
|
||||
settings['outgoing']['max_request_timeout'] = None
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
@@ -17,7 +17,8 @@ def _settings(f_name):
|
||||
return str(Path(__file__).parent.absolute() / "settings" / f_name)
|
||||
|
||||
|
||||
class TestLoad(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestLoad(SearxTestCase):
|
||||
|
||||
def test_load_zero(self):
|
||||
with self.assertRaises(SearxSettingsException):
|
||||
settings_loader.load_yaml('/dev/zero')
|
||||
@@ -28,7 +29,8 @@ class TestLoad(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
self.assertEqual(settings_loader.load_yaml(_settings("empty_settings.yml")), {})
|
||||
|
||||
|
||||
class TestDefaultSettings(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestDefaultSettings(SearxTestCase):
|
||||
|
||||
def test_load(self):
|
||||
settings, msg = settings_loader.load_settings(load_user_settings=False)
|
||||
self.assertTrue(msg.startswith('load the default settings from'))
|
||||
@@ -42,7 +44,8 @@ class TestDefaultSettings(SearxTestCase): # pylint: disable=missing-class-docst
|
||||
self.assertIsInstance(settings['default_doi_resolver'], str)
|
||||
|
||||
|
||||
class TestUserSettings(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestUserSettings(SearxTestCase):
|
||||
|
||||
def test_is_use_default_settings(self):
|
||||
self.assertFalse(settings_loader.is_use_default_settings({}))
|
||||
self.assertTrue(settings_loader.is_use_default_settings({'use_default_settings': True}))
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
from tests import SearxTestCase
|
||||
from searx import compat
|
||||
from searx.favicons.config import DEFAULT_CFG_TOML_PATH
|
||||
|
||||
|
||||
class CompatTest(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class CompatTest(SearxTestCase):
|
||||
|
||||
def test_toml(self):
|
||||
with DEFAULT_CFG_TOML_PATH.open("rb") as f:
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring, invalid-name
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
import random
|
||||
import string
|
||||
@@ -16,7 +16,8 @@ def random_string(length, choices=string.ascii_letters):
|
||||
return ''.join(random.choice(choices) for _ in range(length))
|
||||
|
||||
|
||||
class TestUtils(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestUtils(SearxTestCase):
|
||||
|
||||
def test_gen_useragent(self):
|
||||
self.assertIsInstance(utils.gen_useragent(), str)
|
||||
self.assertIsNotNone(utils.gen_useragent())
|
||||
@@ -109,7 +110,10 @@ class TestUtils(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
|
||||
|
||||
class TestHTMLTextExtractor(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
|
||||
self.html_text_extractor = utils._HTMLTextExtractor() # pylint: disable=protected-access
|
||||
|
||||
def test__init__(self):
|
||||
|
||||
@@ -1,52 +1,38 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
import searx.plugins
|
||||
|
||||
from searx.preferences import Preferences
|
||||
from searx.engines import engines
|
||||
|
||||
import searx.search
|
||||
from searx.preferences import Preferences
|
||||
from searx.search import EngineRef
|
||||
from searx.webadapter import validate_engineref_list
|
||||
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
PRIVATE_ENGINE_NAME = 'general private offline'
|
||||
TEST_ENGINES = [
|
||||
{
|
||||
'name': PRIVATE_ENGINE_NAME,
|
||||
'engine': 'dummy-offline',
|
||||
'categories': 'general',
|
||||
'shortcut': 'do',
|
||||
'timeout': 3.0,
|
||||
'engine_type': 'offline',
|
||||
'tokens': ['my-token'],
|
||||
},
|
||||
]
|
||||
SEARCHQUERY = [EngineRef(PRIVATE_ENGINE_NAME, 'general')]
|
||||
PRIVATE_ENGINE_NAME = "dummy private engine" # from the ./settings/test_settings.yml
|
||||
SEARCHQUERY = [EngineRef(PRIVATE_ENGINE_NAME, "general")]
|
||||
|
||||
|
||||
class ValidateQueryCase(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
searx.search.initialize(TEST_ENGINES)
|
||||
class ValidateQueryCase(SearxTestCase):
|
||||
|
||||
def test_query_private_engine_without_token(self): # pylint:disable=invalid-name
|
||||
preferences = Preferences(['simple'], ['general'], engines, [])
|
||||
def test_without_token(self):
|
||||
preferences = Preferences(['simple'], ['general'], engines, searx.plugins.STORAGE)
|
||||
valid, unknown, invalid_token = validate_engineref_list(SEARCHQUERY, preferences)
|
||||
self.assertEqual(len(valid), 0)
|
||||
self.assertEqual(len(unknown), 0)
|
||||
self.assertEqual(len(invalid_token), 1)
|
||||
|
||||
def test_query_private_engine_with_incorrect_token(self): # pylint:disable=invalid-name
|
||||
preferences_with_tokens = Preferences(['simple'], ['general'], engines, [])
|
||||
def test_with_incorrect_token(self):
|
||||
preferences_with_tokens = Preferences(['simple'], ['general'], engines, searx.plugins.STORAGE)
|
||||
preferences_with_tokens.parse_dict({'tokens': 'bad-token'})
|
||||
valid, unknown, invalid_token = validate_engineref_list(SEARCHQUERY, preferences_with_tokens)
|
||||
self.assertEqual(len(valid), 0)
|
||||
self.assertEqual(len(unknown), 0)
|
||||
self.assertEqual(len(invalid_token), 1)
|
||||
|
||||
def test_query_private_engine_with_correct_token(self): # pylint:disable=invalid-name
|
||||
preferences_with_tokens = Preferences(['simple'], ['general'], engines, [])
|
||||
def test_with_correct_token(self):
|
||||
preferences_with_tokens = Preferences(['simple'], ['general'], engines, searx.plugins.STORAGE)
|
||||
preferences_with_tokens.parse_dict({'tokens': 'my-token'})
|
||||
valid, unknown, invalid_token = validate_engineref_list(SEARCHQUERY, preferences_with_tokens)
|
||||
self.assertEqual(len(valid), 1)
|
||||
|
||||
@@ -1,41 +1,33 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
import logging
|
||||
import json
|
||||
from urllib.parse import ParseResult
|
||||
import babel
|
||||
from mock import Mock
|
||||
from searx.results import Timing
|
||||
|
||||
import searx.webapp
|
||||
import searx.search
|
||||
import searx.search.processors
|
||||
from searx.search import Search
|
||||
|
||||
from searx.results import Timing
|
||||
from searx.preferences import Preferences
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
class ViewsTestCase(SearxTestCase): # pylint: disable=missing-class-docstring, too-many-public-methods
|
||||
class ViewsTestCase(SearxTestCase): # pylint: disable=too-many-public-methods
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
|
||||
# skip init function (no external HTTP request)
|
||||
def dummy(*args, **kwargs): # pylint: disable=unused-argument
|
||||
pass
|
||||
|
||||
self.setattr4test(searx.search.processors, 'initialize_processor', dummy)
|
||||
|
||||
log = logging.getLogger("searx")
|
||||
log_lev = log.level
|
||||
log.setLevel(logging.ERROR)
|
||||
from searx import webapp # pylint: disable=import-outside-toplevel
|
||||
|
||||
log.setLevel(log_lev)
|
||||
|
||||
webapp.app.config['TESTING'] = True # to get better error messages
|
||||
self.app = webapp.app.test_client()
|
||||
|
||||
# remove sha for the static file
|
||||
# so the tests don't have to care about the changing URLs
|
||||
for k in webapp.static_files:
|
||||
webapp.static_files[k] = None
|
||||
# remove sha for the static file so the tests don't have to care about
|
||||
# the changing URLs
|
||||
self.setattr4test(searx.webapp, 'static_files', {})
|
||||
|
||||
# set some defaults
|
||||
test_results = [
|
||||
@@ -85,7 +77,7 @@ class ViewsTestCase(SearxTestCase): # pylint: disable=missing-class-docstring,
|
||||
)
|
||||
search_self.search_query.locale = babel.Locale.parse("en-US", sep='-')
|
||||
|
||||
self.setattr4test(Search, 'search', search_mock)
|
||||
self.setattr4test(searx.search.Search, 'search', search_mock)
|
||||
|
||||
original_preferences_get_value = Preferences.get_value
|
||||
|
||||
@@ -100,7 +92,7 @@ class ViewsTestCase(SearxTestCase): # pylint: disable=missing-class-docstring,
|
||||
self.maxDiff = None # pylint: disable=invalid-name
|
||||
|
||||
def test_index_empty(self):
|
||||
result = self.app.post('/')
|
||||
result = self.client.post('/')
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(
|
||||
b'<div class="title"><h1>SearXNG</h1></div>',
|
||||
@@ -108,34 +100,34 @@ class ViewsTestCase(SearxTestCase): # pylint: disable=missing-class-docstring,
|
||||
)
|
||||
|
||||
def test_index_html_post(self):
|
||||
result = self.app.post('/', data={'q': 'test'})
|
||||
result = self.client.post('/', data={'q': 'test'})
|
||||
self.assertEqual(result.status_code, 308)
|
||||
self.assertEqual(result.location, '/search')
|
||||
|
||||
def test_index_html_get(self):
|
||||
result = self.app.post('/?q=test')
|
||||
result = self.client.post('/?q=test')
|
||||
self.assertEqual(result.status_code, 308)
|
||||
self.assertEqual(result.location, '/search?q=test')
|
||||
|
||||
def test_search_empty_html(self):
|
||||
result = self.app.post('/search', data={'q': ''})
|
||||
result = self.client.post('/search', data={'q': ''})
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(b'<div class="title"><h1>SearXNG</h1></div>', result.data)
|
||||
|
||||
def test_search_empty_json(self):
|
||||
result = self.app.post('/search', data={'q': '', 'format': 'json'})
|
||||
result = self.client.post('/search', data={'q': '', 'format': 'json'})
|
||||
self.assertEqual(result.status_code, 400)
|
||||
|
||||
def test_search_empty_csv(self):
|
||||
result = self.app.post('/search', data={'q': '', 'format': 'csv'})
|
||||
result = self.client.post('/search', data={'q': '', 'format': 'csv'})
|
||||
self.assertEqual(result.status_code, 400)
|
||||
|
||||
def test_search_empty_rss(self):
|
||||
result = self.app.post('/search', data={'q': '', 'format': 'rss'})
|
||||
result = self.client.post('/search', data={'q': '', 'format': 'rss'})
|
||||
self.assertEqual(result.status_code, 400)
|
||||
|
||||
def test_search_html(self):
|
||||
result = self.app.post('/search', data={'q': 'test'})
|
||||
result = self.client.post('/search', data={'q': 'test'})
|
||||
|
||||
self.assertIn(
|
||||
b'<span class="url_o1"><span class="url_i1">http://second.test.xyz</span></span>',
|
||||
@@ -147,11 +139,11 @@ class ViewsTestCase(SearxTestCase): # pylint: disable=missing-class-docstring,
|
||||
)
|
||||
|
||||
def test_index_json(self):
|
||||
result = self.app.post('/', data={'q': 'test', 'format': 'json'})
|
||||
result = self.client.post('/', data={'q': 'test', 'format': 'json'})
|
||||
self.assertEqual(result.status_code, 308)
|
||||
|
||||
def test_search_json(self):
|
||||
result = self.app.post('/search', data={'q': 'test', 'format': 'json'})
|
||||
result = self.client.post('/search', data={'q': 'test', 'format': 'json'})
|
||||
result_dict = json.loads(result.data.decode())
|
||||
|
||||
self.assertEqual('test', result_dict['query'])
|
||||
@@ -160,11 +152,11 @@ class ViewsTestCase(SearxTestCase): # pylint: disable=missing-class-docstring,
|
||||
self.assertEqual(result_dict['results'][0]['url'], 'http://first.test.xyz')
|
||||
|
||||
def test_index_csv(self):
|
||||
result = self.app.post('/', data={'q': 'test', 'format': 'csv'})
|
||||
result = self.client.post('/', data={'q': 'test', 'format': 'csv'})
|
||||
self.assertEqual(result.status_code, 308)
|
||||
|
||||
def test_search_csv(self):
|
||||
result = self.app.post('/search', data={'q': 'test', 'format': 'csv'})
|
||||
result = self.client.post('/search', data={'q': 'test', 'format': 'csv'})
|
||||
|
||||
self.assertEqual(
|
||||
b'title,url,content,host,engine,score,type\r\n'
|
||||
@@ -174,11 +166,11 @@ class ViewsTestCase(SearxTestCase): # pylint: disable=missing-class-docstring,
|
||||
)
|
||||
|
||||
def test_index_rss(self):
|
||||
result = self.app.post('/', data={'q': 'test', 'format': 'rss'})
|
||||
result = self.client.post('/', data={'q': 'test', 'format': 'rss'})
|
||||
self.assertEqual(result.status_code, 308)
|
||||
|
||||
def test_search_rss(self):
|
||||
result = self.app.post('/search', data={'q': 'test', 'format': 'rss'})
|
||||
result = self.client.post('/search', data={'q': 'test', 'format': 'rss'})
|
||||
|
||||
self.assertIn(b'<description>Search results for "test" - SearXNG</description>', result.data)
|
||||
|
||||
@@ -191,28 +183,28 @@ class ViewsTestCase(SearxTestCase): # pylint: disable=missing-class-docstring,
|
||||
self.assertIn(b'<description>first test content</description>', result.data)
|
||||
|
||||
def test_redirect_about(self):
|
||||
result = self.app.get('/about')
|
||||
result = self.client.get('/about')
|
||||
self.assertEqual(result.status_code, 302)
|
||||
|
||||
def test_info_page(self):
|
||||
result = self.app.get('/info/en/search-syntax')
|
||||
result = self.client.get('/info/en/search-syntax')
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(b'<h1>Search syntax</h1>', result.data)
|
||||
|
||||
def test_health(self):
|
||||
result = self.app.get('/healthz')
|
||||
result = self.client.get('/healthz')
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(b'OK', result.data)
|
||||
|
||||
def test_preferences(self):
|
||||
result = self.app.get('/preferences')
|
||||
result = self.client.get('/preferences')
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(b'<form id="search_form" method="post" action="/preferences"', result.data)
|
||||
self.assertIn(b'<div id="categories_container">', result.data)
|
||||
self.assertIn(b'<legend id="pref_ui_locale">Interface language</legend>', result.data)
|
||||
|
||||
def test_browser_locale(self):
|
||||
result = self.app.get('/preferences', headers={'Accept-Language': 'zh-tw;q=0.8'})
|
||||
result = self.client.get('/preferences', headers={'Accept-Language': 'zh-tw;q=0.8'})
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(
|
||||
b'<option value="zh-Hant-TW" selected="selected">',
|
||||
@@ -226,42 +218,43 @@ class ViewsTestCase(SearxTestCase): # pylint: disable=missing-class-docstring,
|
||||
)
|
||||
|
||||
def test_browser_empty_locale(self):
|
||||
result = self.app.get('/preferences', headers={'Accept-Language': ''})
|
||||
result = self.client.get('/preferences', headers={'Accept-Language': ''})
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(
|
||||
b'<option value="en" selected="selected">', result.data, 'Interface locale ignored browser preference.'
|
||||
)
|
||||
|
||||
def test_locale_occitan(self):
|
||||
result = self.app.get('/preferences?locale=oc')
|
||||
result = self.client.get('/preferences?locale=oc')
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(
|
||||
b'<option value="oc" selected="selected">', result.data, 'Interface locale ignored browser preference.'
|
||||
)
|
||||
|
||||
def test_stats(self):
|
||||
result = self.app.get('/stats')
|
||||
result = self.client.get('/stats')
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(b'<h1>Engine stats</h1>', result.data)
|
||||
|
||||
def test_robots_txt(self):
|
||||
result = self.app.get('/robots.txt')
|
||||
result = self.client.get('/robots.txt')
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(b'Allow: /', result.data)
|
||||
|
||||
def test_opensearch_xml(self):
|
||||
result = self.app.get('/opensearch.xml')
|
||||
result = self.client.get('/opensearch.xml')
|
||||
self.assertEqual(result.status_code, 200)
|
||||
self.assertIn(
|
||||
b'<Description>SearXNG is a metasearch engine that respects your privacy.</Description>', result.data
|
||||
)
|
||||
|
||||
def test_favicon(self):
|
||||
result = self.app.get('/favicon.ico')
|
||||
result = self.client.get('/favicon.ico')
|
||||
result.close()
|
||||
self.assertEqual(result.status_code, 200)
|
||||
|
||||
def test_config(self):
|
||||
result = self.app.get('/config')
|
||||
result = self.client.get('/config')
|
||||
self.assertEqual(result.status_code, 200)
|
||||
json_result = result.get_json()
|
||||
self.assertTrue(json_result)
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
# pylint: disable=missing-module-docstring,disable=missing-class-docstring,invalid-name
|
||||
|
||||
import mock
|
||||
from parameterized.parameterized import parameterized
|
||||
@@ -7,7 +7,7 @@ from searx import webutils
|
||||
from tests import SearxTestCase
|
||||
|
||||
|
||||
class TestWebUtils(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestWebUtils(SearxTestCase):
|
||||
|
||||
@parameterized.expand(
|
||||
[
|
||||
@@ -78,8 +78,10 @@ class TestWebUtils(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
self.assertEqual(webutils.highlight_content(content, query), expected)
|
||||
|
||||
|
||||
class TestUnicodeWriter(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestUnicodeWriter(SearxTestCase):
|
||||
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
self.unicode_writer = webutils.CSVWriter(mock.MagicMock())
|
||||
|
||||
def test_write_row(self):
|
||||
@@ -93,7 +95,8 @@ class TestUnicodeWriter(SearxTestCase): # pylint: disable=missing-class-docstri
|
||||
self.assertEqual(self.unicode_writer.writerow.call_count, len(rows))
|
||||
|
||||
|
||||
class TestNewHmac(SearxTestCase): # pylint: disable=missing-class-docstring
|
||||
class TestNewHmac(SearxTestCase):
|
||||
|
||||
@parameterized.expand(
|
||||
[
|
||||
b'secret',
|
||||
|
||||
Reference in New Issue
Block a user