Commit Graph

2825 Commits

Author SHA1 Message Date
Markus Heiser 57cdf180fe [fix] engine archive is - search_url has been changed
BTW: set soft_max_redirects from *archive is* by 1, to prevent logging::

    DEBUG:httpx._client:HTTP Request: GET https://archive.is/search/?q=www.python.org "HTTP/2 302 Found"
    DEBUG:httpx._client:HTTP Request: GET https://archive.is/www.python.org "HTTP/2 200 OK"
    DEBUG:searx:archive is: ErrorContext('searx/search/processors/online.py', 110, 'count_error(self.engine_name,', None, '1 redirects, maximum: 0', ('200', 'OK', 'archive.is')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-17 15:21:53 +02:00
Markus Heiser ffcebf5e12 [enh] xpath engine - add request parameter 'soft_max_redirects'
Make 'soft_max_redirects' configurable per Xpath engine::

    - name : <engine-name>
      engine : xpath
      soft_max_redirects: 1
      ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-17 15:04:55 +02:00
Alexandre Flament 90fe232e08 [fix] webapp.py: pylint: disable=assigning-non-slot for request.*
Note: actually, the code should use flask.g.* instead of request.*
2021-05-14 11:15:35 +02:00
Alexandre Flament d12033e166 [mod] Server-Timing header: add render field
Add a "render" field to the Server-Timing header
This field is the time spent in the searx.webapp.render function (Jinja2 rendering)

See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing
2021-05-06 09:39:52 +02:00
Alexandre Flament 0f4e995ab4 [mod] searx.network.client: the same configuration reuses the same ssl.SSLContext
before there was one ssl.SSLContext per client.

see https://github.com/encode/httpx/issues/978
2021-05-05 20:36:37 +02:00
Markus Heiser fa0d05c313 [pylint] checker/__main__.py & checker/background.py
Lint files that has been touched by [PR #58]

[PR #58] https://github.com/searxng/searxng/pull/58

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-05 16:47:02 +02:00
Alexandre Flament 8c1a65d32f [mod] multithreading only in searx.search.* packages
it prepares the new architecture change,
everything about multithreading in moved in the searx.search.* packages

previously the call to the "init" function of the engines was done in searx.engines:
* the network was not set (request not sent using the defined proxy)
* it requires to monkey patch the code to avoid HTTP requests during the tests
2021-05-05 13:12:42 +02:00
Markus Heiser d36adfa59f
Merge pull request #55 from searxng/upd-relialibility-column
[themes] /preferences, reliablity: warning icons & error logs link
2021-05-04 12:20:45 +00:00
Markus Heiser 5253df946b
[mod] option to enable or disable "proxy" button next to each result (#54)
* [mod] option to enable or disable "proxy" button next to each result

Closes: https://github.com/searxng/searxng/issues/51
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: Alexandre Flament <alex@al-f.net>
2021-05-04 11:57:14 +02:00
Alexandre Flament e83862c22f [themes] /preferences, reliablity: warning icons & error logs link
When there is at least one errors or one failed checker test:
* the warning icon is displayed in the reliability column
* the link "View error logs and submit a bug report" is displayed on engine name tooltip.

Before:
* the warning icon was displayed only when one or more checker test(s) failed.
* the link "View error logs and submit a bug report" was not shown when a checker test failed but there were no error.
2021-05-04 10:50:03 +02:00
Markus Heiser 45293270e2
Merge pull request #44 from searxng/dependabot/pip/master/pylint-2.8.2
Bump pylint from 2.7.4 to 2.8.2
2021-05-03 13:58:14 +00:00
dependabot[bot] 2a53f79a2d Bump pylint from 2.7.4 to 2.8.2
Bumps [pylint](https://github.com/PyCQA/pylint) from 2.7.4 to 2.8.2.
- [Release notes](https://github.com/PyCQA/pylint/releases)
- [Changelog](https://github.com/PyCQA/pylint/blob/master/ChangeLog)
- [Commits](https://github.com/PyCQA/pylint/compare/pylint-2.7.4...v2.8.2)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-03 15:45:30 +02:00
Marc Abonce Seguin 448bfe6005 fix Qwant's fetch_languages function 2021-05-02 17:46:40 -07:00
Alexandre Flament 2756e170ce
Merge pull request #49 from searxng/update_data_update_ahmia_blacklist.py
Update searx.data - update_ahmia_blacklist.py
2021-05-01 10:28:32 +02:00
Alexandre Flament a3a7be90ff
Merge pull request #50 from searxng/update_data_update_currencies.py
Update searx.data - update_currencies.py
2021-05-01 10:28:22 +02:00
Alexandre Flament 4558c0c027
Merge pull request #48 from searxng/update_data_update_wikidata_units.py
Update searx.data - update_wikidata_units.py
2021-05-01 10:28:11 +02:00
dalf dd8a5a00dc Update searx.data - update_currencies.py 2021-05-01 08:23:05 +00:00
dalf de54209f9e Update searx.data - update_ahmia_blacklist.py 2021-05-01 08:23:00 +00:00
dalf 982a5a7d2e Update searx.data - update_wikidata_units.py 2021-05-01 08:22:48 +00:00
dalf c004712eef Update searx.data - update_firefox_version.py 2021-05-01 08:22:43 +00:00
Alexandre Flament e5bcc3cbbe
Merge pull request #40 from searxng/link-to-stats
[mod] engine errors: link to the stats to create an github issue
2021-04-29 14:37:12 +02:00
Markus Heiser 8026ed0d37 [mod] UI: engine pref-page, about toolbox: add link to stats
In the preference page, in the 'about' toolbox of an engine, add a link to the
stats page of the engine, if the engine had one or more errors.

Condition is::

    reliabilities[<engine.name>].errors

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-29 11:11:05 +02:00
Alexandre Flament 34bced29ae [mod] engine errors: link to the stats to create an github issue 2021-04-29 11:10:38 +02:00
Michael Ilsaas 0c43cf89ca [fix] URL to solidtorrent result page
Reported-by: https://github.com/searx/searx/pull/2786
2021-04-29 10:40:47 +02:00
Markus Heiser 8291804082 [static] make themes.all - from commit 9e8171e38
Based on commit 9e8171e38 this patch is generated by::

     make themes.all

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-28 16:22:47 +02:00
Markus Heiser 70cc196e2d [fix] new_issue.html: drop inline style attributes (CSP conformance)
Inline styles are blocked by default with Content Security Policy (CSP).  Move
the inline styles from 'new_issue.html' to::

    searx/static/themes/__common__/less/new_issue.less

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-28 16:22:37 +02:00
Markus Heiser 7f2dc530b2 [fix] simple theme - remove no longer used stats.less file
File searx/static/themes/simple/less/stats.less is not used (imported) in any
other less file.  I can't say when it's usage was dropped or if it has ever been
used.  ATM this file is without any usage.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-28 15:42:08 +02:00
Markus Heiser 54d57c8296 [fix] grunt/less/development: oscar light & dark logicodev mixes CSS
closes: https://github.com/searxng/searxng/issues/39

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-28 15:15:29 +02:00
Alexandre Flament 283ae7bfad [fix] searx.network: fix rare cases where LOOP is None
* searx.network.client.LOOP is initialized in a thread
* searx.network.__init__ imports LOOP which may happen
  before the thread has initialized LOOP

This commit adds a new function "searx.network.client.get_loop()"
to fix this issue
2021-04-27 17:47:36 +02:00
Alexandre Flament 87e914e398
Merge pull request #28 from searxng/mod-processors-error-message
[mod] processors: show identical error messages on /search and /stats
2021-04-27 16:50:41 +02:00
Markus Heiser 924f9afea3 [lint] pylint searx/search/processors files / BTW add some doc-strings
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-27 15:13:39 +02:00
Alexandre Flament b1557b5443 [mod] processors: show identical error messages on /search and /stats 2021-04-27 14:20:07 +02:00
Alexandre Flament b8f5adf500 [mod] searx.metrics.error_recorder: store relative file name instead of the full absolute file name. 2021-04-27 10:42:00 +02:00
Markus Heiser dc29f1d826 [pylint] tag PYLINT_FILES by comment `# lint: pylint`
These py files are linted by `test.pylint`, all other files are linted by
`test.pep8`.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-26 20:18:20 +02:00
Markus Heiser 6b92e8c6fd [upd] ./manage pyenv.cmd searx_extra/update/update_pygments.py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-26 19:11:58 +02:00
Markus Heiser 1a402be175 [fix] debug log: UnicodeEncodeError: 'ascii' codec can't encode
The issue exists only in the debug log::

     --- Logging error ---
     Traceback (most recent call last):
       File "/usr/lib/python3.9/logging/__init__.py", line 1086, in emit
	 stream.write(msg + self.terminator)
     UnicodeEncodeError: 'ascii' codec can't encode characters in position 79-89: ordinal not in range(128)
     Call stack:
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__
	 return self.wsgi_app(environ, start_response)
       File "/usr/local/searx/searx-src/searx/webapp.py", line 1316, in __call__
	 return self.app(environ, start_response)
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/werkzeug/middleware/proxy_fix.py", line 169, in __call__
	 return self.app(environ, start_response)
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
	 response = self.full_dispatch_request()
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
	 rv = self.dispatch_request()
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
	 return self.view_functions[rule.endpoint](**req.view_args)
       File "/usr/local/searx/searx-src/searx/webapp.py", line 766, in search
	 number_of_results=format_decimal(number_of_results),
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask_babel/__init__.py", line 458, in format_decimal
	 locale = get_locale()
       File "/usr/local/searx/searx-pyenv/lib/python3.9/site-packages/flask_babel/__init__.py", line 226, in get_locale
	 rv = babel.locale_selector_func()
       File "/usr/local/searx/searx-src/searx/webapp.py", line 249, in get_locale
	 logger.debug("%s uses locale `%s` from %s", request.url, locale, locale_source)
     Unable to print the message and arguments - possible formatting error.
     Use the traceback above to help find the error.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-26 12:55:26 +02:00
Alexandre Flament 51494849fc
Merge pull request #23 from searxng/metrics-stats-engines
Metrics stats engines
2021-04-25 14:31:52 +02:00
Alexandre Flament df41b77121 [mod] /stats : detail per engine
allow to submit a github issue including the technical details
(exceptions, errors, warning, checker result)
2021-04-25 14:28:06 +02:00
Markus Heiser 28b25185c5 [brand] searxng -- fix links to issue tracker & WEB-GUI
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-25 14:25:08 +02:00
Markus Heiser c6a5cc019a [brand] searxng is a fork from searx
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-25 11:42:42 +02:00
Markus Heiser 6d41255eb1 [fix] Springer-Nature & core.ac need API key to work
It is the whole engine definition that should be comment out : without the
api_key nothing works.

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-24 09:00:53 +02:00
Markus Heiser 8efabd3ab7 [mod] core.ac.uk engine
- add to list of pylint scripts
- add debug log messages
- move API key int `settings.yml`
- improved readability
- add some metadata to results

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-24 09:00:53 +02:00
spongebob33 7528e38c8a add core.ac.uk engine 2021-04-24 08:55:45 +02:00
Alexandre Flament 525bfe7830
Fix PEP8 2021-04-24 07:19:44 +02:00
Alexandre Flament 7032f7521c [mod] /stats: simple theme implementation 2021-04-24 06:58:49 +02:00
Alexandre Flament 09e7ecdce2 [mod] /stats : add reliability column and sort by column links 2021-04-24 06:58:49 +02:00
Markus Heiser c54bf42cb9 [mod] metrics: add to pylint and justify indentations
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-24 06:58:49 +02:00
Alexandre Flament 65c29081cc [mod] update /stats 2021-04-24 06:58:22 +02:00
Alexandre Flament 3755cf2eba More branding 2021-04-22 21:10:22 +02:00
Alexandre Flament 04b34c70b8 More branding 2021-04-22 20:14:16 +02:00