e9afc4f8ce
[mod] Startpage: reversed engineered & upgrade to data_type: traits_v1
...
One reason for the often seen CAPTCHA of the Startpage requests are the
incomplete requests SearXNG sends to startpage.com: this patch is a complete new
implementation of the ``request()`` function, reversed engineered from the
Startpage's search form. The new implementation:
- use traits of data_type: traits_v1 and drop deprecated data_type: supported_languages
- adds time-range support
- adds save-search support
- fix searxng/searxng/issues 1884
- fix searxng/searxng/issues 1081 --> improvements to avoid CAPTCHA
In preparation for more categories (News, Images, Videos ..) from Startpage, the
variable ``startpage_categ`` was set up. The default value is ``web`` and other
categories from Startpage are not yet implemented.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
858aa3e604
[mod] wikipedia & wikidata: upgrade to data_type: traits_v1
...
BTW this fix an issue in wikipedia: SearXNG's locales zh-TW and zh-HK are now
using language `zh-classical` from wikipedia (and not `zh`).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
e0a6ca96cc
[doc] add a description of bing engines (web, news, video, images)
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
15eaf0f15f
[mod] bing_news: use async API & upgrade to data_type: traits_v1
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
ff80e7637e
[mod] bing_images: use async API & upgrade to data_type: traits_v1
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
bc21d28298
[mod] bing_videos: use async API & upgrade to data_type: traits_v1
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
d0f465e6fa
[mod] bing: add time_range support & upgrade to data_type: traits_v1
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
c9cd376186
[mod] replace searx.languages by searx.sxng_locales
...
With the language and region tags from the EngineTraitsMap the handling of
SearXNG's tags of languages and regions has been normalized and is no longer
a *mystery*. The "languages" became "locales" that are supported by babel and
by this, the update_engine_traits.py can be simplified a lot.
Other code places can be simplified as well, but these simplifications
should (respectively can) only be done when none of the engines work with the
deprecated EngineTraits.supported_languages interface anymore.
This commit replaces searx.languages by searx.sxng_locales and fix the naming of
some names from "language" to "locale" (e.g. language_codes --> sxng_locales).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
7daf4f95ef
[mod] Wikipedia: fetch engine traits (data_type: supported_languages)
...
Implements a fetch_traits function for the Wikipedia engines.
.. note::
Does not include migration of the request methode from 'supported_languages'
to 'traits' (EngineTraits) object!
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
f78f908383
[mod] Google: fetch engine traits (data_type: supported_languages)
...
Implements a fetch_traits function for the Google engines.
.. note::
Does not include migration of the request methode from 'supported_languages'
to 'traits' (EngineTraits) object!
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
dba8977b09
[mod] DuckDuckGo: fetch engine traits (data_type: supported_languages)
...
Implements a fetch_traits function for the DuckDuckGo engines.
.. note::
Does not include migration of the request methode from 'supported_languages'
to 'traits' (EngineTraits) object!
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
ef143729a0
[mod] yahoo: fetch engine traits (data_type: traits_v1)
...
Implements a fetch_traits function for the Yahoo engine.
.. note::
Includes migration of the request methode from 'supported_languages' to
'traits' (EngineTraits) object!
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
c1ae2ef57c
[mod] qwant: fetch engine traits (data_type: traits_v1)
...
Implements a fetch_traits function for the Qwant engines.
.. note::
Includes migration of the request methode from 'supported_languages' to
'traits' (EngineTraits) object!
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
fc0c775030
[mod] Dailymotion: fetch engine traits (data_type: supported_languages)
...
Implements a fetch_traits function for the Dailymotion engine.
.. note::
Does not include migration of the request methode from 'supported_languages'
to 'traits' (EngineTraits) object!
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
61383edb27
[mod] Startpage: fetch engine traits (data_type: supported_languages)
...
Implements a fetch_traits function for the Startpage engine.
.. note::
Does not include migration of the request methode from 'supported_languages'
to 'traits' (EngineTraits) object!
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
d3aa690a7a
[mod] bing: fetch engine traits (data_type: supported_languages)
...
Implements a fetch_traits function for the Bing engines.
.. note::
Does not include migration of the request methode from 'supported_languages'
to 'traits' (EngineTraits) object!
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
a7fe22770a
[mod] Peertube: re-engineered & upgrade to data_type: traits_v1
...
- fetch_traits(): Fetch languages from peertube's search-index source code.
[mod] Include migration of the request methode from 'supported_languages'
to 'traits' (EngineTraits) object.
[fix] old supported_languages_url is no longer valid since the sources
has been moved to a different path.
- fixed code to pass pylint
- request(): complete re-implementation based on the API docs [1]
- response(): complete re-implementation, adds serveral fields missed before
- add source code documentation
[1] https://docs.joinpeertube.org/api-rest-reference.html#tag/Search/operation/searchVideos
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
6e5f22e558
[mod] replace engines_languages.json by engines_traits.json
...
Implementations of the *traits* of the engines.
Engine's traits are fetched from the origin engine and stored in a JSON file in
the *data folder*. Most often traits are languages and region codes and their
mapping from SearXNG's representation to the representation in the origin search
engine.
To load traits from the persistence::
searx.enginelib.traits.EngineTraitsMap.from_data()
For new traits new properties can be added to the class::
searx.enginelib.traits.EngineTraits
.. hint::
Implementation is downward compatible to the deprecated *supported_languages
method* from the vintage implementation.
The vintage code is tagged as *deprecated* an can be removed when all engines
has been ported to the *traits method*.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
912f877dec
Bump ionicons from 6.1.3 to 7.1.0 in /searx/static/themes/simple
...
Bumps [ionicons](https://github.com/ionic-team/ionicons ) from 6.1.3 to 7.1.0.
- [Release notes](https://github.com/ionic-team/ionicons/releases )
- [Changelog](https://github.com/ionic-team/ionicons/blob/main/CHANGELOG.md )
- [Commits](https://github.com/ionic-team/ionicons/compare/v6.1.3...v7.1.0 )
---
updated-dependencies:
- dependency-name: ionicons
dependency-type: direct:development
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-24 07:57:14 +00:00
9f3a57c901
[translations] update from Weblate
...
abfec8f4f02ea21c3fc6c653342bbf46
2023-03-24 07:07:52 +00:00
ac169a0f75
Pass black formatting test
2023-03-21 00:41:36 +01:00
e26bce33d4
WIKIDATA: Add description for results
2023-03-21 00:14:54 +01:00
b61b845951
Merge pull request #2266 from return42/shuffle-cipher
...
[mod] Shuffle httpx's default ciphers of a SSL context randomly.
2023-03-20 12:28:05 +01:00
94430e104c
Merge pull request #2238 from return42/fix-2027
...
[fix] fix threshold in replace_auto_language
2023-03-19 15:30:37 +01:00
f2962a2f4a
Merge pull request #2239 from return42/fix-eslintrc
...
[fix] remove duplicate key in simple theme ESLint configuration
2023-03-19 15:30:12 +01:00
8fa54ffddf
[mod] Shuffle httpx's default ciphers of a SSL context randomly.
...
From the analyse of @9Ninety [1] we know that DDG (and may be other engines / I
have startpage in mind) does some kind of TLS fingerprint to block bots.
This patch shuffles the default ciphers from httpx to avoid a cipher profile
that is known to httpx (and blocked by DDG).
[1] https://github.com/searxng/searxng/issues/2246#issuecomment-1467895556
----
From `What Is TLS Fingerprint and How to Bypass It`_
> When implementing TLS fingerprinting, servers can't operate based on a
> locked-in whitelist database of fingerprints. New fingerprints appear
> when web clients or TLS libraries release new versions. So, they have to
> live off a blocklist database instead.
> ...
> It's safe to leave the first three as is but shuffle the remaining ciphers
> and you can bypass the TLS fingerprint check.
.. _What Is TLS Fingerprint and How to Bypass It:
https://www.zenrows.com/blog/what-is-tls-fingerprint#how-to-bypass-tls-fingerprinting
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Closes: https://github.com/searxng/searxng/issues/2246
2023-03-19 13:40:31 +01:00
677903c355
Merge pull request #2257 from Solirs/fix_bad_escape
...
re.escape() the query in highlight_content to prevent a server side error.
2023-03-17 08:54:54 +01:00
fbb0e9d275
[fix] server side error: escape backslashes in the query highlight_content
...
Any backslash escapes in the replacement are processed [1], backslashes should
be escaped [2].
[1] https://docs.python.org/3/library/re.html#re.sub
[2] https://docs.python.org/3/library/re.html#re.escape
closes:
- https://github.com/searxng/searxng/issues/2256
- https://github.com/searxng/searxng/issues/2250
2023-03-17 08:46:00 +01:00
86c3757872
[translations] update from Weblate
...
32926a197aabc876c0ed00f56cf287f68c4c5f83dffe61fac7736cace831b8e3ef3c60afc046a677142041d6119b51df
2023-03-17 07:07:53 +00:00
3e9cddc606
rollback test
2023-03-15 19:55:20 +01:00
41ed0ef0c7
test
2023-03-15 19:53:53 +01:00
097d092a7f
Merge pull request #2224 from searxng/update_data_update_currencies.py
...
Update searx.data - update_currencies.py
2023-03-15 18:36:29 +01:00
85ef1af343
Merge pull request #2222 from searxng/update_data_update_wikidata_units.py
...
Update searx.data - update_wikidata_units.py
2023-03-15 18:35:36 +01:00
a7f1649190
[fix] remove duplicate key in simple theme ESLint configuration
...
Partial merge of [PR-1736]
[PR-1736] https://github.com/searxng/searxng/pull/1736
Suggested-by: @FunctionalHacker in [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-05 09:12:13 +01:00
150a90c84e
[fix] fix threshold in replace_auto_language
...
[1] https://github.com/searxng/searxng/pull/2027#pullrequestreview-1322157677
[2] https://github.com/searxng/searxng/pull/1969#issuecomment-1345354529
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-05 08:29:58 +01:00
1f36fc3a45
[translations] update from Weblate
...
0d8ebfe11358dd6d4d8c13db1ae581b60003698f31c796179015ec7303619a68fa90585bc902c5e5
2023-03-03 07:08:08 +00:00
714e83d5ea
Merge pull request #2220 from Solirs/gentoo_engine_timeout
...
Increase timeout for gentoo wiki engine
2023-03-01 17:57:21 +01:00
1632f18631
Merge pull request #2227 from searxng/update_data_update_engine_descriptions.py
...
Update searx.data - update_engine_descriptions.py
2023-03-01 17:52:57 +01:00
5bbbb14b62
Merge pull request #2226 from searxng/update_data_update_ahmia_blacklist.py
...
Update searx.data - update_ahmia_blacklist.py
2023-03-01 17:52:02 +01:00
5042d94dea
Update searx.data - update_engine_descriptions.py
2023-03-01 01:48:22 +00:00
e30a45812f
Update searx.data - update_ahmia_blacklist.py
2023-03-01 01:38:07 +00:00
935415bfcf
Update searx.data - update_currencies.py
2023-03-01 01:37:46 +00:00
ccd00518fd
Update searx.data - update_firefox_version.py
2023-03-01 01:37:42 +00:00
d7f10909fa
Update searx.data - update_wikidata_units.py
2023-03-01 01:37:25 +00:00
35fbb3578b
Increase timeout for gentoo wiki engine
2023-02-28 13:54:44 +01:00
d669da81fb
Merge pull request #2027 from dalf/fix_2018
...
Add "auto" as a language.
2023-02-20 12:17:38 +01:00
297e463e49
[translations] update from Weblate
...
8ff0fa33
2023-02-19 11:46:59 +00:00
0b1444b61e
[doc] improved docs of implementations for automatic speech recognition
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-19 10:09:52 +00:00
363203c579
Merge pull request #2201 from return42/fix-2190
...
[doc] slight improvements to the doc of the settings (base_url)
2023-02-18 18:21:14 +01:00
6748e8e2d5
Add "Auto-detected" as a language.
...
When the user choose "Auto-detected", the choice remains on the following queries.
The detected language is displayed.
For example "Auto-detected (en)":
* the next query language is going to be auto detected
* for the current query, the detected language is English.
This replace the autodetect_search_language plugin.
2023-02-17 15:17:36 +00:00
bb83036f48
[fix] typo in searx/plugins/tor_check.py
...
Related: https://github.com/searxng/searxng/pull/2189
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-17 13:09:14 +01:00
5820dc78ce
[doc] slight improvements to the doc of the settings (base_url)
...
Closes: https://github.com/searxng/searxng/issues/2190
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-17 12:08:58 +01:00
52f6bc745b
Merge pull request #2188 from ahmad-alkadri/fix/petalsearch
...
Fix the petalsearch engine
2023-02-15 13:57:28 +01:00
ba5b6c382f
Merge pull request #2192 from return42/fix-tineye
...
[mod] disbale engine tineye by default
2023-02-15 06:43:41 +01:00
f6af59899b
Fix petalsearch and remove petalsearch news
2023-02-14 18:43:55 +01:00
2be373a18f
[fix] spelling: Tor, SearXNG
2023-02-14 17:13:53 +01:00
7d446dfdb2
[mod] disbale engine tineye by default
...
Tineye becomes active as soon as a https:// signature is found in the search
term, but most of the time a reverse image search is not requested when a URL is
specified, often the URL is just from a C&P.
The frequent requests to tineye lead in the end to the SearXNG instance being
blocked by tineye and the user seeing unexpected error messages.
BTW: many maintainers have disabled this engine in their local SearXNG settings.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-14 08:27:19 +01:00
3abff182ea
[fix] remove engine neeva from settings.yml
...
Engine is broken and can't by used any longer as a simple XPath engine.
@allendema tested a engines/neeva.py version using json from the dom, but
without luck: There was some kind of captcha for pagination.
[1] https://github.com/searxng/searxng/issues/2007#issuecomment-1426061698
Closes: https://github.com/searxng/searxng/issues/2007
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-10 18:46:37 +01:00
a530ac4099
fix: typo in suggestions
2023-02-10 15:33:25 +01:00
415f6398d6
[fix] time_range in HTML form should not be a string 'None'
...
Before this patch SearXNG returns 'time_range': 'None'::
<form method="GET" action="/searx/search">
...
<input type="hidden" name="time_range" value="None">
...
</form>
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-10 15:32:21 +01:00
6707354bc8
[mod] engine wiby: add pagination
...
Suggested by: @wibyweb in searx https://github.com/searx/searx/pull/3465
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-10 15:31:24 +01:00
4c06837a50
[mod] make python code pylint 2.16.1 compliant
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-10 13:59:21 +01:00
7320b0c796
Merge pull request #2130 from searxng/translations_update
...
Update translations
2023-02-04 17:53:20 +01:00
a5a73d77b3
Bump grunt from 1.5.3 to 1.6.1 in /searx/static/themes/simple
...
Bumps [grunt](https://github.com/gruntjs/grunt ) from 1.5.3 to 1.6.1.
- [Release notes](https://github.com/gruntjs/grunt/releases )
- [Changelog](https://github.com/gruntjs/grunt/blob/main/CHANGELOG )
- [Commits](https://github.com/gruntjs/grunt/compare/v1.5.3...v1.6.1 )
---
updated-dependencies:
- dependency-name: grunt
dependency-type: direct:development
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-04 08:10:18 +00:00
257dc7d6c4
[fix-2146] set different HTTP Referer header to DuckDuckGo requests
...
For what ever reasons, ddg-lite don't like the Referer
https://lite.duckduckgo.com/
In an interactive session in the WEB browser the the Reverer has exactly this
value, but ddg-lite don't like this value when the request is build up by
SearXNG. The new value is:
https://google.com/
What fakes a user comes from a google link.
Related: https://github.com/searxng/searxng/pull/2081
Closes: https://github.com/searxng/searxng/issues/2146
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-02-03 08:45:51 +01:00
8db35f8e9c
[translations] update from Weblate
...
009d48c2a5353bad1617f45879bc60f4
2023-02-03 07:15:25 +00:00
9d102fb08f
Merge pull request #2132 from dalf/update_pr_1967
...
search.suspended_time settings: bug fixes
2023-01-29 20:48:43 +01:00
feccee01c0
[doc] Add doc-strings to searx.exceptions
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-01-29 19:06:19 +01:00
4f808088f1
Merge pull request #2139 from dalf/fix_update_languages
...
Fix update languages
2023-01-29 17:59:59 +01:00
d989c9429f
Merge pull request #2134 from searxng/update_data_update_wikidata_units.py
...
Update searx.data - update_wikidata_units.py
2023-01-29 17:59:40 +01:00
5c2667b956
Merge pull request #2135 from searxng/update_data_update_ahmia_blacklist.py
...
Update searx.data - update_ahmia_blacklist.py
2023-01-29 17:59:03 +01:00
e6a17f2fae
Merge pull request #2136 from searxng/update_data_update_firefox_version.py
...
Update searx.data - update_firefox_version.py
2023-01-29 17:58:22 +01:00
c4cbf8829a
Merge pull request #2137 from searxng/update_data_update_currencies.py
...
Update searx.data - update_currencies.py
2023-01-29 17:57:56 +01:00
bfca63c536
wikipedia engine: update _fetch_supported_languages
...
the layout https://meta.wikimedia.org/wiki/List_of_Wikipedias has changed
2023-01-29 10:01:58 +00:00
8256de2fe8
peertube engine: update _fetch_supported_languages
...
There is now an API to get the list of supported languages
https://docs.joinpeertube.org/api-rest-reference.html#tag/Video/operation/getLanguages
2023-01-29 10:01:54 +00:00
f900e33092
Update searx.data - update_engine_descriptions.py
2023-01-29 01:33:29 +00:00
dafe026815
Update searx.data - update_currencies.py
2023-01-29 01:24:14 +00:00
47435be70b
Update searx.data - update_firefox_version.py
2023-01-29 01:23:43 +00:00
53a4a90d0c
Update searx.data - update_wikidata_units.py
2023-01-29 01:23:42 +00:00
9dd95457a9
Update searx.data - update_ahmia_blacklist.py
2023-01-29 01:23:42 +00:00
37addec69e
search.suspended_time settings: bug fixes
...
* fix type in settings.yml: replace suspend_times by suspended_times
* always use delay defined in settings.yml:
* HTTP status 402 and 403: read the value from settings.yml instead of using the hardcoded value of 1 day.
* startpage engine: CAPTCHA suspend the engine for one day instead of one week
2023-01-28 10:24:14 +00:00
28356f4284
Merge pull request #2115 from searxng/dependabot/pip/master/flask-babel-3.0.0
...
Bump flask-babel from 2.0.0 to 3.0.0
2023-01-27 10:43:17 +01:00
b3883dc395
[build] /static
2023-01-23 23:00:38 +01:00
8ead23347f
[fix] use input elements for pagination requests
...
* use html input elements instead of buttons for the pagination forms at the bottom of the result page
* move the less section that hides the pagination number widget on mobile to the mobile section
* clean up the less code for styling the numbers of the pagination widget
* fix: add the margin for box results (like in news category) to the bottom of the result to have a margin between pagination widget and article result
2023-01-23 22:56:55 +01:00
d2b2300ee1
Feature: Pagination Widget with list of numbers.
...
Adds to the navigation widget, preserving forward/backward nav, and
inserting a list of clickable page numbers between them.
Phone sized devices continue without this widget as deterministic
display under small screen sizes has not been solved.
The widget is agnostic to the actual amount of pages there are that one
can navigate to and as such shows all plausible, albeit not necessarilly
valid, possibilities.
This widget does not interfere with infinite scroll in any fashion.
2023-01-23 21:22:06 +01:00
06b5068a2e
SearXNG version: YYYY.MM.DD without leading zero
2023-01-22 10:45:00 +00:00
a9d6f7532a
weblate: migration to https://translate.codeberg.org/
2023-01-21 15:45:12 +00:00
6e4c8b3516
[translations] update from Weblate
2023-01-20 11:32:23 +00:00
f759a84af4
SearXNG version: fix make docker
...
continuation of #2117
related to #2111
This commit:
* fixes the Docker tag using an additional variable DOCKER_TAG, see searx/version.py
* fixes the Docker labels org.label-schema.vcs-ref and org.opencontainers.image.revision
* adds searx/version_frozen to .gitignore
2023-01-20 10:46:00 +00:00
70fff2ebf1
Bump flask-babel from 2.0.0 to 3.0.0
...
Bumps [flask-babel](https://github.com/python-babel/flask-babel ) from 2.0.0 to 3.0.0.
- [Release notes](https://github.com/python-babel/flask-babel/releases )
- [Changelog](https://github.com/python-babel/flask-babel/blob/master/CHANGELOG )
- [Commits](https://github.com/python-babel/flask-babel/compare/v2.0.0...v3.0.0 )
---
updated-dependencies:
- dependency-name: flask-babel
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-20 09:38:30 +00:00
0515f2974b
Version number follows PEP440 : replace - by +
...
Close #2111
2023-01-20 09:05:44 +00:00
6d72ef3cbe
Merge pull request #2109 from ahmad-alkadri/fix/highlight-full-word
...
Standalone words highlighting for query result in non-CJK characters
2023-01-17 23:24:04 +01:00
13b0c251c4
Merge pull request #2100 from nexryai/master
...
Add goo engine
2023-01-15 23:08:28 +01:00
99b5272d9a
A little fix and modified the testing for content highlight
2023-01-15 16:51:31 +01:00
0cedb1c6d8
Add search.suspended_times settings
...
Make suspended_time changeable in settings.yml
Allow different values to be set for different exceptions.
Co-authored-by: Alexandre Flament <alex@al-f.net>
2023-01-15 09:00:32 +00:00
6c421110b5
Perso branch - added cjk check and enclosement
2023-01-14 23:00:08 +00:00
3e5ea08504
[translations] update from Weblate
...
6a57ee0bb180a8108710174a529a2591a254bd01860c3fcec140fb52
2023-01-13 07:14:55 +00:00
4e7bb1bf9a
Add goo engine
2023-01-12 16:28:09 +09:00
7fc8d72889
[fix] bing: parsing result; check to see if the element contains links
...
This patch is to hardening the parsing of the bing response:
1. To fix [2087] check if the selected result item contains a link, otherwise
skip result item and continue in the result loop. Increment the result
pointer when a result has been added / the enumerate that counts for skipped
items is no longer valid when result items are skipped.
To test the bugfix use: ``!bi :all cerbot``
2. Limit the XPath selection of result items to direct children nodes (list
items ``li``) of the ordered list (``ol``).
To test the selector use: ``!bi :en pontiac aztek wiki``
.. in the result list you should find the wikipedia entry on top,
compare [2068]
[2087] https://github.com/searxng/searxng/issues/2087
[2068] https://github.com/searxng/searxng/issues/2068
2023-01-09 15:08:24 +01:00
cf4db4be37
[fix] Mojeek Xpath showing suggestions and searches + add lang support
2023-01-09 09:33:47 +01:00
54e63839db
Merge pull request #2088 from llmII/fix-suggestions
...
Fix #2083
2023-01-08 18:53:32 +01:00
9d6b048fba
[build] /static
2023-01-08 18:39:37 +01:00
33d97fd01a
Fix #2083
...
This changes the Suggestions to be a single column, not a wrapping row,
changing the input to be incapable of overflowing into visually adjacent
elements.
2023-01-08 10:44:58 -06:00
1407a055b2
[simple theme] set tab index for search page input field #1891
...
Previously it took five tab keypresses to reach search input field
on search results page. Now one tab press is enough.
2023-01-08 13:41:48 +03:00
9ee99423fe
[fix] Bing-Web engine: XPath to get the wikipedia result
...
Modify the XPath selector to get the wikipedia result plus small fixes.
About result content: especially with the Wikipedia result, we'd get several
paragraph elements, only the first paragraph would be taken and displayed on the
search result
2023-01-08 09:11:16 +01:00
4e355564d2
Merge pull request #1903 from LalleSX/patch-1
...
Added info about adding the enginge to the brownser
2023-01-07 15:03:12 +01:00
212acf7f7b
[mod] about page: add information about adding enginge to the brownser
2023-01-07 14:48:06 +01:00
4cd3be8964
[fix] indentation and line size of the the markdown files
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-01-07 14:46:31 +01:00
b2a80347e3
[build] /static
2023-01-07 11:53:51 +01:00
db70c262c3
Add flex-wrap to the list of engines
2023-01-07 11:53:05 +01:00
128b8c7f0a
Add HTTP Referer header to DuckDuckGo requests
...
closes #2080
2023-01-06 16:07:37 -06:00
b241015e93
Merge pull request #2078 from searxng/dependabot/pip/master/pygments-2.14.0
...
Bump pygments from 2.13.0 to 2.14.0
2023-01-06 08:40:46 +01:00
e9a7a6aea6
[build] /static
2023-01-06 08:35:52 +01:00
2420ab6fc3
[translations] update from Weblate
...
b1c60c5b891335484398061a9857916a0ba500780189e64824ded833
2023-01-06 07:14:08 +00:00
38c22c8c31
[translations] update from Weblate
...
ea56f7b60dbbb03181f33cb75ff824600ea6d8c3b19bb597904fa9abe7c7d624
2022-12-30 07:14:21 +00:00
0639769274
Update searx.data - update_firefox_version.py
2022-12-29 09:17:56 +01:00
174e6851b3
Merge pull request #2054 from searxng/update_data_update_currencies.py
...
Update searx.data - update_currencies.py
2022-12-29 08:35:51 +01:00
d531f89316
Merge pull request #2057 from searxng/update_data_update_engine_descriptions.py
...
Update searx.data - update_engine_descriptions.py
2022-12-29 08:31:19 +01:00
76cd808a49
Merge pull request #2053 from searxng/update_data_update_ahmia_blacklist.py
...
Update searx.data - update_ahmia_blacklist.py
2022-12-29 08:28:19 +01:00
000734cb93
Update searx.data - update_engine_descriptions.py
2022-12-29 01:25:31 +00:00
36eea5b7cc
Update searx.data - update_currencies.py
2022-12-29 01:16:59 +00:00
66780efda7
Update searx.data - update_wikidata_units.py
2022-12-29 01:16:59 +00:00
de793fb71e
Update searx.data - update_ahmia_blacklist.py
2022-12-29 01:16:52 +00:00
0d4896170f
Merge pull request #2047 from dalf/fasttext-predict
...
Use fasttext-predict instead of fasttext(-wheel)
2022-12-26 20:50:09 +01:00
f3515041af
Lazy load fasttext-predict
2022-12-26 18:44:23 +00:00
dd125321e4
[translations] update from Weblate
...
7f3e72fd5d48e55656e67eb028e1abe1b19d55f7274f7ee9ab00649b
2022-12-23 07:14:39 +00:00
6804ff048d
Fix: add trailing slash to duckduckgo url
...
Close #1854
2022-12-22 07:49:58 -06:00
735e388cec
Merge branch 'master' into fasttext
2022-12-16 19:43:10 +00:00
1f8f8c1e91
Replace langdetect with fasttext
2022-12-16 21:07:39 +02:00
2a51c85672
Merge pull request #1974 from fehho/query-string-deserial
...
[fix] Deserialize preferences which are explicitly blank strings
2022-12-16 18:56:30 +01:00
03ce848e04
[translations] update from Weblate
...
4cf39daef67457cfdee211c29eeb4290
2022-12-16 07:15:19 +00:00
9e9f57e48b
Merge pull request #1954 from dalf/fix.redis.init.2
...
[fix] follow up of PR-1856
2022-12-14 07:08:19 +01:00
ed901ab18e
[mod] improve 'Autodetect search language' plugin
...
- Add documentation to the plugin
- Harmonize FastText language model with SearXNG's language model
Reosurces::
import fasttext # --> +10 MB
fasttext.load_model(str(data_dir / 'lid.176.ftz')) # --> +4MB
Suggested-by: @dalf
- To speed up and simplify the deployment use fasttext-wheel instead of fasttext
- Building numpy on the Alpine Linux of docker-images takes ages --> install
py3-numpy from Alpines package manager (apk)
- Alpine Linux on docker-images (musl libc) do not support fasttext-wheel (gnu
libc) --> patch Dockerfile and build from fastetxt:
sed -i s/fasttext-wheel/fasttext/ requirements.txt
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-12-11 11:26:07 +01:00
9925a20950
[mod] new plugin: Autodetect search language
2022-12-10 13:11:47 +01:00
b971167ced
move searx.shared.redisdb to searx.redisdb
2022-12-10 09:26:38 +01:00
34e260f88f
[fix] follow up of PR-1856
...
- the environment variable SEARXNG_REDIS_URL overrides the setting value redis.url
- ./manage sets SEARXNG_REDIS_URL to unix:///usr/local/searxng-redis/run/redis.sock if:
- the socket exists
- SEARXNG_REDIS_URL is not already defined
Update of PR #1856
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2022-12-10 09:26:38 +01:00
759d53c892
[translations] update from Weblate
...
1b4600d0e3b49827975217cf
2022-12-09 07:15:14 +00:00
269326063a
Fix: don't crash when engine or name is missing in settings.yml
...
SearXNG crashes if the engine or name fields are missing.
With this commit, the app displays an error in the log and keeps loading.
Close #1951
2022-12-04 23:43:59 +01:00
f40d1c0004
[translations] update from Weblate
...
cc5cf5d897776fbfc3c9f988a1cf339e4429167c
2022-12-02 07:15:43 +00:00
e19218c6a9
Copy client connection kwargs before mutating
2022-11-29 12:15:18 +00:00
a8359dd4d3
Merge pull request #1981 from searxng/update_data_update_firefox_version.py
...
Update searx.data - update_firefox_version.py
2022-11-29 07:48:26 +01:00
1b2f1c17f6
Merge pull request #1982 from searxng/update_data_update_wikidata_units.py
...
Update searx.data - update_wikidata_units.py
2022-11-29 07:47:52 +01:00
3579a38a2d
Merge pull request #1983 from searxng/update_data_update_currencies.py
...
Update searx.data - update_currencies.py
2022-11-29 07:46:49 +01:00
82af2f4460
Merge pull request #1984 from searxng/update_data_update_ahmia_blacklist.py
...
Update searx.data - update_ahmia_blacklist.py
2022-11-29 07:45:51 +01:00
a508cdbd4b
Update searx.data - update_engine_descriptions.py
2022-11-29 01:34:42 +00:00
c4512cf64d
Update searx.data - update_ahmia_blacklist.py
2022-11-29 01:24:48 +00:00
b0b45fd2d0
Update searx.data - update_currencies.py
2022-11-29 01:24:39 +00:00
a9be73b4da
Update searx.data - update_wikidata_units.py
2022-11-29 01:24:38 +00:00
9fc72419e1
Update searx.data - update_firefox_version.py
2022-11-29 01:24:35 +00:00
981a2c9626
[translations] update from Weblate
...
c13727fc21f638f6b8ed9e1c
2022-11-25 07:16:11 +00:00
7cab51f98f
Deserialize explicitly blank strings
...
Default behavior of urllib.parse_qs is to discard blank values, causing a preference of none to be deserialized as undefined, using the instance default rather than the selected preference.
2022-11-24 13:14:12 -06:00
9d06115d07
[translations] update from Weblate
...
7569f2666418da9aacc0dc1a6c2beba6b49457e3
2022-11-19 16:21:15 +00:00
7a39952558
[translations] update from Weblate
...
f123be21fa2cfa8f
2022-11-18 07:17:51 +00:00
46ad32343a
Switch back to protobuf for raw HTML
2022-11-11 07:39:48 +00:00
78be4b4c70
Fix Google search engine.
...
- Fix broken links. Resolves #1794
- Fix missing results. Resolves #1829
2022-11-11 07:34:19 +01:00
408200c87e
[fix] disabling zlibrary due to z-lib.org domain seizure
2022-11-10 21:18:21 +01:00
ee4475ff4b
Merge pull request #1934 from dalf/fix_user_agent
...
User agent: don't include the patch number in the Firefox version
2022-11-09 06:50:33 +01:00
73b5a58d9e
[FIX] Redis initialization
...
redis.Redis.from_url(url) doesn't check if the url is valid
Before this commit: actual error are detected later when the client is actually used.
With this commit, client() makes sure to return a valid Redis client or None.
Also, the code makes sure not to log the password of the Redis URL
2022-11-07 22:08:32 +00:00
8f19bdaf17
Merge pull request #1882 from fehho/metacpan
...
Add MetaCPAN engine
2022-11-07 21:54:11 +01:00
fe351c2802
Add MetaCPAN engine
2022-11-07 08:07:06 -06:00
1517724615
Update network.py
2022-11-06 20:35:30 +01:00
2dc5c0e1d6
Merge pull request #1928 from searxng/translations_update
...
Update translations
2022-11-06 11:29:28 +01:00
4fe5463671
Merge pull request #1869 from vgerak/1836-ddg-weather
...
Fix floating point format in DDG weather humidity
2022-11-05 22:46:50 +01:00
d8dee21277
searx.locale: add Dhivehi language
2022-11-05 21:26:15 +00:00
20d951fc5c
searx.locales: improve support for languages not supported by babel
...
* refactor get_translations() to rely on ADDITIONAL_TRANSLATIONS and LOCALE_BEST_MATCH
* update RTL_LOCALES for languages in ADDITIONAL_TRANSLATIONS
2022-11-05 21:26:07 +00:00
e473addaff
User agent: don't include the patch number in the Firefox version
...
The Firefox version in the user agent doesn't include the patch version: 106.0 not 106.0.2
Close #1914
2022-11-05 22:04:37 +01:00
e92755d358
Initialize Redis in searx/webapp.py
...
settings.yml:
* The default URL was unix:///usr/local/searxng-redis/run/redis.sock?db=0
* The default URL is now "false"
The default URL makes the log difficult to deal with:
if the admin didn't install a Redis instance, the logs record a false error.
It worked before because SearXNG initialized the Redis connection when the limiter started.
In this commit, SearXNG initializes Redis in searx/webapp.py
so various components can use Redis without taking care of the initialization step.
2022-11-05 17:45:52 +01:00
fe419e355b
The checker requires Redis
...
Remove the abstraction in searx.shared.SharedDict.
Implement a basic and dedicated scheduler for the checker using a Redis script.
2022-11-05 12:04:50 +01:00
7438e48d13
[translations] update from Weblate
...
41ebf8e71de65e6afff11c22f677049febf69b4aa21b6221ac384800c4736fd41b967129
2022-11-04 07:18:35 +00:00
fc9986de0a
Merge pull request #1905 from searxng/update_data_update_ahmia_blacklist.py
...
Update searx.data - update_ahmia_blacklist.py
2022-10-29 08:22:43 +02:00
3f1d594c23
Merge pull request #1906 from searxng/update_data_update_external_bangs.py
...
Update searx.data - update_external_bangs.py
2022-10-29 08:21:33 +02:00
a9deead102
Merge pull request #1907 from searxng/update_data_update_firefox_version.py
...
Update searx.data - update_firefox_version.py
2022-10-29 08:20:53 +02:00
d49ccb54ea
Merge pull request #1908 from searxng/update_data_update_wikidata_units.py
...
Update searx.data - update_wikidata_units.py
2022-10-29 08:20:27 +02:00
fa59ff9b78
Merge pull request #1909 from searxng/update_data_update_currencies.py
...
Update searx.data - update_currencies.py
2022-10-29 08:19:56 +02:00
e70589c598
Update searx.data - update_engine_descriptions.py
2022-10-29 01:56:57 +00:00
985a2a87a6
Update searx.data - update_currencies.py
2022-10-29 01:46:40 +00:00
c0ecc2f865
Update searx.data - update_wikidata_units.py
2022-10-29 01:46:38 +00:00
c24abc2f8c
Update searx.data - update_firefox_version.py
2022-10-29 01:46:34 +00:00
f0c7898445
Update searx.data - update_external_bangs.py
2022-10-29 01:46:27 +00:00
799c5270ae
Update searx.data - update_ahmia_blacklist.py
2022-10-29 01:46:24 +00:00
5a181ea1f5
Merge pull request #1901 from searxng/dependabot/npm_and_yarn/searx/static/themes/simple/master/svgo-3.0.0
...
Bump svgo from 2.8.0 to 3.0.0 in /searx/static/themes/simple
2022-10-28 10:22:56 +02:00
d913f0d8aa
[build] /static
2022-10-28 10:17:56 +02:00
bc3db6fd16
[translations] update from Weblate
...
bf52cdbcfbb36c7618cc8a367314d2b6b25147b9d39ade65cb835cdc563f4e0ba698a2e9df5ce28155b369a33eddf243edd3871d5aab9f7b0b2cfe68732b7865d652c20f0bcaaa8d6fd87821618f213c30d38ee9a934ce3bb696c22e6ee1efe2
2022-10-28 07:18:50 +00:00
3cade9a7eb
Bump svgo from 2.8.0 to 3.0.0 in /searx/static/themes/simple
...
Bumps [svgo](https://github.com/svg/svgo ) from 2.8.0 to 3.0.0.
- [Release notes](https://github.com/svg/svgo/releases )
- [Changelog](https://github.com/svg/svgo/blob/main/CHANGELOG-old.md )
- [Commits](https://github.com/svg/svgo/compare/v2.8.0...v3.0.0 )
---
updated-dependencies:
- dependency-name: svgo
dependency-type: direct:development
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-10-28 07:13:58 +00:00
18ba8628dd
[translations] update from Weblate
...
c7fddee3f559a626fe7af6628a7c3da9a84dad5c9ee1ac8bb91bedc4dff99ef4cd639f6c49507a60ad30558219e9707597581f70fcd52cea
2022-10-21 07:32:09 +00:00
947b62c9d5
Fix floating point format in DDG weather humidity
...
Fixes #1836
2022-10-20 11:44:17 +03:00
035bc507ec
[fix] startpage engine
2022-10-14 18:27:53 +00:00
32e8c2cf09
searx.network: add "verify" option to the networks
...
Each network can define a verify option:
* false to disable certificate verification
* a path to existing certificate.
SearXNG uses SSL_CERT_FILE and SSL_CERT_DIR when they are defined
see https://www.python-httpx.org/environment_variables/#ssl_cert_file
2022-10-14 13:59:22 +00:00
6e86f33342
[translations] update from Weblate
...
eee3d5e2c4eb981827423f5b9e8b7f861203aa6a22c78af56a4776f8c00bdc7de5a39537e3fad460da5f0bc35704a98bf3ef3ebb620636c1767b3bb260d38c87
2022-10-14 07:38:20 +00:00
23d797ee6f
Update searx.data - update_osm_keys_tags.py
...
Patch was generated by::
./manage pyenv.cmd searxng_extra/update/update_osm_keys_tags.py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-10-11 11:49:06 +02:00
325eadfd04
[translations] update from Weblate
...
70909b19acf1f0aa94ce447f59959050e27a5c792c05470f53644516
2022-10-07 07:30:33 +00:00
8d5653e60d
Merge branch 'searxng:master' into master
2022-09-30 23:06:54 +00:00
4fea4acea1
[translations] update from Weblate
...
fbac3535cf5d71b76dc351bd42daf1f8f5fb629f4e5f2c795131835f02055902716bd1d7768c8e43e8ad24c1a88c5f91fc70c33a
2022-09-30 07:43:45 +00:00
a7d693231c
Merge pull request #1819 from searxng/update_data_update_currencies.py
...
Update searx.data - update_currencies.py
2022-09-29 16:03:34 +02:00
f3d25f9cef
Merge pull request #1821 from searxng/update_data_update_engine_descriptions.py
...
Update searx.data - update_engine_descriptions.py
2022-09-29 16:02:57 +02:00
5060732486
Merge pull request #1818 from searxng/update_data_update_ahmia_blacklist.py
...
Update searx.data - update_ahmia_blacklist.py
2022-09-29 16:02:02 +02:00
520a873ab4
Merge pull request #1820 from searxng/update_data_update_wikidata_units.py
...
Update searx.data - update_wikidata_units.py
2022-09-29 16:01:31 +02:00
9cb792c0d6
Update searx.data - update_engine_descriptions.py
2022-09-29 02:22:00 +00:00
2d429ebd75
Update searx.data - update_wikidata_units.py
2022-09-29 02:13:02 +00:00
8fc3940448
Update searx.data - update_currencies.py
2022-09-29 02:12:54 +00:00
Markus Heiser
dependabot[bot]
searxng-bot
Solirs
Alexandre Flament
dalf
Ahmad Alkadri
Allan Nordhøy
Lea
wibyweb
mrpaulblack
llmII
Léon Tiekötter
nexryai
Milad-Laly
allixx
PrivateLalle
Rudis Muiznieks
ArtikusHG
Ryan Kaskel
fehho
Émilien Devos
ngosang
Ryan Draga
Alexandre FLAMENT
Evhorizon
Vasilis Gerakaris
Mohamed Elashri