jandieman
/
searxng
mirror of https://github.com/searxng/searxng.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
7,362 Commits 7 Branches 17 Tags 124 MiB
Commit Graph

332 Commits

Author SHA1 Message Date
dalf 8d6d32bf6e Update searx.data - update_ahmia_blacklist.py 2022-06-01 06:31:54 +00:00
dalf 012d7fcaa0 Update searx.data - update_languages.py 2022-06-01 06:31:54 +00:00
dalf ab9616c2cc Update searx.data - update_wikidata_units.py 2022-06-01 06:31:52 +00:00
Markus Heiser 8a79eb4d7b
Merge pull request #1158 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-05-01 09:51:56 +02:00
Markus Heiser 6bd6e09e14
Merge pull request #1159 from searxng/update_data_update_engine_descriptions.py 
Update searx.data - update_engine_descriptions.py
 2022-05-01 09:51:21 +02:00
Markus Heiser 9c7e9bf1d9
Merge pull request #1157 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-05-01 09:50:28 +02:00
Markus Heiser 9e6000407a
Merge pull request #1156 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-05-01 09:49:55 +02:00
dalf 6f0a717dce Update searx.data - update_engine_descriptions.py 2022-05-01 06:33:42 +00:00
dalf bdfbc0ec02 Update searx.data - update_currencies.py 2022-05-01 06:26:53 +00:00
dalf 6cb08a568a Update searx.data - update_wikidata_units.py 2022-05-01 06:26:44 +00:00
dalf 10023344a5 Update searx.data - update_ahmia_blacklist.py 2022-05-01 06:26:33 +00:00
dalf e36d7067a9 Update searx.data - update_firefox_version.py 2022-05-01 06:26:31 +00:00
Markus Heiser 62982c8812 [fix] add back missing languages & regions (followup of PR #1071) 
In PR #1071 the language catalog of dailymotion has been cleaned up, before
there had been over 7000 "languages" in the catalog.

As a side effect of this clean-up the language & region catalog in SearXNG has
been reduced [1].

This patch reduce the ``min_engines_per_lang`` from 13 to 12 to get the missed
languages back in language & region catalog of SearXNG.

[1] 3bb62823ec (diff-f3f00db0f87f95b882624a192e0aac21525638af0b18c9514e765fcf1991678d)

Requested-by: @tiekoetter in a Matrix chat
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-04-22 12:09:42 +02:00
Alexandre Flament 7bf9e12db9 Update searx.data - update_engine_descriptions.py 2022-04-16 13:17:39 +02:00
Markus Heiser 3bb62823ec [fix] dailymotion engine: filter by language & country 
- fix the issue of fetching more the 7000 *languages*
- improve the request function and filter by language & country
- implement time_range_support & safesearch
- add more fields to the response from dailymotion (allow_embed, length)
- better clean up of HTML tags in the 'content' field.

This is more or less a complete rework based on the '/videos' API from [1].
This patch cleans up the language list in SearXNG that has been polluted by the
ISO-639-3 2 and 3 letter codes from dailymotion languages which have never been
used.

[1] https://developers.dailymotion.com/tools/

Closes: https://github.com/searxng/searxng/issues/1065
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-04-16 09:27:34 +02:00
Alexandre Flament 765504ceca
Merge pull request #1031 from dalf/fix-bing-_fetch_supported_languages 
Fix bing  fetch supported languages
 2022-04-02 08:23:58 +02:00
Markus Heiser 099840f4d6
Merge pull request #1035 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-04-01 09:02:19 +02:00
Markus Heiser b853ab717e
Merge pull request #1034 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-04-01 09:01:48 +02:00
Markus Heiser c2de356497
Merge pull request #1033 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-04-01 09:00:51 +02:00
dalf 086517d269 Update searx.data - update_wikidata_units.py 2022-04-01 06:27:23 +00:00
dalf d245ca97ca Update searx.data - update_ahmia_blacklist.py 2022-04-01 06:27:22 +00:00
dalf 084d9ab4e4 Update searx.data - update_currencies.py 2022-04-01 06:27:18 +00:00
dalf 4d55523247 Update searx.data - update_firefox_version.py 2022-04-01 06:27:13 +00:00
Alexandre FLAMENT 7d31b2c0af Update languages 2022-03-31 20:04:46 +00:00
Markus Heiser 0ddcc12474
Merge pull request #934 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-03-01 07:54:58 +01:00
Markus Heiser 69f84d941f
Merge pull request #935 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-03-01 07:53:33 +01:00
Markus Heiser 5508155252
Merge pull request #936 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-03-01 07:52:54 +01:00
dalf f4c6eba227 Update searx.data - update_currencies.py 2022-03-01 06:25:18 +00:00
dalf 6946c4566f Update searx.data - update_wikidata_units.py 2022-03-01 06:24:57 +00:00
dalf 49bc3971ae Update searx.data - update_firefox_version.py 2022-03-01 06:24:55 +00:00
dalf 6add9dd6d3 Update searx.data - update_ahmia_blacklist.py 2022-03-01 06:24:53 +00:00
Markus Heiser 314ea2dc61
Merge pull request #852 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-02-01 06:52:57 +00:00
Markus Heiser 4d0bd47ff0
Merge pull request #853 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-02-01 06:52:19 +00:00
Markus Heiser 20cbabff96
Merge pull request #851 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-02-01 06:48:54 +00:00
dalf f4f90aa3a4 Update searx.data - update_firefox_version.py 2022-02-01 06:23:16 +00:00
dalf 7b402e1aa0 Update searx.data - update_ahmia_blacklist.py 2022-02-01 06:23:01 +00:00
dalf 1c47f4b6e0 Update searx.data - update_currencies.py 2022-02-01 06:23:00 +00:00
dalf e78dc45d1e Update searx.data - update_wikidata_units.py 2022-02-01 06:22:58 +00:00
Alexandre Flament f3f61df6a0 [mod] remove deprecate code 
remove code to support Python 3.5 and Python 3.6
 2022-01-29 08:54:12 +01:00
Markus Heiser a7c3c352d8 [upd] searx/data/external_bangs.json 
Updated external bangs by::

    ./manage pyenv.cmd ./searxng_extra/update/update_external_bangs.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-12 19:37:13 +01:00
Markus Heiser 1461d6a07e [upd] engine_descriptions.json 
Updated by:

    ./manage pyenv.cmd python3 \
        ./searxng_extra/update/update_engine_descriptions.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-07 13:59:03 +01:00
Markus Heiser 3cdf2d6570 Update searx.data - update_languages.py 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-01 18:37:35 +01:00
Alexandre Flament 6b49f32d3e
Merge pull request #670 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-01-01 12:16:50 +01:00
Markus Heiser 3dcba2f67c
Merge pull request #671 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-01-01 12:08:26 +01:00
Markus Heiser 877293c470
Merge pull request #673 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-01-01 12:08:02 +01:00
dalf 127da09fc9 Update searx.data - update_wikidata_units.py 2022-01-01 06:25:06 +00:00
dalf 291483af53 Update searx.data - update_currencies.py 2022-01-01 06:25:02 +00:00
dalf a0016250ec Update searx.data - update_ahmia_blacklist.py 2022-01-01 06:24:59 +00:00
dalf 184242d36b Update searx.data - update_firefox_version.py 2022-01-01 06:24:49 +00:00
Markus Heiser 3d96a9839a [format.python] initial formatting of the python code 
This patch was generated by black [1]::

    make format.python

[1] https://github.com/psf/black

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-12-27 09:26:22 +01:00
Markus Heiser c5d604f06f [upd] engine_descriptions.json 
Updated by:

    ./manage pyenv.cmd python3 \
        ./searxng_extra/update/update_engine_descriptions.py

Include documentation of new added engines:

- superuser
- stackoverflow
- askubuntu
- indb
- ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-12-20 14:39:10 +01:00
Markus Heiser e62733fcf3
Merge pull request #571 from searxng/update_data_update_languages.py 
Update searx.data - update_languages.py
 2021-12-01 07:30:41 +01:00
Markus Heiser 24d04f2854
Merge pull request #570 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-12-01 07:29:56 +01:00
Markus Heiser a3fc1304cc
Merge pull request #569 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-12-01 07:29:24 +01:00
dalf c74b465ece Update searx.data - update_languages.py 2021-12-01 06:24:47 +00:00
dalf 56e653011f Update searx.data - update_firefox_version.py 2021-12-01 06:24:38 +00:00
dalf b833e08fc3 Update searx.data - update_currencies.py 2021-12-01 06:24:29 +00:00
dalf 5fdf8f04b7 Update searx.data - update_ahmia_blacklist.py 2021-12-01 06:24:25 +00:00
Markus Heiser 07f849cd56
Merge pull request #468 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-11-01 06:49:09 +00:00
Markus Heiser 853a144bb4
Merge pull request #467 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-11-01 06:48:43 +00:00
Markus Heiser 82a0dfab61
Merge pull request #466 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-11-01 06:48:15 +00:00
dalf f3dfdad1d1 Update searx.data - update_ahmia_blacklist.py 2021-11-01 06:24:06 +00:00
dalf 2be4c35f76 Update searx.data - update_firefox_version.py 2021-11-01 06:23:59 +00:00
dalf 073d3edc0c Update searx.data - update_wikidata_units.py 2021-11-01 06:23:58 +00:00
dalf 3d9be86519 Update searx.data - update_currencies.py 2021-11-01 06:23:54 +00:00
Markus Heiser f63ffbb22b [fix] engine - yahoo: rewrite and fix issues 
Languages are supported by mapping the language to a domain.  If domain is not
found in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.

BTW: fix issue reported at https://github.com/searx/searx/issues/3020

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-10-16 20:05:26 +00:00
Alexandre Flament 1bb82a6b54 SearXNG: searxng_extra 2021-10-02 17:30:39 +02:00
Markus Heiser fda5609a17
Merge pull request #360 from return42/ddg-lite 
Implement a DuckDuckGo Lite engine
 2021-10-02 05:16:29 +00:00
Markus Heiser 97419a770d [upd] make data.languages 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-10-01 20:01:41 +02:00
Markus Heiser c74f45b1df
Merge pull request #365 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-10-01 07:11:13 +00:00
Markus Heiser 62ea3c7074
Merge pull request #366 from searxng/update_data_update_languages.py 
Update searx.data - update_languages.py
 2021-10-01 06:41:09 +00:00
Markus Heiser d814f1182f
Merge pull request #364 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-10-01 06:34:20 +00:00
Markus Heiser 8479a83411
Merge pull request #363 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-10-01 06:33:51 +00:00
dalf db51a64390 Update searx.data - update_languages.py 2021-10-01 06:23:31 +00:00
dalf a582e55af0 Update searx.data - update_wikidata_units.py 2021-10-01 06:23:25 +00:00
dalf 56c68beb42 Update searx.data - update_firefox_version.py 2021-10-01 06:23:14 +00:00
dalf 453114fbe4 Update searx.data - update_ahmia_blacklist.py 2021-10-01 06:23:12 +00:00
dalf 0c24a03f05 Update searx.data - update_currencies.py 2021-10-01 06:23:04 +00:00
Alexandre Flament bfd24d1226 [mod] add /engine_descriptions.json endpoint 
returns engine descriptions (JSON):
* key: engine name
* value: description in the user locale, use English description as a fallback
 2021-09-24 20:20:30 +02:00
Markus Heiser 065b4dab56
Merge pull request #278 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-09-01 06:48:55 +00:00
Markus Heiser a7a879181d
Merge pull request #279 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-09-01 06:46:29 +00:00
Markus Heiser 00a51fa17f
Merge pull request #280 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-09-01 06:44:22 +00:00
dalf 9e3113865d Update searx.data - update_firefox_version.py 2021-09-01 06:23:44 +00:00
dalf a45fc66e83 Update searx.data - update_currencies.py 2021-09-01 06:23:35 +00:00
dalf 44f7e1fcaa Update searx.data - update_ahmia_blacklist.py 2021-09-01 06:23:20 +00:00
dalf a556d0c1aa Update searx.data - update_wikidata_units.py 2021-09-01 06:23:11 +00:00
Markus Heiser e0ca749ff5
Merge pull request #242 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-08-02 07:40:19 +00:00
Alexandre Flament b6711ba0ad
Merge pull request #241 from searxng/update_data_update_languages.py 
Update searx.data - update_languages.py
 2021-08-02 09:01:57 +02:00
Markus Heiser 0fc1dcbfa5
Merge pull request #243 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-08-01 09:42:01 +00:00
dalf 7fae5647d3 Update searx.data - update_wikidata_units.py 2021-08-01 06:24:23 +00:00
dalf 2963c83daa Update searx.data - update_currencies.py 2021-08-01 06:24:13 +00:00
dalf 4d3f2f48d2 Update searx.data - update_languages.py 2021-08-01 06:24:12 +00:00
dalf f5389fd722 Update searx.data - update_firefox_version.py 2021-08-01 06:24:01 +00:00
Markus Heiser ef6e1bd6b9 [fix] Qwant engines - implement API v3 and add 'quant videos' 
The implementation uses the Qwant API (https://api.qwant.com/v3). The API is
undocumented but can be reverse engineered by reading the network log of
https://www.qwant.com/ queries.

This implementation is used by different qwant engines in the settings.yml::

  - name: qwant
    categories: general
    ...
  - name: qwant news
    categories: news
    ...
  - name: qwant images
    categories: images
    ...
  - name: qwant videos
    categories: videos
    ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-14 09:47:32 +02:00
Alexandre Flament 961dd287a1
Merge pull request #185 from searxng/update_data_update_external_bangs.py 
Update searx.data - update_external_bangs.py
 2021-07-01 18:43:18 +02:00
Alexandre Flament 508195a850
Merge pull request #186 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-07-01 18:41:54 +02:00
Alexandre Flament 89a27f4f96
Merge pull request #187 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-07-01 18:41:43 +02:00
dalf 973d507410 Update searx.data - update_ahmia_blacklist.py 2021-07-01 06:22:48 +00:00
dalf f6a8a5d017 Update searx.data - update_firefox_version.py 2021-07-01 06:22:48 +00:00
dalf 77a432cc95 Update searx.data - update_wikidata_units.py 2021-07-01 06:22:43 +00:00
dalf 726ad776a3 Update searx.data - update_external_bangs.py 2021-07-01 06:22:24 +00:00
Markus Heiser 5cf1ae2672 [pylint] searx/data/__init__.py 
BTW: add doc strings and moved __all__ to the top [1]

[1] https://www.python.org/dev/peps/pep-0008/#module-level-dunder-names

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-06-09 18:08:23 +02:00
Alexandre Flament 2e5d823162 [fix] searx/data/__init__.py: rename __init__ as __all__ 2021-06-09 18:08:23 +02:00
Alexandre Flament c75425655f [enh] openstreetmap / map template: improve results 
implements ideas described in #69

* update the engine
* use wikidata
* update map.html template
 2021-06-09 18:08:23 +02:00
Alexandre Flament cf2b549967 Update engines_languages.json 2021-06-04 16:17:22 +02:00
Alexandre Flament 925c21ce42
Merge pull request #122 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-06-01 10:03:50 +02:00
Alexandre Flament 3266738e7a
Merge pull request #121 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-06-01 10:03:35 +02:00
Alexandre Flament 54ae95b95b
Merge pull request #120 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-06-01 10:03:22 +02:00
dalf 43d06e8224 Update searx.data - update_ahmia_blacklist.py 2021-06-01 08:01:50 +00:00
dalf e53ed4742c Update searx.data - update_currencies.py 2021-06-01 08:01:48 +00:00
dalf 666a25314a Update searx.data - update_wikidata_units.py 2021-06-01 08:01:45 +00:00
dalf 9f3a455777 Update searx.data - update_firefox_version.py 2021-06-01 08:01:33 +00:00
Marc Abonce Seguin 448bfe6005 fix Qwant's fetch_languages function 2021-05-02 17:46:40 -07:00
Alexandre Flament 2756e170ce
Merge pull request #49 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-05-01 10:28:32 +02:00
Alexandre Flament a3a7be90ff
Merge pull request #50 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-05-01 10:28:22 +02:00
Alexandre Flament 4558c0c027
Merge pull request #48 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-05-01 10:28:11 +02:00
dalf dd8a5a00dc Update searx.data - update_currencies.py 2021-05-01 08:23:05 +00:00
dalf de54209f9e Update searx.data - update_ahmia_blacklist.py 2021-05-01 08:23:00 +00:00
dalf 982a5a7d2e Update searx.data - update_wikidata_units.py 2021-05-01 08:22:48 +00:00
dalf c004712eef Update searx.data - update_firefox_version.py 2021-05-01 08:22:43 +00:00
Noémi Ványi 7612e5d9fd
Merge pull request #2712 from searx/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-04-03 23:15:43 +02:00
Noémi Ványi 985e8b28fe
Merge pull request #2710 from searx/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-04-03 23:02:44 +02:00
Noémi Ványi feb2d81874
Merge pull request #2709 from searx/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-04-03 23:02:26 +02:00
Noémi Ványi 9a962c5369
Merge pull request #2711 from searx/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-04-03 23:01:43 +02:00
dalf c0668d248e Update searx.data - update_languages.py 2021-04-01 07:00:46 +00:00
dalf ad74c42aa1 Update searx.data - update_firefox_version.py 2021-04-01 07:00:39 +00:00
dalf 23dc7ef6d1 Update searx.data - update_wikidata_units.py 2021-04-01 07:00:37 +00:00
dalf eb5cd7a543 Update searx.data - update_ahmia_blacklist.py 2021-04-01 07:00:33 +00:00
dalf f804f54ca3 Update searx.data - update_currencies.py 2021-04-01 07:00:29 +00:00
Alexandre Flament 87f4cc4a9e
Merge pull request #2631 from searx/update_data_update_languages.py 
Update searx.data - update_languages.py
 2021-03-06 10:03:00 +01:00
Alexandre Flament 0165e14a7f
Merge pull request #2632 from searx/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-03-05 11:59:44 +01:00
Alexandre Flament 152f6fc1da
Merge pull request #2630 from searx/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-03-05 11:59:20 +01:00
dalf 1e8b846954 Update searx.data - update_currencies.py 2021-03-05 10:56:57 +00:00
dalf 2f8a708481 Update searx.data - update_wikidata_units.py 2021-03-05 10:56:49 +00:00
dalf d9dc3376d0 Update searx.data - update_languages.py 2021-03-05 10:56:46 +00:00
dalf 2857473553 Update searx.data - update_ahmia_blacklist.py 2021-03-05 10:56:33 +00:00
Markus Heiser d48e2e7b0b [enh] google scholar - python implementation of the engine 
The old xpath configuration for google scholar did not work and is replaced by a
python implementation.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-03-01 15:16:37 +01:00
Alexandre Flament 0c663e25fc
Merge pull request #2604 from searx/update_data_firefox_version 
Update searx.data - firefox_version
 2021-03-01 10:03:39 +01:00
Alexandre Flament b05f4d0664
Merge pull request #2605 from searx/update_data_currencies 
Update searx.data - currencies
 2021-02-26 12:56:32 +01:00
Alexandre Flament aec5188b51
Merge pull request #2606 from searx/update_data_wikidata_units 
Update searx.data - wikidata_units
 2021-02-26 12:55:51 +01:00
dalf 893b6e4901 Update searx.data - ahmia_blacklist 2021-02-26 08:31:15 +00:00
dalf 7b9005df31 Update searx.data - wikidata_units 2021-02-26 08:31:01 +00:00
dalf 4c8ae5b7ed Update searx.data - firefox_version 2021-02-26 08:30:45 +00:00
dalf d2778b5efe Update searx.data - currencies 2021-02-26 08:30:45 +00:00
Alexandre Flament 7c1847d5f2 [mod] add utils/fetch_external_bangs.py 
Based on duckduckgo bangs
Store bangs on a trie to allow autocomplete (not in this commit)
 2021-02-24 18:48:36 +01:00
Alexandre Flament 5f4a085fc4
Merge pull request #2595 from dalf/update-wikidata-units 
[mod] update wikidata_units.json and fetch_wikidata_units.py
 2021-02-23 17:22:37 +01:00
Alexandre Flament 46ca32c3cc [mod] update currencies.json and fetch_currencies.py 
use a sparql request on wikidata to get the list of currencies.

currencies.json contains the translation for all supported searx languages.

Supersede #993
 2021-02-23 16:42:28 +01:00
Alexandre Flament 93d1da4906 [mod] update wikidata_units.json and fetch_wikidata_units.py 
The fetch_wikidata_units.py result won't change randomly.
See comments in the script.
 2021-02-23 13:10:38 +01:00
Markus Heiser ff6804e545 [data] make engines.languages 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-01-24 09:52:32 +01:00
Alexandre Flament f5c3cb7afa [mod] drop Python 3.5 support 2021-01-12 09:45:16 +01:00
Markus Heiser 4de276e364 [upd] make SEARX_DEBUG=1 useragents.update 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-12-22 14:23:58 +01:00
Alexandre Flament ed6696e6bf [mod] add external_urls.json and wikidata_units.json 2020-10-28 08:09:25 +01:00
Alexandre Flament 5e7060053c [mod] ahmia_filter.py: minor changes 
- use result['parsed_url']
- load ahmia_blacklist.txt in searx.datae
 2020-10-27 20:00:04 +01:00
Marc Abonce Seguin 32957cdf49 add Ahmia filter plugin for onion results 2020-10-25 17:59:43 -07:00
Alexandre Flament a9dc54bebc [mod] Add searx.data module 
Instead of loading the data/*.json in different location,
load these files in the new searx.data module.
 2020-10-07 10:29:34 +02:00
Marc Abonce Seguin ecf5899153 fetch google's search langs rather than ui langs 2020-09-22 11:37:44 +02:00
Marc Abonce Seguin c86504b47a fix utils/fetch_languages to work with new languages 2020-09-22 11:37:44 +02:00
Michael Ilsaas a1ce141c99
add peertube engine (#2109) 2020-08-08 19:22:53 +02:00
Dalf b339b8b6d6 [mod] bangs.json: remove the favicons 
fix #2076
 2020-07-15 09:15:26 +02:00
Lukas van den Berk 4829a76aae
Created new plugin type custom_results. Added new plugin bang_redirect (#2027) 
* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Refactored getting search query. Also changed bang operator to ! and is now working.

* Removed prints

* Removed temporary bangs_redirect.js file. Updated plugin documentation

* Added unit test for the bangs plugin

* Fixed a unit test and added 2 more for bangs plugin

* Changed back to default settings.yml

* Added myself to AUTHORS.rst

* Refacored working of custom plugin.

* Refactored _get_bangs_data from list to dict to improve search speed.

* Decoupled bangs plugin from webserver with redirect_url

* Refactored bangs unit tests

* Fixed unit test bangs. Removed dubbel parsing in bangs.py

* Removed a dumb print statement

* Refactored bangs plugin to core engine.

* Removed bangs plugin.

* Refactored external bangs unit tests from plugin to core.

* Removed custom_results/bangs documentation from plugins.rst

* Added newline in settings.yml so the PR stays clean.

* Changed searx/plugins/__init__.py back to the old file

* Removed newline search.py

* Refactored get_external_bang_operator from utils to external_bang.py

* Removed unnecessary import form test_plugins.py

* Removed _parseExternalBang and _isExternalBang from query.py

* Removed get_external_bang_operator since it was not necessary

* Simplified external_bang.py

* Simplified external_bang.py

* Moved external_bangs unit tests to test_webapp.py. Fixed return in search with external_bang

* Refactored query parsing to unicode to support python2

* Refactored query parsing to unicode to support python2

* Refactored bangs plugin to core engine.

* Refactored search parameter to search_query in external_bang.py
 2020-07-03 13:25:04 +00:00
Émilien Devos 7a50522a82
Update default user agents 2020-04-21 20:50:16 +00:00
Markus Heiser 809e6f7933 make project: re-build generic files of the searx project 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-26 13:35:17 +01:00
Markus Heiser 79412828cd update languages: engines_languages.json languages.py 
build by Makefile target:

    make project

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-01 08:06:51 +01:00
Markus Heiser 40843fe95a searx/data/engines_languages.json: sort json file (no content change) 
To get meaningfull diffs, the json file has to be sorted.  Before applying any
further content patch, the json file needs a inital sort (without changing any
content).

Sorted by::

  import sys, json

  with open('engines_languages.json') as f:
      j = json.load(f)

  with open('engines_languages.json', 'w') as f:
      json.dump(j, f, indent=2, sort_keys=True)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2020-03-01 07:56:46 +01:00
Marc Abonce Seguin d0d91fa331 update user agents 2019-11-23 19:27:49 -07:00
Marc Abonce Seguin 94ca30de1b add a shortcut for utils/fetch_firefox_version.py in manage.sh 2019-11-23 19:26:48 -07:00
Marc Abonce Seguin 0169b63e84 [fix] fetch google's supported languages 2019-01-06 21:31:45 -06:00
Ivan Skytte Jørgensen 0b7f0bde62 Use a more human-friendly format in searx/data/engines_languages.json (#1399) 
Instead of a single line with 500000 characters use nicely formatted JSON.
Sort the lists in engine_languages.py so when updating it is possible to
more easily see the differences (search engines do change the order their
languages are listed in)
 2018-11-29 16:31:05 +01:00
Alex 50c836864a fetch_firefox_version.py : compatible with Python 3 and minor fixes. 2018-08-05 10:55:42 +02:00
Alexandre Flament 066bd916bf [mod] fetch firefox versions in a standalone script 2018-08-05 10:10:15 +02:00
rinpatch a305dae9d8 switched currency convert to duckduckgo and added Bitcoin to currency list 2018-04-22 13:46:13 +03:00
Marc Abonce Seguin 772c048d01 refactor engine's search language handling 
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
 2018-03-27 00:08:03 -06:00
Marc Abonce Seguin d1eae9359f fix fetch_langauges to be more accurate 
Add languages supported by either all default general engines or 10 engines.
 2018-03-20 17:58:20 -06:00
marc 44085e31d0 update engines_languages.json and languages.py 
Also, fix fetch_languages.py so it can run on python3.
 2017-10-10 16:53:28 -05:00
marc 405e5c8f24 [fix] duckduckgo images doesn't fail with countryless language 2017-07-10 18:41:50 +02:00
marc fd65c12921 make search language handling less strict 
languages.py can change, so users may query on a language that is not
on the list anymore, even if it is still recognized by a few engines.

also made no and nb the same because they seem to return the same,
though most engines will only support one or the other.
 2017-03-18 23:44:21 +01:00
marc 805fb02ed1 add language support for qwant 
closes issue #863
 2017-03-18 23:44:21 +01:00
marc 1175b3906f change language list to only include languages with a minimum of engines 
that support them.
users can still query lesser supported through the :lang_code bang.
 2016-12-29 01:55:30 -06:00
marc 4a1ff56389 minor fixes in utils/fetch_languages.py 2016-12-16 22:14:14 -06:00
marc af35eee10b tests for _fetch_supported_languages in engines 
and refactor method to make it testable without making requests
 2016-12-15 00:40:21 -06:00
marc f62ce21f50 [mod] fetch supported languages for several engines 
utils/fetch_languages.py gets languages supported by each engine and
generates engines_languages.json with each engine's supported language.
 2016-12-13 19:58:10 -06:00
Alexandre Flament 5525625dae [enh] currency_convert engine: user can write "1 dollars in euros" instead of "1 USD in EUR". 
The currency names are fetched wikidata and store into a static file : searx/data/currencies.json
This file is loaded when the currency_converter is loaded.
A database is perhaps more appropiated.
 2015-05-12 20:52:08 +02:00