597c68b4aa
[doc] move dosc of botdetection from developer to admin section
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-23 14:00:03 +02:00
75c9de02d1
[feat] engine: implementation of imgur
2023-09-22 20:50:53 +02:00
fb72f71f0a
[fix] internet archive scholar: crash when there's no title
2023-09-22 18:49:39 +02:00
0660d20e1d
[translations] update from Weblate
...
f27a3bbd4a97336d0a91e8946b319f53731edaf10bd75ab004b8b8cdedd99c9
2023-09-22 18:05:02 +02:00
95885e7f9f
[build] /static
2023-09-22 17:54:55 +02:00
8b0b6731fc
[fix] search.less: start page search view on mobile devices
2023-09-22 17:54:55 +02:00
71358e9c67
Revert "[fix] engine - duckduckgo vqd edge-case"
...
This reverts commit 102502a4f0
2023-09-22 09:31:25 +02:00
51236ae47a
[feat] engine: implementation of chefkoch.de
2023-09-21 17:23:59 +02:00
8bcca0e620
[fix] engine - brave don't show ads
2023-09-21 16:55:39 +02:00
b729542a66
[fix] engine - google images error when no results
2023-09-21 16:38:37 +02:00
f0df53673e
[mod] search-syntax.md: add tutorial about the feeling lucky feature
2023-09-21 15:17:23 +02:00
cc2e0537a3
[feat] engine: implementation of google icons/material design icons
2023-09-21 15:16:49 +02:00
c999cfb422
[feat] engine: implementation of wallhaven
2023-09-21 14:25:43 +02:00
102502a4f0
[fix] engine - duckduckgo vqd edge-case
2023-09-20 20:05:06 +02:00
d013f51a25
[fix] Search on category select without JavaScript PR #2740
...
Small addendum to #2740 ; search_on_category_select is now no longer a plugin.
Related: https://github.com/searxng/searxng/pull/2740#issuecomment-1725437584
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-19 21:03:02 +02:00
0ce10046c4
[build] /static
2023-09-19 20:15:51 +02:00
a76c8e4101
Improve the layout of the search page on mobile
2023-09-19 20:15:51 +02:00
33496e7f51
Switch back to qwant web normal API
2023-09-19 17:07:22 +02:00
043dcbf7c5
[fix] engine qwant (web-lite) - ignore advertising adds
...
Closes: https://github.com/searxng/searxng/issues/2812
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-19 17:06:56 +02:00
3ac7c40b6a
[build] /static
2023-09-19 11:15:25 +02:00
33e722f83b
better error message when no results found
2023-09-19 11:10:48 +02:00
ad725ce7d7
wikipedia wikidata infobox + disable wikisource ( #2806 )
...
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-19 10:31:02 +02:00
dcee823345
[feat] implement feeling lucky feature
2023-09-19 09:40:57 +02:00
71508abcbf
[build] /static
2023-09-19 08:06:06 +02:00
2ec77ef813
[mod] reformatted keyBindingLayouts in JS client (default, vim)
2023-09-19 08:06:06 +02:00
317db5b04f
[mod] preferences: implement drop-down menu for hotkeys (default, vim)
...
Replace the on/off checkbox of the vim-hotkeys in the preferences by a drop-down
menu.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-19 08:06:06 +02:00
09935e2897
[build] /static
2023-09-18 21:29:11 +02:00
a55e0ac553
[feat] search on category select without JS
...
Co-authored-by: Alexandre Flament <alex@al-f.net>
2023-09-18 21:29:11 +02:00
efd3a2d6d1
[feat] engine: implementation of internet archive scholar
2023-09-18 18:12:00 +02:00
223b3487c3
[fix] spelling
2023-09-18 16:20:27 +02:00
a9b6963971
[fix] engine - qwant delivers only 5 pages maximum
...
all qwant engines (incl qwant-lite) delivers only 5 pages maximum
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-18 01:43:43 +02:00
da1446c5ed
[fix] engine - qwant wrong error type
2023-09-18 01:43:43 +02:00
7398d525c8
[fix] qwant: subsequent fix of commit d9dbcedeb
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-17 17:52:56 +02:00
d9dbcedeb6
[feat] implementation of qwant lite for web search
...
Related: https://github.com/searxng/searxng/issues/2719
Replace: https://github.com/searxng/searxng/pull/2748
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-17 16:53:25 +02:00
b4e0d2eedc
[feat] engine: implemenation of moviepilot (de)
2023-09-17 14:30:56 +02:00
7dfcc3386e
[fix] tagesschau videos
2023-09-16 18:40:26 +02:00
89fbac5763
[build] /static
2023-09-16 13:45:43 +02:00
92d0c378e0
[fix] keyboard navigation / simple theme (UI)
...
- avoid loop select
- fix select next item in mixed result lists
Replaces: https://github.com/searxng/searxng/pull/2789
Closes: https://github.com/searxng/searxng/issues/2751
Closes: https://github.com/searxng/searxng/issues/2788
2023-09-16 13:45:43 +02:00
ec540a967a
[fix] brave.videos
2023-09-15 22:00:09 +02:00
27477f51fd
[fix] brave.news
2023-09-15 22:00:09 +02:00
140125e641
[feat] implementation of the movie database
2023-09-15 19:41:37 +02:00
4eebb25882
[translations] update from Weblate
...
4f055ccb89edaacb09ccb3e8934206640415d8db608c26db95be680792cd970949ad5db80921f948e3677f0ccb2dea9cd3a5038588e86b63e980b858a8660b5852633d292d25097b8ae9d7a20989ccd21a1c309f3fc3ee5e679e7420db0cf7327ffb1f542814e225f26236007bea3da4568a854d931781efe60016a6044200f1431198e61bb7acd04327a418412274159ed21b385914b774daff7b45ec84146f8707383602051b050d9ee48d6e7a20cee77e20720420f44b6
2023-09-15 09:12:13 +02:00
41ef73ca3a
[fix] rumble redirect
2023-09-14 19:21:21 +02:00
b701ee257b
[fix] habrahabr
2023-09-14 18:43:14 +02:00
be888810ba
[fix] pubmed content being None
2023-09-14 18:40:15 +02:00
cf734e6008
[fix] engine - pub.dev
2023-09-14 18:07:55 +02:00
92d39de410
[fix] solidtorrents redirects
2023-09-14 18:03:21 +02:00
cf8a6cf6db
[fix] solidtorrents pagination
2023-09-14 18:03:21 +02:00
8172f89075
[fix] solidtorrents
2023-09-14 18:03:21 +02:00
74600c028d
[fix] engine - Crossref
...
Crossref was broken on result types journal-issue and component .. The old code
had lots of assumptions, and broke during parsing. Now the assumptions are more
explicit and checked them with the API.
2023-09-14 17:39:23 +02:00
ed6a5a01bb
[build] /static
2023-09-14 16:26:10 +02:00
0984b00b7e
[mod] UI smooth scroll added / CSS scroll-behavior
...
This PR improves the UX by making auto-scroll more smoother. The css is changed
so all the auto-scroll will be smoother but User-scroll will not be influenced.
The scroll-behavior CSS property sets the behavior for a scrolling box when
scrolling is triggered by the navigation or CSSOM scrolling APIs.[1]
[1] https://developer.mozilla.org/en-US/docs/Web/CSS/scroll-behavior
2023-09-14 16:26:10 +02:00
bf7af99ef2
[mod] make data.traits: add Odysee
2023-09-13 21:41:33 +02:00
3568a3cafb
[feat] odysee: implement fetch_traits for language support
2023-09-13 21:41:33 +02:00
09c61dabc9
[mod] odysee: time range support
2023-09-13 21:41:33 +02:00
b98907e91f
[fix] engine - piped.music incorrect timestamps
2023-09-13 21:39:37 +02:00
6039dbf211
[fix] engine - invidious thumbnails
2023-09-13 11:37:42 +02:00
ed3819bdd5
add inv.tux.pizza
2023-09-13 11:12:01 +02:00
122b4342eb
remove invidious.snopyta.org
2023-09-13 11:12:01 +02:00
e2a5e183cc
[fix] settings.yml - change default invidious instances
2023-09-13 11:12:01 +02:00
b2fd6304bf
[fix] engine - openstreetmap currency rendering
2023-09-13 10:56:52 +02:00
976b3e21c4
[fix] engine - pypi redirect
2023-09-12 21:14:48 +02:00
54a3e03b45
[fix] engine - openstreetmap currency matching
2023-09-12 20:57:05 +02:00
64d9587ac8
[feat] new engine: svgrepo
2023-09-12 20:38:36 +02:00
aa1453db0c
[build] /static
2023-09-12 20:30:50 +02:00
a987672340
[mod] improve handling of the hotkeys
...
- KeyboardEvent: keyCode property is depricated, replaced by key property [2]
- the check for ifDetailOpened is not necessary, because the hotkeys are not
only applicable to image-results, by example:
`!goi !go !scc hello`
- Key bindings like h for help are to be used in general (not only in vim-mode)
[1] https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/keyCode
[2] https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/key
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-12 20:30:50 +02:00
e6b160da62
[feature] key bindings: left & right arrow are always on
2023-09-12 20:30:50 +02:00
b189578b6b
[fix] engine - brave
2023-09-12 11:31:43 +02:00
f182abd6f8
[mod] library of congress: fix engine
2023-09-11 19:42:31 +02:00
e73a6f5d14
[fix] engine deviantart: review of the result-scrapper
...
The deviantart site changed and hence deviantart is currently unusable.
2023-09-11 13:22:36 +02:00
1f7366060e
[build] /static
2023-09-11 12:27:56 +02:00
935aed7ca4
[feature] dark theme for code highlighter in the result list
...
Closes: https://github.com/searxng/searxng/issues/1354
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-11 12:27:56 +02:00
d07c006aed
Replace chompjs with pure Python code
...
The new implementation is good enough for the current usage (brave)
2023-09-09 13:02:36 +02:00
8e45ac4271
[build] /static
2023-09-09 00:39:51 +02:00
15a3dbf7a0
[fix] preferences - layout of the "special queries" tab
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-09 00:39:51 +02:00
9e83c0dedc
[feat] engine: implementation of Yummly
...
Co-authored-by: Markus Heiser <markus.heiser@damarit.de>
2023-09-08 11:47:13 +02:00
a3d7e9c285
[mod] utils.py: add markdown_to_text helper function
2023-09-08 11:47:13 +02:00
c62605b3e3
[translations] update from Weblate
...
304f6af5c76a4b6496
2023-09-08 09:20:19 +02:00
019bd97f4b
[fix] settings.yml - fix docs-links in the comments
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-06 11:35:39 +02:00
1f21ac7d62
[feat] engine: implementation of bilibili https://www.bilibili.com
2023-09-05 22:53:03 +02:00
696c35d2c3
[fix] engine - duckduckgo_images / determination of vqd value incorrect
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-05 22:24:51 +02:00
4fb26cd96d
[fix] engine duckduckgo weather api changes
2023-09-05 16:55:00 +02:00
01be9e0e20
[fix] engine: wikicommons - don't quoute ':|' in URL parameters
...
From [1]: It seems to be because of [2] For some reason it gets url encoded
twice, resulting in
- ``filetype%253Abitmap%257Cdrawing+birds`` instead of
- ``filetype:bitmap%7Cdrawing+birds``
[1] https://github.com/searxng/searxng/issues/2707
[2] https://github.com/searxng/searxng/blob/master/searx/engines/wikicommons.py#L43
Closes : #2707
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-03 18:45:39 +02:00
4f8895c6de
[fix] follow-up of 4da7003ae / add missing review from @Bnyro
...
[1] https://github.com/searxng/searxng/pull/2656#pullrequestreview-1607956209
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-02 09:26:45 +02:00
4da7003ae0
[feat] engine: implementation of odysee
2023-09-02 09:14:12 +02:00
9c4e9d3814
[feat] implementation of Wikimedia commons for images
2023-09-01 18:39:24 +02:00
9fce459c7f
[translations] update from Weblate
...
979909accf40af3403
2023-09-01 09:29:07 +02:00
7af66736c0
Create search-syntax.md fa
2023-08-30 16:39:10 +02:00
26cd770cf7
[mod] Persian translation of the about page (fa_IR)
2023-08-30 16:13:44 +02:00
faa4280e1a
[mod] bing: resolve redirect without additional requests
...
Remove the usage of searx.network.multi_requests
The results from Bing contains the target URL encoded in base64
See the u parameter, remove the first two character "a1", and done.
Also add a comment the check of the result_len / pageno
( from https://github.com/searx/searx/pull/1387 )
2023-08-29 07:39:06 +02:00
f57842b05d
Update searx.data - update_firefox_version.py
2023-08-29 07:21:46 +02:00
a5cd777350
Update searx.data - update_ahmia_blacklist.py
2023-08-29 07:21:21 +02:00
f361a10af6
Update searx.data - update_currencies.py
2023-08-29 07:20:42 +02:00
978062bdbe
Update searx.data - update_engine_descriptions.py
2023-08-29 07:20:06 +02:00
f59fd8dc9b
Update searx.data - update_wikidata_units.py
2023-08-29 07:17:44 +02:00
3294e7cfe4
[fix] reduce error logs related to X-Forwarded-For & X-Real-IP headers
...
To test start a `make run` instance and query multiple times for `ip` (the
answerer), you should see only once the error message(s):
ERROR searx.botdetection : X-Forwarded-For header is not set!
ERROR searx.botdetection : X-Real-IP header is not set!
[1] https://github.com/searxng/searxng-docker/discussions/174
[2] https://github.com/searxng/searxng-docker/pull/162#issuecomment-1692834457
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-28 08:05:17 +02:00
1428385d1c
[mod] preferences.py: add and use BooleanSetting for checkbox preferences
2023-08-27 21:23:43 +02:00
e5535ec078
[feat] settings: replace boolean select preferences with checkboxes
2023-08-27 21:23:43 +02:00
5a5cfc1930
[mod] random answerer: add random hex color generator
2023-08-27 18:59:54 +02:00
b0d2cd5ca9
[doc] add documentation of Mwmbl engine & autocompleter
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-27 17:25:26 +02:00
df71c24b20
[mod] autocomplete.py: add support for mwmbl completions
2023-08-27 17:25:26 +02:00
19300a5659
[mod] engine mwmbl: add link to official api docs
2023-08-27 17:25:26 +02:00
97b1df1629
[mod] searx.network: memory optimization
...
Avoid to create a SSLContext in AsyncHTTPTransportNoHttp
See:
* 0f61aa58d6/httpx/_transports/default.py (L271)https://github.com/encode/httpx/issues/2298
2023-08-27 11:49:40 +02:00
e16c007c22
[fix] openstreetmap engine
...
It seems there is an API change:
extratags can be either a dictionnary or None.
This commit avoid crash when extratags is None
Test query "!osm gare du nord"
2023-08-27 11:49:16 +02:00
0647f83a3e
[fix] google engine: don't overspecify the search query to Google
...
The method EngineTraits.get_region(..) returns engine's region string
that **best fits** to SearXNG's locale. This means it returns a
region (country) if only a language is set in the locale. By example the method
returns for a locale tag `es` a region `ES`.
Google's search parameter `cr` restricts search results to documents originating
in a particular country / in case of a locale tag (language) as described above,
this argument should be unset in the query send to Google.
Closes: https://github.com/searxng/searxng/issues/2672
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-26 07:47:07 +02:00
7d806d1020
[translations] update from Weblate
...
a6c2d198838bc33a710ff21aa3c
2023-08-25 09:36:49 +02:00
9069da051a
Merge pull request #2644 from dalf/update_httpx
...
Bump httpx 0.21.2 from to 0.24.1
2023-08-24 23:12:06 +02:00
4b42644579
[fix] engine google_video: google has changed the layout of the rsponse
...
Closes: https://github.com/searxng/searxng/issues/2664
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-22 08:34:04 +02:00
b4e4cfc026
Bump httpx 0.21.2 from to 0.24.1
2023-08-21 22:05:12 +02:00
d52919cc3e
[mod] decrease the weighting of wikivoyage, wikibooks, wikisource and wikiversity
...
The search engines deliver hits for many search terms [1], but these are usually
not the focus of the user. In order to arrange these hits further down in the
list, their weighting is reduced.
[1] https://github.com/searxng/searxng/pull/2589#issuecomment-1670915089
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-21 21:52:40 +02:00
018b0a9322
[feat] engine: XPath configuration for pkg.go.dev
2023-08-19 19:19:31 +02:00
c59ae91b76
[feat] engine: implementation of mwmbl
2023-08-19 18:23:42 +02:00
bb9e9ab773
[translations] update from Weblate
...
d8e4100b6b8e03f3bc
2023-08-18 19:22:13 +02:00
c741fc6f00
[mod] currency_convert: support for showing the answer source url
...
Show URL of the ddg-search page, not the URL of a (generic) Javascript. The
latter one is not usefull for the user.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-18 19:07:14 +02:00
e2744520f8
[mod] google: support for showing the answer source url
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-18 19:07:14 +02:00
5ec7df3480
[mod] engine duckduckgo definitions: support for answer source
2023-08-18 19:07:14 +02:00
64bc98b5fb
[mod] brave: support for showing the answer source url
2023-08-18 19:07:14 +02:00
6fc2ee514d
[build] /static
2023-08-18 19:07:14 +02:00
13d801b75d
[feat] results: show source of suggested answer
2023-08-18 19:07:14 +02:00
9100a48541
[mod] improve seekr engines and add documentation
...
Tis patch adds some more fields to the result items and changed paging to the
``nextResultSet`` given in seekr's JSON response.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-15 16:17:42 +02:00
2bab658d39
[feat] engine: implementation of seekr for news, images and videos
2023-08-15 16:17:42 +02:00
7052a1a7cf
[drop] engine archive.is / blocked by CAPTCHA [1]
...
Sadly archive.is is blocked by a CAPTCHA that can't be avoid (at least in a
XPath engine).
[1] https://github.com/searxng/searxng/issues/2643
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-14 08:50:13 +02:00
621b09b9b9
[fix] preferences: change engine table category colspan to 8
...
* this is a small fix to increase the colspan of the category in engine preferences from 7 to 8, since there was a column added
=> fixing a small fallout from 4731290317
2023-08-14 07:11:06 +02:00
f175574f37
[fix] typos in documentation & messages
2023-08-13 08:50:29 +02:00
c348427f07
[update] make pygments.less
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-11 10:54:50 +02:00
99df7b84d1
[translations] update from Weblate
...
12c527bcdea46b66fcae3d8dcdfab6a59e351db1affcc94b682aa007c49bb6c
2023-08-11 09:17:56 +02:00
c76c999a1e
[feat] engine: implementation of anaconda
2023-08-10 23:51:07 +02:00
e25d1c7288
[feat] engine: implementation of German news, Tagesschau
...
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-10 20:27:54 +02:00
2256ba2ffb
[mod] settings.yml: move wolframalpha to categ general & disable by default
...
As suggested in comment [1] as a good compromise between [2] and [3].
This will help users that want Wolfram Alpha instant answers by default with
every query and don't mind the slowness. It will be disabled by default so it's
fast for all users, and those that need it can enable it as a user-configurable
option without requiring setting up their own instance to change the Wolfram
Alpha categorization.
[1] https://github.com/searxng/searxng/pull/813#issuecomment-1574851572
[2] https://github.com/searxng/searxng/pull/813#issuecomment-1020581765
[3] https://github.com/searxng/searxng/issues/966#issue-1171107451
2023-08-10 14:31:09 +02:00
834e1c3f12
[mod] engine lemmy: increase thumbnail quality to align with theme
2023-08-10 12:58:40 +02:00
c381fc001f
[mod] settings: remove lemmy from categ 'general' & enable by default
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-10 12:58:40 +02:00
fda111c0c9
[mod] engine lemmy: add more info fields to the result items
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-10 12:58:40 +02:00
224f2250ae
[feat] engine: support for lemmy communities, posts, comments and users
2023-08-10 12:58:40 +02:00
9f82c39610
[mod] engine google_play: raise error on unsupported category
2023-08-10 12:35:24 +02:00
0a99dc85b9
[mod] engine brave: raise error on unsupported category
2023-08-10 12:35:24 +02:00
4731290317
[mod] settings: show weight of each instance
2023-08-10 10:50:14 +02:00
733b795d53
[fix] make flask_babel.gettext() work in engine modules (L10n & threads)
...
incident:
flask_babel.gettext() does not work in the engine modules.
cause:
the request() and response() functions of the engine modules run in the
processor, whose search() method runs in a thread and in the threads the
context of the Flask app does not exist. The context of the Flask app is
needed by the gettext() function for the L10n.
Solution:
copy context of the Flask app into the threads. [1]
special case:
We cannot equip the search() method of the processors with the decorator [1],
because the decorator requires a context (Flask app) that does not yet exist
at the time of the initialization of the processors (the initialization of the
processors is part of the initialization of the Flask app).
[1] https://flask.palletsprojects.com/en/2.3.x/api/#flask.copy_current_request_context
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-09 13:27:43 +02:00
c00c0c5434
[mod] remove discontinued petalsearch engines
2023-08-09 07:17:40 +02:00
b8352eca0c
[mod] brave engines: add fetch_traits() / improve language support
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-08 16:21:45 +02:00
460bbe5b81
[mod] implement brave (WEB) engine to replace XPath configuration
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-08 16:21:45 +02:00
d151497db3
[feat] engine: brave - support for news
2023-08-08 16:21:45 +02:00
cae06f2781
[feat] engine: brave - support for videos
2023-08-08 16:21:45 +02:00
73364e158e
[feat] engine: brave - support for images
2023-08-08 16:21:45 +02:00
bcaaae699f
[doc] fix some minor typos in the doc-strings of class EngineTraits
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-08 08:15:53 +02:00
a2108432f7
[fix] settings.yml: typo in engine name wikipecies -> wikispecies
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-07 10:35:44 +02:00
1d0abb7157
[doc] engine bt4g: add documentation to docs/dev/engines/online/
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-06 09:30:48 +02:00
0fc8f99ecc
[feat] new engine: bt4g added & enabled and disable by default btdigg
...
Disable btdigg because on most SearXNG instances, SearXNG is blocked by btdigg
due to cloudflare too many requests.
This impementation did not parse the HTML page because there is an API in
XML (RSS). The RSS feed provides fewer data like amount of seeders/leechers and
the files in the torrent file. It's a tradeoff for a "stable" engine as the XML
from RSS content will change way less than the HTML page.
Closes: https://github.com/searxng/searxng/issues/2553
2023-08-06 09:30:48 +02:00
dad16c0949
[mod] settings.yml: enable engines from Wikimedia Foundation
...
The Wikimedia wikis [1] engines provide good answers and have short response
times --> no reason to disable these enhgines by default. BTW: this patch adds
a (sub-) category ``wikimedia`` for the engines [1].
[1] https://meta.wikimedia.org/wiki/Wikimedia_wikis
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-04 19:06:50 +02:00
db522cf76d
[mod] engine: wikimedia - improve results, add addition settings & doc
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-04 19:06:50 +02:00
7d8c20c80d
[feat] new engine: wikispecies
2023-08-04 19:06:50 +02:00
9a5e91ce84
[translations] update from Weblate
...
310daeb7201be8a264
2023-08-04 09:30:11 +02:00
1b030d4b41
[doc] engine: Yacy
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-03 19:58:51 +02:00
ca518c6803
add option to change yacy search mode
2023-08-03 19:58:51 +02:00
203f1f0928
[fix] engine piped: 'invalid content'
...
SearXNG does not allow a None value in the content field of a result item.
If the key (shortDescription, uploaderName) in the JSON response from piped
exists but is set to None, SearXNG ignores this result item::
DEBUG searx : result: invalid content: { .., 'content': None, ..}
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-03 16:23:36 +02:00
207fcc0c8c
[mod] engine piped: add paging support
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-03 16:23:36 +02:00
ef5831cd84
[mod] engine piped: split into two dedicated engiens for video & music
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-03 16:23:36 +02:00
7aa95d2d52
[doc] engine piped: add documentation to docs/dev/engines/online/
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-08-03 16:23:36 +02:00
636bfdac68
[feat] engine: implementation of Piped
2023-08-03 16:23:36 +02:00
8b4ba204b7
Update searx.data - update_engine_descriptions.py
2023-07-29 11:02:54 +02:00
d421385dc6
Update searx.data - update_wikidata_units.py
2023-07-29 11:02:23 +02:00
290f16207b
Update searx.data - update_currencies.py
2023-07-29 11:02:14 +02:00
fcd2013511
Update searx.data - update_firefox_version.py
2023-07-29 11:01:28 +02:00
cfa8169509
Update searx.data - update_ahmia_blacklist.py
2023-07-29 09:25:24 +02:00
f45d1920d9
[translations] update from Weblate
...
ba4888c966ec8a8a280a7b701b3c0b34cbdb37cbd41c2
2023-07-29 08:15:21 +02:00
65d8b1a310
[fix] remove disabled: false from engine definitions in settings.yml
...
* setting disabled: false is not needed, since it is by default enabled
2023-07-22 18:19:01 +02:00
51c531d450
[translations] update from Weblate
...
b7f1e9ae3c7c821e0e305f8480745a22afef0e2e4a7687ac
2023-07-22 17:47:05 +02:00
b477349824
[build] /static
2023-07-19 15:07:45 +02:00
de5c1cedca
fix "#backToTop" button always being clickable
...
`pointer-events` never gets set to "none" when the button is hidden,
allowing you to click the button. And your mouse further changes it's
cursor to the pointer style.
2023-07-19 15:06:09 +02:00
b7b184244d
[translations] update from Weblate
...
01350cf15f037a4d820a78ad73037743e656795c0ee18285c087c7fb6eb318c53b4a3d1fb3187499c12266462356a402c9a74b523d9f293868af8585261a2a72fcea15cf7685385eec0a37270130ddf7b93f96094a5cdcb39cba3939d973d937ce0762455c36ccab226ff7d48148a9ed840bc18951ffc22e394ec63e428c16a8218cf51e70260934c6244c2bb92dc5c143917957df1bf630d1c00dff0a6da54f12377e28b5b8ea78ec31e65f6c33b1fe393d390ca4f6b3530f8d6b6b67f2fc965f2d3f025ae2b8dc0bd4fb1ece768726a22ae2f2b5b8774f
2023-07-14 10:21:27 +02:00
cada89ee36
[feat] engine: re-enables z-library (zlibrary-global.se)
...
- re-enables z-library as the new domain zlibrary-global.se is now available
from the open web. The announcement of the domain:
https://www.reddit.com/r/zlibrary/comments/13whe08/mod_note_zlibraryglobalse_domain_is_officially/
It is an official domain, it requires to log in to the "personal" subdomain
only to download files, but the search works.
- changes the result template of zlibrary to paper.html, filling the appropriate fields
- implements language filtering for zlibrary
- implement zlibrary custom filters (engine traits)
- refactor and document the zlibrary engine
2023-07-07 21:36:51 +02:00
cb92767f19
[feat] enigine: add CrowdView forum search engine
2023-07-07 21:36:11 +02:00
4a2f310da3
[translations] update from Weblate
...
152f20089dbf6b224ad4c00f
2023-07-07 21:13:47 +02:00
5720844fcd
[doc] rearranges Settings & Engines docs for better readability
...
We have built up detailed documentation of the *settings* and the *engines* over
the past few years. However, this documentation was still spread over various
chapters and was difficult to navigate in its entirety.
This patch rearranges the Settings & Engines documentation for better
readability.
To review new ordered docs::
make docs.clean docs.live
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-07-01 22:45:19 +02:00
81c9a18456
[translations] update from Weblate
...
2238e87bc70d228a389c0c62656d9fcba9c9b116528b845fb8c50f2339f47c0fae0aa811c8216259
2023-06-30 11:49:07 +02:00
fbb72fc1f4
Update searx.data - update_engine_descriptions.py
2023-06-29 13:59:25 +02:00
87e7926ae9
[fix] engine: Anna's Archive - grep results from '.js-scroll-hidden' elements
...
The renderuing of the WEB page is very strange; except the firts position all
other positions of Anna's result page are enclosed in SGML comments. These
cooments are *uncommented* by some JS code, see query of the class
'.js-scroll-hidden' in Anna's HTML template [1].
[1] https://annas-software.org/AnnaArchivist/annas-archive/-/blob/main/allthethings/templates/macros/md5_list.html
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-29 09:32:57 +02:00
e2df6b77a3
[mod] engine: Anna's Archive - additionl settings (content, sort, ext)
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-29 09:32:57 +02:00
eafc2906f1
[mod] engine: Anna's Archive - fetch search arguments from search form
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-29 09:32:57 +02:00
7adb9090e5
[mod] engine: Anna's Archive - add language support
2023-06-29 09:32:57 +02:00
e5637fe7b9
[feat] engine: implementation of Anna's Archive
...
Anna's Archive [1] is a free non-profit online shadow library metasearch engine
providing access to a variety of book resources (also via IPFS), created by a
team of anonymous archivists [2].
[1] https://annas-archive.org/
[2] https://annas-software.org/AnnaArchivist/annas-archive
2023-06-29 09:32:57 +02:00
fd26f37073
[upd] make data.all
...
- ahmia_blacklist.txt
- currencies.json
- engine_descriptions.json
- engine_traits.json
- osm_keys_tags.json
- useragents.json
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-28 21:21:53 +02:00
efea962504
[fix] simple template: preferences - add missing icon_smal import
...
Related: https://github.com/searxng/searxng/commit/2149e88bdd64#r119535272
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-28 18:36:52 +02:00
401561cb58
[mod] engine torznab - refactor & option to hide links
...
- torznab engine using types and clearer code
- torznab option to hide torrent and magnet links.
- document the torznab engine
- add myself to authors
Closes: https://github.com/searxng/searxng/issues/1124
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-28 10:03:44 +02:00
da7c30291d
[fix] Google API changed
...
It seems that Google is rolling out a modified WEB API [1][2].
In the past there was only the UI language in the `hl` argument but nowadays it
seems a combination of the UI language and the "search region" is mixed in this
argument and the `gl` argument has been removed. I'm very surprised that google
is starting to mix the parameters of the UI with the parameters of the search
index.
This patch modifies the get_google_info(..) function. Beside Google-WEB this
function is also used by other Google services, here are some examples to test
region & language of ..
- Google-WEB: `!go dragon boat :en-CA`
- Google-News: `!gon dragon boat :en-CA`
- Google-Videos: `!gov bmw :en-CA`
- Goolge-Images `!goi bmw :en-CA`
- [1] https://github.com/searxng/searxng/issues/2515#issuecomment-1606294635
- [2] https://github.com/searxng/searxng/issues/2515#issuecomment-1607150817
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-26 18:28:09 +02:00
e8706fb738
[fix] engine & network issues / documentation and type annotations
...
This patch fixes some quirks and issues related to the engines and the network.
Each engine has its own network and this network was broken for the following
engines[1]:
- archlinux
- bing
- dailymotion
- duckduckgo
- google
- peertube
- startpage
- wikipedia
Since the files have been touched anyway, the type annotaions of the engine
modules has also been completed so that error messages from the type checker are
no longer reported.
Related and (partial) fixed issue:
- [1] https://github.com/searxng/searxng/issues/762#issuecomment-1605323861
- [2] https://github.com/searxng/searxng/issues/2513
- [3] https://github.com/searxng/searxng/issues/2515
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-25 13:58:26 +02:00
2e4a435134
[translations] update from Weblate
...
9512b92aca08c51e56ad4f213ee419d6
2023-06-23 09:34:46 +02:00
86db08793b
[fix] implement a JSONEncoder for the json format
...
This patch implements a simple JSONEncoder just to fix #2502 / on the long term
SearXNG needs a data schema for the result items and a json generator for the
result list.
Closes: https://github.com/searxng/searxng/issues/2505
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-19 19:49:44 +02:00
fa1ef9a07b
[mod] move some code from webapp module to webutils module (no functional change)
...
Over the years the webapp module became more and more a mess. To improve the
modulaization a little this patch moves some implementations from the webapp
module to webutils module.
HINT: this patch brings non functional change
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-19 19:49:44 +02:00
71b6ff07ca
[translations] update from Weblate
...
98f61c70a1679b93ebd1d574b28a1da356409bf0abc4916cb1900abeb48e84c4bf395e32c9c0a3c93f50d31e9da1c142
2023-06-16 09:20:43 +02:00
1be27d5d83
[translations] update from Weblate
...
b40da1a3666ee7d41e0e8ead404b9937a627f9a1a234d2f8cc41f9b524651eacc37b06279a435ea140e0adad6833b14200f397ad7d3d4a97f7d713a4b1ec316004591a3acb3ac67cfe81dbc77882670f38882f3bc6df50476ca23c3b72f1ee09
2023-06-09 07:07:51 +00:00
22b13f4fa5
[mod] tools.Config.get(): add missing type annotations
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-05 14:07:19 +02:00
f3763d73ad
[mod] limiter: blocklist and passlist (ip_lists)
...
A blocklist and a passlist can be configured in /etc/searxng/limiter.toml::
[botdetection.ip_lists]
pass_ip = [
'51.15.252.168', # IPv4 of check.searx.space
]
block_ip = [
'93.184.216.34', # IPv4 of example.org
]
Closes: https://github.com/searxng/searxng/issues/2127
Closes: https://github.com/searxng/searxng/pull/2129
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-05 14:07:19 +02:00
f77807257b
[fix] engines: don't spam marginalia.nu with default settings
...
The engine configuration of marginalia [2][3][4][5] spams marginalia.nu with
requests from SearXNG instances [1]. It is not in the interest of SearXNG to
disturb other FOSS projects, so the engine will be removed::
- name: marginalia
engine: json_engine
shortcut: mar
categories: general
paging: false
# Key and license: https://www.marginalia.nu/marginalia-search/api/
# index: 0 popular, 1 blogs, 2 big_sites, 3 default, 4 experimental
search_url: https://api.marginalia.nu/ <insert your key here>/search/{query}?index=4&count=20
results_query: results
url_query: url
title_query: title
content_query: description
timeout: 1.5
disabled: true
about:
website: https://www.marginalia.nu/
official_api_documentation: https://api.marginalia.nu/
use_official_api: true
require_api_key: true
results: JSON
[1] https://github.com/searxng/searxng/issues/1673
[2] https://github.com/searxng/searxng/pull/1627
[3] https://github.com/searxng/searxng/issues/1620
[4] https://news.ycombinator.com/item?id=35874640
[5] d82a858491/code/services-satellite/api-service/src/main/java/nu/marginalia/api/svc/ResponseCache.java (L12-L20)
2023-06-05 08:23:17 +02:00
80aaef6c95
Merge pull request #2357 / limiter -> botdetection
...
The monolithic implementation of the limiter was divided into methods and
implemented in the Python package searx.botdetection. Detailed documentation on
the methods has been added.
The methods are divided into two groups:
1. Probe HTTP headers
- Method http_accept
- Method http_accept_encoding
- Method http_accept_language
- Method http_connection
- Method http_user_agent
2. Rate limit:
- Method ip_limit
- Method link_token (new)
The (reduced) implementation of the limiter is now in the module
searx.botdetection.limiter. The first group was transferred unchanged to this
module. The ip_limit contains the sliding windows implemented by the limiter so
far.
This merge also fixes some long outstandig issue:
- limiter does not evaluate the Accept-Language correct [1]
- limiter needs a IPv6 prefix to block networks instead of IPs [2]
Without additional configuration the limiter works as before (apart from the
bugfixes). For the commissioning of additional methods (link_toke), a
configuration must be made in an additional configuration file. Without this
configuration, the limiter runs as before (zero configuration).
The ip_limit Method implements the sliding windows of the vanilla limiter,
additionally the link_token method can be used in this method. The link_token
method can be used to investigate whether a request is suspicious. To activate
the link_token method in the ip_limit method add the following to your
/etc/searxng/limiter.toml::
[botdetection.ip_limit]
link_token = true
[1] https://github.com/searxng/searxng/issues/2455
[2] https://github.com/searxng/searxng/issues/2477
2023-06-03 06:00:15 +02:00
1a1ab34d9d
[fix] URL percent-encoding in translations fail in babel
...
Closes: https://github.com/searxng/searxng/issues/2482
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-02 20:30:41 +02:00
b867c39ce0
[build] /static
2023-06-02 19:05:43 +02:00
2149e88bdd
[mod] template preferences: split into elements (no functional change)
...
HINT: this patch has no functional change / it is the preparation for following
changes and bugfixes
Over the years, the preferences template became an unmanageable beast. To make
the source code more readable the monolith is splitted into elements. The
splitting into elements also has the advantage that a new template can make use
of them.
The reversed checkbox is a quirk that is only used in the prefereces and must be
eliminated in the long term. For this the macro 'checkbox_onoff_reversed' was
added to the preferences.html template. The 'checkbox' macro is also a quirk of
the preferences.html we don't want to use in other templates (it is an
input-checkbox in a HTML form that was misused for status display).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-02 19:05:43 +02:00
789b43ab60
[translations] update from Weblate
...
5344314fee8fd9551ce31caffe75c53dca60af52f34b88f322d76a2643d8c98243a92e852bfc12dde2b5fb5f9f088420bdf81b4cf6a24c5d01bcea568c0209f8c629c610a4e4945d96bad166b0032d90366adaef2e4271bfc5856fd6790b5a6f6c9f92a9f5a6a35d4c8eeb327b8c06184e851dd40fa6006e877f4396c3bb1da7e66e6fae1cac4771949e994f8b18158265f8fb93e5088e1cf151100c51d169fae68ac961c336c5a188bda0d06a57c29a0c585b4de8ca9891817b2da46b2508aa3a5b1842fd826ab8a3938c4330cad6b2e997055fde6bd3d8ba5e0129e48fd248b0e7d3f1
2023-06-02 09:34:36 +02:00
80af38d37b
[mod] increase SUSPICIOUS_IP_WINDOW from one day to 30 days
...
In my tests I see bots rotating IPs (with endless IP lists). If such a bot has
100 IPs and has three attempts (SUSPICIOUS_IP_MAX = 3) then it can successfully
send up to 300 requests in one day while rotating the IP. To block the bots for
a longer period of time the SUSPICIOUS_IP_WINDOW, as the time period in which an
IP is observed, must be increased.
For normal WEB-browsers this is no problem, because the SUSPICIOUS_IP_WINDOW is
deleted as soon as the CSS with the token is loaded.
SUSPICIOUS_IP_WINDOW = 3600 * 24 * 30
Time (sec) before sliding window for one suspicious IP expires.
SUSPICIOUS_IP_MAX = 3
Maximum requests from one suspicious IP in the :py:obj:`SUSPICIOUS_IP_WINDOW`."""
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-01 16:00:49 +02:00
281e36f4b7
[fix] limiter: replace real_ip by IPv4/v6 network
...
Closes: https://github.com/searxng/searxng/issues/2477
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-01 15:51:14 +02:00
Markus Heiser
Bnyro
searxng-bot
jazzzooo
ormai
Émilien (perso)
Jinyuan Huang
Alexandre Flament
Hackurei
bonswouar
Jafar Farganlooj
dalf
Paul Braeuning
PlayDay
ToxyFlog1627
GeckoLinux
allendema_searxng_pi
zutto
Kiru
Paolo Basso