searxng/searx
Markus Heiser e0a3dee3bf [POC] limiter: change PING of link_token mehtod from CSS to <img>
while PR #2357 [1] was being implemented the question came up:

    would be better to change the PING resource from CSS to an image so that
    some terminal based browser may still able to pass the test [1]

This patch implements a POC in where a <img src=token> tag is loaded instaed a
CSS.

To test this patch activate limiter and link_token method [3] and start a
developer instance::

    make run

In your terminal browser open http://127.0.0.1:8888/search?q=foo

If the browser is suitable for the link_token method, it loads the image and the
following messages appear::

    DEBUG   searx.botdetection.limiter    : OK 127.0.0.1/32: /clientft61aak7fzyu6o6v.svg ...
    DEBUG   searx.botdetection.link_token : token is valid --> True
    DEBUG   searx.botdetection.link_token : store ping_key for (client) network 127.0.0.1/32 (IP 127.0.0.1) -> SearXNG_limiter.ping[...]

Browsers that do not load images will be blocked: If you try by example::

    lynx http://127.0.0.1:8888/search?q=foo

you will see a WARNING message like::

    WARNING searx.botdetection.link_token : missing ping (IP: 127.0.0.1/32) / request: SearXNG_limiter.ping[...]

----

[1] 80aaef6c95
[2] https://github.com/searxng/searxng/pull/2357#issuecomment-1574898834
[3] activate limiter and link_token method

```diff
diff --git a/searx/botdetection/limiter.toml b/searx/botdetection/limiter.toml
index 71a231e8f..7e1dba755 100644
--- a/searx/botdetection/limiter.toml
+++ b/searx/botdetection/limiter.toml
@@ -17,6 +17,6 @@ ipv6_prefix = 48
 filter_link_local = false

 # acrivate link_token method in the ip_limit method
-link_token = false
+link_token = true

diff --git a/searx/settings.yml b/searx/settings.yml
index a82a3432d..e7b983afc 100644
--- a/searx/settings.yml
+++ b/searx/settings.yml
@@ -73,7 +73,7 @@ server:
   # public URL of the instance, to ensure correct inbound links. Is overwritten
   # by ${SEARXNG_URL}.
   base_url: false  # "http://example.com/location"
-  limiter: false  # rate limit the number of request on the instance, block some bots
+  limiter: true  # rate limit the number of request on the instance, block some bots

   # If your instance owns a /etc/searxng/settings.yml file, then set the following
   # values there.
```

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-03 18:49:21 +02:00
..
answerers [fix] issue reported by pylint 22.1.0 2022-02-04 09:45:35 +01:00
botdetection [mod] increase SUSPICIOUS_IP_WINDOW from one day to 30 days 2023-06-01 16:00:49 +02:00
data Update searx.data - update_engine_descriptions.py 2023-05-29 07:28:50 +02:00
enginelib [fix] searxng_extra/update/update_engine_descriptions.py (part 1) 2023-04-15 16:03:59 +02:00
engines use logger.warning 2023-05-19 19:35:29 +05:30
infopage [fix] minor typo in de/search-syntax page 2023-04-08 10:25:52 +02:00
metrics [fix] prepare for pylint 2.14.0 2022-06-03 15:41:52 +02:00
network [mod] Shuffle httpx's default ciphers of a SSL context randomly. 2023-03-19 13:40:31 +01:00
plugins [fix] limiter: replace real_ip by IPv4/v6 network 2023-06-01 15:51:14 +02:00
search use logger.warning 2023-05-19 19:35:29 +05:30
static [build] /static 2023-06-02 19:05:43 +02:00
templates/simple [POC] limiter: change PING of link_token mehtod from CSS to <img> 2023-06-03 18:49:21 +02:00
tools [mod] limiter: add config file /etc/searxng/limiter.toml 2023-06-01 14:38:53 +02:00
translations [translations] update from Weblate 2023-06-02 09:34:36 +02:00
__init__.py [format.python] initial formatting of the python code 2021-12-27 09:26:22 +01:00
autocomplete.py [mod] remove obsolete EngineTraits.supported_languages 2023-03-24 10:37:42 +01:00
babel_extract.py [mod] move category and names of constants to searx/searxng.msg 2022-03-16 09:55:53 +01:00
compat.py [fix] pyright repported errors 2022-07-30 18:04:44 +02:00
exceptions.py [doc] Add doc-strings to searx.exceptions 2023-01-29 19:06:19 +01:00
external_bang.py [mod] external bang: go to main instead of search page when query is empty 2023-04-25 15:02:34 +02:00
external_urls.py [fix] typos / reported by @kianmeng in searx PR-3366 2022-09-27 18:32:14 +02:00
flaskfix.py [format.python] initial formatting of the python code 2021-12-27 09:26:22 +01:00
locales.py use logger.warning 2023-05-19 19:35:29 +05:30
preferences.py [fix] ClientPref - don't raise exception if Accept-Language is invalid 2023-05-22 12:38:59 +02:00
query.py [mod] replace searx.languages by searx.sxng_locales 2023-03-24 10:37:42 +01:00
redisdb.py move searx.shared.redisdb to searx.redisdb 2022-12-10 09:26:38 +01:00
redislib.py [mod] redislib - optimize LUA script registration. 2022-05-30 11:21:09 +02:00
results.py [fix] typos / reported by @kianmeng in searx PR-3366 2022-09-27 18:32:14 +02:00
searxng.msg [mod] clarify the difference of the default category and subgrouping 2023-04-07 11:03:25 +02:00
settings.yml [mod] donation_url: disable by default 2023-05-15 09:19:17 +02:00
settings_defaults.py [mod] replace searx.languages by searx.sxng_locales 2023-03-24 10:37:42 +01:00
settings_loader.py [fix] categories can't be removed from UI (categories_as_tabs) 2023-04-03 19:08:27 +02:00
sxng_locales.py Update searx.data - update_engine_traits.py 2023-04-16 08:40:44 +02:00
unixthreadname.py [format.python] initial formatting of the python code 2021-12-27 09:26:22 +01:00
utils.py [mod] replace utils.match_language by locales.match_locale 2023-03-24 10:37:42 +01:00
version.py [fix] version format string generated by 'git show' 2023-05-10 18:33:45 +02:00
webadapter.py [mod] move language recognition to get_search_query_from_webapp 2023-04-15 22:23:33 +02:00
webapp.py [POC] limiter: change PING of link_token mehtod from CSS to <img> 2023-06-03 18:49:21 +02:00
webutils.py [mod] in the preference page, show !bang of subgrouping categories 2023-04-08 11:10:14 +02:00