mirror of
https://github.com/searxng/searxng.git
synced 2025-12-22 19:50:00 +00:00
[mod] limiter -> botdetection: modularization and documentation
In order to be able to meet the outstanding requirements, the implementation is modularized and supplemented with documentation. This patch does not contain functional change, except it fixes issue #2455 ---- Aktivate limiter in the settings.yml and simulate a bot request by:: curl -H 'Accept-Language: de-DE,en-US;q=0.7,en;q=0.3' \ -H 'Accept: text/html' -H 'User-Agent: xyz' \ -H 'Accept-Encoding: gzip' \ 'http://127.0.0.1:8888/search?q=foo' In the LOG: DEBUG searx.botdetection.link_token : missing ping for this request: ..... Since ``BURST_MAX_SUSPICIOUS = 2`` you can repeat the query above two time before you get a "Too Many Requests" response. Closes: https://github.com/searxng/searxng/issues/2455 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
@@ -235,7 +235,7 @@ Global Settings
|
||||
|
||||
``limiter`` :
|
||||
Rate limit the number of request on the instance, block some bots. The
|
||||
:ref:`limiter plugin` requires a :ref:`settings redis` database.
|
||||
:ref:`limiter src` requires a :ref:`settings redis` database.
|
||||
|
||||
.. _image_proxy:
|
||||
|
||||
|
||||
45
docs/src/searx.botdetection.rst
Normal file
45
docs/src/searx.botdetection.rst
Normal file
@@ -0,0 +1,45 @@
|
||||
.. _botdetection:
|
||||
|
||||
=============
|
||||
Bot Detection
|
||||
=============
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.botdetection
|
||||
:members:
|
||||
|
||||
.. automodule:: searx.botdetection.limiter
|
||||
:members:
|
||||
|
||||
|
||||
Rate limit
|
||||
==========
|
||||
|
||||
.. automodule:: searx.botdetection.ip_limit
|
||||
:members:
|
||||
|
||||
.. automodule:: searx.botdetection.link_token
|
||||
:members:
|
||||
|
||||
|
||||
Probe HTTP headers
|
||||
==================
|
||||
|
||||
.. automodule:: searx.botdetection.http_accept
|
||||
:members:
|
||||
|
||||
.. automodule:: searx.botdetection.http_accept_encoding
|
||||
:members:
|
||||
|
||||
.. automodule:: searx.botdetection.http_accept_language
|
||||
:members:
|
||||
|
||||
.. automodule:: searx.botdetection.http_connection
|
||||
:members:
|
||||
|
||||
.. automodule:: searx.botdetection.http_user_agent
|
||||
:members:
|
||||
@@ -1,13 +0,0 @@
|
||||
.. _limiter plugin:
|
||||
|
||||
==============
|
||||
Limiter Plugin
|
||||
==============
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
The :ref:`limiter plugin` requires a :ref:`Redis <settings redis>` database.
|
||||
|
||||
.. automodule:: searx.plugins.limiter
|
||||
:members:
|
||||
|
||||
Reference in New Issue
Block a user