jandieman/searxng
2
0
Fork 0
mirror of https://github.com/searxng/searxng.git synced 2025-12-23 20:20:05 +00:00
Code Issues Projects Releases Wiki Activity
7,404 Commits 6 Branches 0 Tags
c348427f07b51b0bd9d88d25bffd1737228919e5
Commit Graph

7404 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Markus Heiser
1a1ab34d9d [fix] URL percent-encoding in translations fail in babel 
Closes: https://github.com/searxng/searxng/issues/2482
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-02 20:30:41 +02:00
Markus Heiser
1541f8660e Merge pull request #2481 / [mod] template preferences: split into elements 
HINT: this patch has no functional change / it is the preparation for following changes and bugfixes

Over the years, the preferences template became an unmanageable beast. To make the source code more readable the monolith is splitted into elements. The splitting into elements also has the advantage that a new template can make use of them.

The reversed checkbox is a quirk that is only used in the prefereces and must be eliminated in the long term. For this the macro 'checkbox_onoff_reversed' was added to the preferences.html template. The 'checkbox' macro is also a quirk of the preferences.html we don't want to use in other templates (it is an input-checkbox in a HTML form that was misused for status display).
 2023-06-02 19:55:43 +02:00
Markus Heiser
b867c39ce0 [build] /static 2023-06-02 19:05:43 +02:00
Markus Heiser
2149e88bdd [mod] template preferences: split into elements (no functional change) 
HINT: this patch has no functional change / it is the preparation for following
      changes and bugfixes

Over the years, the preferences template became an unmanageable beast.  To make
the source code more readable the monolith is splitted into elements.  The
splitting into elements also has the advantage that a new template can make use
of them.

The reversed checkbox is a quirk that is only used in the prefereces and must be
eliminated in the long term.  For this the macro 'checkbox_onoff_reversed' was
added to the preferences.html template.  The 'checkbox' macro is also a quirk of
the preferences.html we don't want to use in other templates (it is an
input-checkbox in a HTML form that was misused for status display).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-02 19:05:43 +02:00
dependabot[bot]
d289a8b225 Bump typing-extensions from 4.6.2 to 4.6.3 
Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.6.2 to 4.6.3.
- [Changelog](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md)
- [Commits](https://github.com/python/typing_extensions/compare/4.6.2...4.6.3)

---
updated-dependencies:
- dependency-name: typing-extensions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-06-02 10:04:46 +02:00
searxng-bot
789b43ab60 [translations] update from Weblate 
5344314f - 2023-05-30 - return42 <markus.heiser@darmarit.de>
ee8fd955 - 2023-06-01 - BBTranslate <357835338@qq.com>
1ce31caf - 2023-05-29 - return42 <markus.heiser@darmarit.de>
fe75c53d - 2023-05-29 - return42 <markus.heiser@darmarit.de>
ca60af52 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
f34b88f3 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
22d76a26 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
43d8c982 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
43a92e85 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
2bfc12dd - 2023-05-29 - return42 <markus.heiser@darmarit.de>
e2b5fb5f - 2023-05-29 - return42 <markus.heiser@darmarit.de>
9f088420 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
bdf81b4c - 2023-05-29 - return42 <markus.heiser@darmarit.de>
f6a24c5d - 2023-05-30 - return42 <markus.heiser@darmarit.de>
01bcea56 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
8c0209f8 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
c629c610 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
a4e4945d - 2023-05-29 - return42 <markus.heiser@darmarit.de>
96bad166 - 2023-06-01 - mradalbert <mister.adalbert@gmail.com>
b0032d90 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
366adaef - 2023-05-29 - return42 <markus.heiser@darmarit.de>
2e4271bf - 2023-05-29 - return42 <markus.heiser@darmarit.de>
c5856fd6 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
790b5a6f - 2023-05-29 - return42 <markus.heiser@darmarit.de>
6c9f92a9 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
f5a6a35d - 2023-05-29 - return42 <markus.heiser@darmarit.de>
4c8eeb32 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
7b8c0618 - 2023-05-30 - nicfab <nicfab@icloud.com>
4e851dd4 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
0fa6006e - 2023-05-29 - return42 <markus.heiser@darmarit.de>
877f4396 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
c3bb1da7 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
e66e6fae - 2023-05-30 - return42 <markus.heiser@darmarit.de>
1cac4771 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
949e994f - 2023-05-28 - ghose <correo@xmgz.eu>
8b181582 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
65f8fb93 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
e5088e1c - 2023-05-29 - return42 <markus.heiser@darmarit.de>
f151100c - 2023-05-29 - return42 <markus.heiser@darmarit.de>
51d169fa - 2023-05-29 - return42 <markus.heiser@darmarit.de>
e68ac961 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
c336c5a1 - 2023-05-31 - dom1torii <djmdmitri.a@gmail.com>
88bda0d0 - 2023-05-30 - Fijxu <fijxu@zzls.xyz>
6a57c29a - 2023-05-29 - return42 <markus.heiser@darmarit.de>
0c585b4d - 2023-05-30 - return42 <markus.heiser@darmarit.de>
e8ca9891 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
817b2da4 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
6b2508aa - 2023-05-29 - return42 <markus.heiser@darmarit.de>
3a5b1842 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
fd826ab8 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
a3938c43 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
30cad6b2 - 2023-05-30 - Ivan Gabaldon <admin@inetol.net>
e997055f - 2023-05-30 - return42 <markus.heiser@darmarit.de>
de6bd3d8 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
ba5e0129 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
e48fd248 - 2023-05-29 - return42 <markus.heiser@darmarit.de>
b0e7d3f1 - 2023-05-30 - return42 <markus.heiser@darmarit.de>
 2023-06-02 09:34:36 +02:00
Markus Heiser
80af38d37b [mod] increase SUSPICIOUS_IP_WINDOW from one day to 30 days 
In my tests I see bots rotating IPs (with endless IP lists).  If such a bot has
100 IPs and has three attempts (SUSPICIOUS_IP_MAX = 3) then it can successfully
send up to 300 requests in one day while rotating the IP.  To block the bots for
a longer period of time the SUSPICIOUS_IP_WINDOW, as the time period in which an
IP is observed, must be increased.

For normal WEB-browsers this is no problem, because the SUSPICIOUS_IP_WINDOW is
deleted as soon as the CSS with the token is loaded.

SUSPICIOUS_IP_WINDOW = 3600 * 24 * 30
  Time (sec) before sliding window for one suspicious IP expires.

SUSPICIOUS_IP_MAX = 3
  Maximum requests from one suspicious IP in the :py:obj:`SUSPICIOUS_IP_WINDOW`."""

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-01 16:00:49 +02:00
Markus Heiser
281e36f4b7 [fix] limiter: replace real_ip by IPv4/v6 network 
Closes: https://github.com/searxng/searxng/issues/2477
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-01 15:51:14 +02:00
Markus Heiser
38431d2e14 [fix] correct determination of the IP for the request 
For correct determination of the IP to the request the function
botdetection.get_real_ip() is implemented.  This fonction is used in the
ip_limit and link_token method of the botdetection and it is used in the
self_info plugin.

A documentation about the X-Forwarded-For header has been added.

[1] https://github.com/searxng/searxng/pull/2357#issuecomment-1566211059

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-01 14:38:53 +02:00
Markus Heiser
b8c7c2c9aa [mod] botdetection - improve ip_limit and link_token methods 
- counting requests in LONG_WINDOW and BURST_WINDOW is not needed when the
  request is validated by the link_token method [1]

- renew a ping-key on validation [2], this is needed for infinite scrolling,
  where no new token (CSS) is loaded. / this does not fix the BURST_MAX issue in
  the vanilla limiter

- normalize the counter names of the ip_limit method to 'ip_limit.*'

- just integrate the ip_limit method straight forward in the limiter plugin /
  non intermediate code --> ip_limit now returns None or a werkzeug.Response
  object that can be passed by the plugin to the flask application / non
  intermediate code that returns a tuple

[1] https://github.com/searxng/searxng/pull/2357#issuecomment-1566113277
[2] https://github.com/searxng/searxng/pull/2357#discussion_r1208542206
[3] https://github.com/searxng/searxng/pull/2357#issuecomment-1566125979

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-01 14:38:53 +02:00
Markus Heiser
52f1452c09 [mod] limiter: ip_limt - monitore suspicious IPs 
To intercept bots that get their IPs from a range of IPs, there is a
``SUSPICIOUS_IP_WINDOW``.  In this window the suspicious IPs are stored for a
longer time.  IPs stored in this sliding window have a maximum of
``SUSPICIOUS_IP_MAX`` accesses before they are blocked.  As soon as the IP makes
a request that is not suspicious, the sliding window for this IP is droped.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-01 14:38:53 +02:00
Markus Heiser
9d7456fd6c [fix] limiter.toml: botdetection.ip_limit turn off link_token by default 
To activate the ``link_token`` method in the ``ip_limit`` method add the
following to your ``/etc/searxng/limiter.toml``::

   [botdetection.ip_limit]
   link_token = true

Related: https://github.com/searxng/searxng/pull/2357#issuecomment-1554116941
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-01 14:38:53 +02:00
Markus Heiser
66fdec0eb9 [mod] limiter: add config file /etc/searxng/limiter.toml 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-01 14:38:53 +02:00
Markus Heiser
1ec325adcc [mod] limiter -> botdetection: modularization and documentation 
In order to be able to meet the outstanding requirements, the implementation is
modularized and supplemented with documentation.

This patch does not contain functional change, except it fixes issue #2455

----

Aktivate limiter in the settings.yml and simulate a bot request by::

    curl -H 'Accept-Language: de-DE,en-US;q=0.7,en;q=0.3' \
         -H 'Accept: text/html'
         -H 'User-Agent: xyz' \
         -H 'Accept-Encoding: gzip' \
         'http://127.0.0.1:8888/search?q=foo'

In the LOG:

    DEBUG   searx.botdetection.link_token : missing ping for this request: .....

Since ``BURST_MAX_SUSPICIOUS = 2`` you can repeat the query above two time
before you get a "Too Many Requests" response.

Closes: https://github.com/searxng/searxng/issues/2455
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-29 14:54:56 +02:00
Markus Heiser
5226044c13 [mod] limiter: add random token to the limiter URL 
By adding a random component in the limiter URL a bot can no longer send a ping
by request a static URL.

Related: https://github.com/searxng/searxng/pull/2357#issuecomment-1518525094
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-29 14:54:56 +02:00
Markus Heiser
dba569462d [mod] limiter: reduce request rates for requests without a ping 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-29 14:54:56 +02:00
dalf
c1b5ff7e1c Update searx.data - update_engine_descriptions.py 2023-05-29 07:28:50 +02:00
dalf
2ba50d392e Update searx.data - update_currencies.py 2023-05-29 07:28:18 +02:00
dalf
cb843ef13c Update searx.data - update_engine_traits.py 2023-05-29 07:27:40 +02:00
dalf
512e001277 Update searx.data - update_firefox_version.py 2023-05-29 07:27:07 +02:00
dalf
f03ac9b152 Update searx.data - update_wikidata_units.py 2023-05-29 07:26:47 +02:00
dalf
e12e350f7f Update searx.data - update_ahmia_blacklist.py 2023-05-29 07:26:20 +02:00
Markus Heiser
3ca97cf5e3 [fix] simple theme: move engine alerts in case of no results into sidebar 
If there were no results but errors in the engines then the error dialogs of the
engines was displayed in the result list.

With the new design errors of the engines should only be displayed in the
sidebar and at the same time duplications of the (template) code will be
avoided.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-28 12:19:32 +02:00
mrpaulblack
60b94dfdca [build] /static 2023-05-28 12:19:32 +02:00
mrpaulblack
f087959b02 [mod] simple theme: build design for details (collapsables) 
* set border top and bottom on sidebar collasables
* inrease peading on summary so its easier to click on mobile
* remove margins and add flex wrapper to normalize elements in sidebar
 2023-05-28 12:19:32 +02:00
Markus Heiser
b7e315563d [mod] simple theme: collaps/expand elements in the sidebar 
Make elements in the sidebar collapse able.  Except infoboxes all elements in
the sidebar are collapsed by default.

By folding out the sidebar elements, the UI looks less cluttered.  Especially on
small devices like smartphones, where the sidebar is above the results list, the
UX should be improved [1].

[1] https://github.com/searxng/searxng/issues/2140

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-28 12:19:32 +02:00
Markus Heiser
79c02456c1 Merge pull request #2462 from return42/fix-typos 
[doc] fix tyops in docs/dev/reST.rst
 2023-05-26 12:06:19 +02:00
Markus Heiser
64cbe80d05 Merge pull request #2459 from searxng/translations_update 
Update translations
 2023-05-26 12:03:58 +02:00
Markus Heiser
ce3230f365 Merge pull request #2460 from searxng/dependabot/pip/master/typing-extensions-4.6.2 
Bump typing-extensions from 4.5.0 to 4.6.2
 2023-05-26 12:03:05 +02:00
Markus Heiser
620415f4cf Merge pull request #2461 from searxng/dependabot/pip/master/yamllint-1.32.0 
Bump yamllint from 1.31.0 to 1.32.0
 2023-05-26 12:02:14 +02:00
Markus Heiser
68a0e1bfc3 [doc] fix tyops in docs/dev/reST.rst 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-26 11:51:35 +02:00
dependabot[bot]
5137452677 Bump yamllint from 1.31.0 to 1.32.0 
Bumps [yamllint](https://github.com/adrienverge/yamllint) from 1.31.0 to 1.32.0.
- [Changelog](https://github.com/adrienverge/yamllint/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/adrienverge/yamllint/compare/v1.31.0...v1.32.0)

---
updated-dependencies:
- dependency-name: yamllint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-05-26 07:56:43 +00:00
dependabot[bot]
8ce08f201b Bump typing-extensions from 4.5.0 to 4.6.2 
Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.5.0 to 4.6.2.
- [Changelog](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md)
- [Commits](https://github.com/python/typing_extensions/compare/4.5.0...4.6.2)

---
updated-dependencies:
- dependency-name: typing-extensions
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-05-26 07:56:37 +00:00
searxng-bot
e8054026fb [translations] update from Weblate 
69171f12 - 2023-05-25 - fabiosantoscode <fabiosantosart@gmail.com>
2caaed0a - 2023-05-23 - trmx <borcan.cristian1@gmail.com>
84d1702b - 2023-05-21 - return42 <markus.heiser@darmarit.de>
65cc6eb8 - 2023-05-21 - return42 <markus.heiser@darmarit.de>
e0ab3383 - 2023-05-22 - return42 <markus.heiser@darmarit.de>
23e87f15 - 2023-05-21 - return42 <markus.heiser@darmarit.de>
14f0fc6b - 2023-05-21 - return42 <markus.heiser@darmarit.de>
5b7c7b7d - 2023-05-21 - return42 <markus.heiser@darmarit.de>
c725b38d - 2023-05-21 - return42 <markus.heiser@darmarit.de>
 2023-05-26 07:08:21 +00:00
Markus Heiser
b2d0b74357 Merge pull request #2453 from return42/fix-typo 
[fix] typo: dues --> does
 2023-05-22 13:23:48 +02:00
Markus Heiser
ebe22a4319 [fix] typo: dues --> does 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-22 13:18:22 +02:00
Markus Heiser
cea277ce52 Merge pull request #2452 from return42/fix-2434 
[fix] ClientPref - don't raise exception if Accept-Language is invalid
 2023-05-22 12:46:26 +02:00
Markus Heiser
bc647fabaf [fix] ClientPref - don't raise exception if Accept-Language is invalid 
If the Accept-Language header [1] is set but empty or holds a value that is
unknown to babel, an excpetion is raised::

    $ curl --header 'Accept-Language: xyz' 'http://127.0.0.1:8888/search?q=foo'
    ...
    Traceback (most recent call last):
      File "searx/preferences.py", line 335, in from_http_request
        return cls(locale=pairs[0][0])
    IndexError: list index out of range

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language

Reported by: @Eolien55 in https://github.com/searxng/searxng/issues/2434#issuecomment-1556199789
Closes: https://github.com/searxng/searxng/issues/2434
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-22 12:38:59 +02:00
Markus Heiser
73c023bbea Merge pull request #2451 from return42/doc-captcha-from-server-ip 
[doc] answer CAPTCHA from server's IP
 2023-05-22 10:59:27 +02:00
Markus Heiser
0ee27ba3aa [doc] answer CAPTCHA from server's IP 
Related: https://github.com/searxng/searxng/issues/2011#issuecomment-1553317619
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-22 10:46:10 +02:00
Markus Heiser
ddbab4b3c9 Merge pull request #2450 from return42/lxc-doc 
[doc] update & fix documentation of the "SearXNG LXC suite"
 2023-05-21 18:26:14 +02:00
Markus Heiser
c9833ded9f [doc] update & fix documentation of the "SearXNG LXC suite" 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-21 18:12:39 +02:00
Markus Heiser
d867bf17e6 Merge pull request #2447 from searxng/dependabot/pip/master/linuxdoc-20230506 
Bump linuxdoc from 20230321 to 20230506
 2023-05-19 20:08:11 +02:00
Markus Heiser
bab730c8a8 Merge pull request #2446 from searxng/translations_update 
Update translations
 2023-05-19 19:59:58 +02:00
Markus Heiser
7b2f1079f6 Merge pull request #2448 from Pankwings/logger_warning 
use logger.warning
 2023-05-19 19:58:47 +02:00
pankaj
4900c091a6 use logger.warning 
logger.warn() is depricated.
logger.warning is already being used in some files.
 2023-05-19 19:35:29 +05:30
dependabot[bot]
ef31fa8097 Bump linuxdoc from 20230321 to 20230506 
Bumps [linuxdoc](https://github.com/return42/linuxdoc) from 20230321 to 20230506.
- [Changelog](https://github.com/return42/linuxdoc/blob/master/CHANGELOG)
- [Commits](https://github.com/return42/linuxdoc/commits)

---
updated-dependencies:
- dependency-name: linuxdoc
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-05-19 07:56:49 +00:00
searxng-bot
23b53a03c2 [translations] update from Weblate 
2eeec66c - 2023-05-13 - return42 <markus.heiser@darmarit.de>
87058e51 - 2023-05-13 - return42 <markus.heiser@darmarit.de>
 2023-05-19 07:07:56 +00:00
Markus Heiser
50cd7e2b03 Merge pull request #2435 from return42/disable-donation 
[mod] donation_url: disable by default
 2023-05-15 09:41:15 +02:00
Markus Heiser
007a615ffa [mod] donation_url: disable by default 
SearXNG's donation campaign has been ended.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-05-15 09:19:17 +02:00