Commit Graph

6 Commits

Author SHA1 Message Date
Markus Heiser 056dd71716
Merge 1184caa0fd into 81aaca8f44 2024-10-10 16:23:53 -04:00
Markus Heiser 542f7d0d7b [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION
In the past, some files were tested with the standard profile, others with a
profile in which most of the messages were switched off ... some files were not
checked at all.

- ``PYLINT_SEARXNG_DISABLE_OPTION`` has been abolished
- the distinction ``# lint: pylint`` is no longer necessary
- the pylint tasks have been reduced from three to two

  1. ./searx/engines -> lint engines with additional builtins
  2. ./searx ./searxng_extra ./tests -> lint all other python files

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-03-11 14:55:38 +01:00
Markus Heiser 1184caa0fd [mod] RangeTable - fast lookup of a integer (IP) in a range table
This patch implements a table of ranges.  A *range* is a series of integers,
e.g. 0-10 with start value 0 and end value 10.

Start and end values are stored in a two-column table, the RangeTable.  This
table is stored in Redis in a sorted list and a ZRANGEBYSCORE query can be used
to determine if a particular integer is contained in one of the ranges in the
table.

The ZRANGEBYSCORE query scales with O(log(N)). To make it more clear: a query in
a table with 1 million ranges takes just twice as long as the query in a table
with 1000 ranges.

A RangeTable is excellent for IP ranges and ultra-fast queries to see if an
IP (which is nothing but an integer) is contained in one of the ranges of the
table.

Typical applications are e.g. large IP block lists or network ranges.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-20 14:55:30 +02:00
jazzzooo 223b3487c3 [fix] spelling 2023-09-18 16:20:27 +02:00
Markus Heiser 1a8ff10e30 [mod] redislib - optimize LUA script registration.
Instead of register the LUA script on each call, the LUA script is registered
once and the `Script` object is cached in global storage.

[1] https://github.com/searxng/searxng/pull/1047#discussion_r884143305

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-05-30 11:21:09 +02:00
Markus Heiser 08876df7c2 [mod] collection of redis functions and lua scripts (initial)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-05-30 11:10:30 +02:00