mirror of
https://github.com/searxng/searxng.git
synced 2025-12-22 19:50:00 +00:00
[mod] migrate from Redis to Valkey (#4795)
This patch migrates from `redis==5.2.1` [1] to `valkey==6.1.0` [2].
The migration to valkey is necessary because the company behind Redis has decided
to abandon the open source license. After experiencing a drop in user numbers,
they now want to run it under a dual license again. But this move demonstrates
once again how unreliable the company is and how it treats open source
developers.
To review first, read the docs::
$ make docs.live
Follow the instructions to remove redis:
- http://0.0.0.0:8000/admin/settings/settings_redis.html
Config and install a local valkey DB:
- http://0.0.0.0:8000/admin/settings/settings_valkey.html
[1] https://pypi.org/project/redis/
[2] https://pypi.org/project/valkey/
Co-authored-by: HLFH <gaspard@dhautefeuille.eu>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
committed by
GitHub
parent
bd593d0bad
commit
f798ddd492
@@ -1,11 +1,11 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# pylint: disable=missing-module-docstring
|
||||
"""Lame scheduler which use Redis as a source of truth:
|
||||
* the Redis key SearXNG_checker_next_call_ts contains the next time the embedded checker should run.
|
||||
* to avoid lock, a unique Redis script reads and updates the Redis key SearXNG_checker_next_call_ts.
|
||||
* this Redis script returns a list of two elements:
|
||||
"""Lame scheduler which use Valkey as a source of truth:
|
||||
* the Valkey key SearXNG_checker_next_call_ts contains the next time the embedded checker should run.
|
||||
* to avoid lock, a unique Valkey script reads and updates the Valkey key SearXNG_checker_next_call_ts.
|
||||
* this Valkey script returns a list of two elements:
|
||||
* the first one is a boolean. If True, the embedded checker must run now in this worker.
|
||||
* the second element is the delay in second to wait before the next call to the Redis script.
|
||||
* the second element is the delay in second to wait before the next call to the Valkey script.
|
||||
|
||||
This scheduler is not generic on purpose: if more feature are required, a dedicate scheduler must be used
|
||||
(= a better scheduler should not use the web workers)
|
||||
@@ -16,8 +16,8 @@ import time
|
||||
from pathlib import Path
|
||||
from typing import Callable
|
||||
|
||||
from searx.redisdb import client as get_redis_client
|
||||
from searx.redislib import lua_script_storage
|
||||
from searx.valkeydb import client as get_valkey_client
|
||||
from searx.valkeylib import lua_script_storage
|
||||
|
||||
|
||||
logger = logging.getLogger('searx.search.checker')
|
||||
@@ -29,7 +29,7 @@ def scheduler_function(start_after_from: int, start_after_to: int, every_from: i
|
||||
"""Run the checker periodically. The function never returns.
|
||||
|
||||
Parameters:
|
||||
* start_after_from and start_after_to: when to call "callback" for the first on the Redis instance
|
||||
* start_after_from and start_after_to: when to call "callback" for the first on the Valkey instance
|
||||
* every_from and every_to: after the first call, how often to call "callback"
|
||||
|
||||
There is no issue:
|
||||
@@ -38,11 +38,11 @@ def scheduler_function(start_after_from: int, start_after_to: int, every_from: i
|
||||
"""
|
||||
scheduler_now_script = SCHEDULER_LUA.open().read()
|
||||
while True:
|
||||
# ask the Redis script what to do
|
||||
# ask the Valkey script what to do
|
||||
# the script says
|
||||
# * if the checker must run now.
|
||||
# * how to long to way before calling the script again (it can be call earlier, but not later).
|
||||
script = lua_script_storage(get_redis_client(), scheduler_now_script)
|
||||
script = lua_script_storage(get_valkey_client(), scheduler_now_script)
|
||||
call_now, wait_time = script(args=[start_after_from, start_after_to, every_from, every_to])
|
||||
|
||||
# does the worker run the checker now?
|
||||
|
||||
Reference in New Issue
Block a user