As stated in .. and other posts, the defaults of uWSGI not suitable for a
productive environment. To give just one example, the workers run indefinitely
and the memory leaks aggregate.
- "Configuring uWSGI for Production: The defaults are all wrong" EuroPython 2019 [1]
- "Configuring uWSGI for Production Deployment" [2]
- "When Paul has tested some PR on his instance, we could clearly see a memory
leak over a week: the memory never dropped to the initial value. Same for my
instance using Docker." [3]
[1] https://av.tib.eu/media/44810
[2] https://www.bloomberg.com/company/stories/configuring-uwsgi-production-deployment/
[3] https://github.com/searxng/searxng/pull/3443#issuecomment-2094347004
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* Docker: add UWSGI_WORKERS and UWSGI_THREAD.
UWSGI_WORKERS specifies the number of process.
UWSGI_THREADS specifies the number of threads.
The Docker convention is to specify the whole configuration
through environment variables. While not done in SearXNG, these two
additional variables allows admins to skip uwsgi.ini
In additional, https://github.com/searxng/preview-environments starts Docker
without additional files through searxng-helm-chat.
Each instance consumes 1Go of RAM which is a lot especially when there are a
lot of instances / pull requests.
* [scripts] add environments UWSGI_WORKERS and UWSGI_THREADS
- UWSGI_WORKERS specifies the number of process.
- UWSGI_THREADS specifies the number of threads.
Templates for uwsgi scripts can be tested by::
UWSGI_WORKERS=8 UWSGI_THREADS=9 \
./utils/searxng.sh --cmd\
eval "echo \"$(cat utils/templates/etc/uwsgi/*/searxng.ini*)\""\
| grep "workers\|threads"
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
---------
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Caching files on the client side for more than a day can confuse the end user
when updating static files[1].
Depending on the way of providing a SearXNG instance via HTTP, there are several
ways to optimize the access to the /static files. However, since we don't know
what optimization an admin has provided for his static files, we should have
moderate settings in the defaults that run robustly in a wide variety of
installations.
In this sense, all caches on the client side should be cleared after one day at
the latest. So far the files were cached for one year on client side; as soon
as changes are made to the static files (with the option `static_use_hash:
true`) the old static files are kept for one year on the CLient side / which can
also be evaluated as unnecessary caching.
[1] https://github.com/searxng/searxng/discussions/2821
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
previously the log (only the exceptions) were log
into /var/log/uwsgi/uwsgi.log
this is disturbing for the admins:
* they see an internal error on HTTP port
* no log where they are expected (docker logs)
this commit fixes this issue
When the image is built, the static files are compressed with gzip and brotli.
The expires header is set to one day (same as Flask)
There is not etag header (Flask does add an etag header)
See #1561 , use uwsgi and Alpine Linux
Volume:
/var/log/uwsgi contains error log for 2 days (file uwsgi.log)
/etc/searx contains the settings.yml and uwsgi.ini files.
The docker image creates them if they don't exist.
The two files can be modified after the first run. See below.
Environement variables:
MORTY_URL : external URL of Morty
MORTY_KEY : base64 encoded key
BASE_URL : external URL of Searx
BIND_ADDRESS : internal HTTP port to listen to
Labels : org.label-schema.schema.*
Parameters:
-h : display this help
-d : will update the settings and quit immediately (settings.yml and uwsgi.ini)
-f : always update the settings (previous version saved with suffix .old).
without this parameter, the new settings are copied with suffix .new
When the Docker image contains newer settings:
- without -f parameter: the new versions are copied to /etc/searx/settings.yml.new and /etc/searx/uwsgi.ini.new.
- with -f parameter: the old versions are renamed with .old suffix. The new version replaces /etc/searx/settings.yml and /etc/searx/uwsgi.ini
Build using "./manage.sh docker_build", add "push" as parameter also push the Docker image.
The script requires a git repository to work (it makes sure that the last git tag matches searx/version.py)
"git describe" is used to create a meaningful version.
Example : 0.15.0-90-49c5bcb4-dirty (dirty means that the docker image was made with uncommited changes).
Use "docker inspect -f {{.Config.Labels.version}} searx" to get the version of an existing image.
.dockerignore based on .gitignore
.travis.yml: include docker stage
Markus Heiser
Ivan G
Alexandre Flament