[POC] limiter: change PING of link_token mehtod from CSS to <img>

while PR #2357 [1] was being implemented the question came up:

    would be better to change the PING resource from CSS to an image so that
    some terminal based browser may still able to pass the test [1]

This patch implements a POC in where a <img src=token> tag is loaded instaed a
CSS.

To test this patch activate limiter and link_token method [3] and start a
developer instance::

    make run

In your terminal browser open http://127.0.0.1:8888/search?q=foo

If the browser is suitable for the link_token method, it loads the image and the
following messages appear::

    DEBUG   searx.botdetection.limiter    : OK 127.0.0.1/32: /clientft61aak7fzyu6o6v.svg ...
    DEBUG   searx.botdetection.link_token : token is valid --> True
    DEBUG   searx.botdetection.link_token : store ping_key for (client) network 127.0.0.1/32 (IP 127.0.0.1) -> SearXNG_limiter.ping[...]

Browsers that do not load images will be blocked: If you try by example::

    lynx http://127.0.0.1:8888/search?q=foo

you will see a WARNING message like::

    WARNING searx.botdetection.link_token : missing ping (IP: 127.0.0.1/32) / request: SearXNG_limiter.ping[...]

Modern terminal WEB browser do support `<img>` tag as well as CSS:

    browsh http://127.0.0.1:8888/search?q=foo

----

[1] 80aaef6c95
[2] https://github.com/searxng/searxng/pull/2357#issuecomment-1574898834
[3] activate limiter and link_token method

```diff
diff --git a/searx/botdetection/limiter.toml b/searx/botdetection/limiter.toml
index 71a231e8f..7e1dba755 100644
--- a/searx/botdetection/limiter.toml
+++ b/searx/botdetection/limiter.toml
@@ -17,6 +17,6 @@ ipv6_prefix = 48
 filter_link_local = false

 # acrivate link_token method in the ip_limit method
-link_token = false
+link_token = true

diff --git a/searx/settings.yml b/searx/settings.yml
index a82a3432d..e7b983afc 100644
--- a/searx/settings.yml
+++ b/searx/settings.yml
@@ -73,7 +73,7 @@ server:
   # public URL of the instance, to ensure correct inbound links. Is overwritten
   # by ${SEARXNG_URL}.
   base_url: false  # "http://example.com/location"
-  limiter: false  # rate limit the number of request on the instance, block some bots
+  limiter: true  # rate limit the number of request on the instance, block some bots

   # If your instance owns a /etc/searxng/settings.yml file, then set the following
   # values there.
```

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

.editorconfig

@@ -16,6 +16,9 @@ max_line_length = 119
 [*.html]
 indent_size = 4
 
+[*.js]
+indent_size = 2
+
 [*.json]
 indent_size = 4
 insert_final_newline = ignore

.github/ISSUE_TEMPLATE/engine-request.md

@@ -20,7 +20,7 @@ assignees: ''
 **How can SearXNG fetch the information from this engine?**
 <!-- List API URL, example code (using the correct markdown) and more
 that could be useful for the developers in order to implement this engine.
-If you don't know what to write, let this part blank.>
+If you don't know what to write, let this part blank. -->
 
 **Applicable category of this engine**
 <!-- Where should this new engine fit in SearXNG? Current categories in SearXNG:

.github/dependabot.yml

@@ -1,4 +1,4 @@
-# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
+# https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
 version: 2
 updates:
   - package-ecosystem: "pip"
@@ -8,6 +8,9 @@ updates:
       day: "friday"
     open-pull-requests-limit: 5
     target-branch: "master"
+    commit-message:
+      prefix: "[upd] pypi:"
+
   - package-ecosystem: "npm"
     directory: "/searx/static/themes/simple"
     schedule:
@@ -15,3 +18,5 @@ updates:
       day: "friday"
     open-pull-requests-limit: 5
     target-branch: "master"
+    commit-message:
+      prefix: "[upd] npm:"

.github/workflows/checker.yml

@@ -0,0 +1,31 @@
+name: "Checker"
+on:
+  schedule:
+    - cron: "0 4 * * 5"
+  workflow_dispatch:
+
+jobs:
+  checker:
+    name: Checker
+    runs-on: ubuntu-22.04
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Install Ubuntu packages
+        run: |
+          sudo ./utils/searxng.sh install packages
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.11'
+          architecture: 'x64'
+
+      - name: Install Python dependencies
+        run: |
+          make V=1 install
+
+      - name: Checker
+        run: |
+          make search.checker

.github/workflows/data-update.yml

@@ -7,7 +7,7 @@ on:
 jobs:
   updateData:
     name: Update data - ${{ matrix.fetch }}
-    runs-on: ubuntu-20.04
+    runs-on: ubuntu-24.04
     if: ${{ github.repository_owner == 'searxng'}}
     strategy:
       fail-fast: false
@@ -22,16 +22,16 @@ jobs:
           - update_engine_descriptions.py
     steps:
       - name: Checkout
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
 
       - name: Install Ubuntu packages
         run: |
           sudo ./utils/searxng.sh install packages
 
       - name: Set up Python
-        uses: actions/setup-python@v2
+        uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: '3.12'
           architecture: 'x64'
 
       - name: Install Python dependencies
@@ -46,18 +46,18 @@ jobs:
 
       - name: Create Pull Request
         id: cpr
-        uses: peter-evans/create-pull-request@v3
+        uses: peter-evans/create-pull-request@v6
         with:
-          commit-message: Update searx.data - ${{ matrix.fetch }}
+          commit-message: '[data] update searx.data - ${{ matrix.fetch }}'
           committer: searxng-bot <noreply@github.com>
           author: ${{ github.actor }} <${{ github.actor }}@users.noreply.github.com>
           signoff: false
           branch: update_data_${{ matrix.fetch }}
           delete-branch: true
           draft: false
-          title: 'Update searx.data - ${{ matrix.fetch }}'
+          title: '[data] update searx.data - ${{ matrix.fetch }}'
           body: |
-            Update searx.data - ${{ matrix.fetch }}
+            update searx.data - ${{ matrix.fetch }}
           labels: |
             data
 

.github/workflows/integration.yml

@@ -16,16 +16,16 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-20.04]
-        python-version:  ["3.8", "3.9", "3.10", "3.11"]
+        python-version:  ["3.9", "3.10", "3.11", "3.12",]
     steps:
     - name: Checkout
-      uses: actions/checkout@v2
+      uses: actions/checkout@v4
     - name: Install Ubuntu packages
       run: |
         sudo ./utils/searxng.sh install packages
         sudo apt install firefox
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
         architecture: 'x64'
@@ -45,27 +45,19 @@ jobs:
         make V=1 gecko.driver
     - name: Run tests
       run: make V=1 ci.test
-    - name: Test coverage
-      run: make V=1 test.coverage
-    - name: Store coverage result
-      uses: actions/upload-artifact@v2
-      with:
-        name: coverage-${{ matrix.python-version }}
-        path: coverage/
-        retention-days: 60
 
   themes:
     name: Themes
     runs-on: ubuntu-20.04
     steps:
     - name: Checkout
-      uses: actions/checkout@v2
+      uses: actions/checkout@v4
     - name: Install Ubuntu packages
       run: sudo ./utils/searxng.sh install buildhost
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
-        python-version: '3.9'
+        python-version: '3.12'
         architecture: 'x64'
     - name: Cache Python dependencies
       id: cache-python
@@ -75,7 +67,7 @@ jobs:
           ./local
           ./.nvm
           ./node_modules
-        key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py','.nvmrc', 'package.json') }}
+        key: python-ubuntu-20.04-3.12-${{ hashFiles('requirements*.txt', 'setup.py','.nvmrc', 'package.json') }}
     - name: Install node dependencies
       run: make V=1 node.env
     - name: Build themes
@@ -88,16 +80,16 @@ jobs:
       contents: write  # for JamesIves/github-pages-deploy-action to push changes in repo
     steps:
     - name: Checkout
-      uses: actions/checkout@v2
+      uses: actions/checkout@v4
       with:
         fetch-depth: '0'
         persist-credentials: false
     - name: Install Ubuntu packages
       run: sudo ./utils/searxng.sh install buildhost
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
-        python-version: '3.9'
+        python-version: '3.12'
         architecture: 'x64'
     - name: Cache Python dependencies
       id: cache-python
@@ -107,7 +99,7 @@ jobs:
           ./local
           ./.nvm
           ./node_modules
-        key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py','.nvmrc', 'package.json') }}
+        key: python-ubuntu-20.04-3.12-${{ hashFiles('requirements*.txt', 'setup.py','.nvmrc', 'package.json') }}
     - name: Build documentation
       run: |
         make V=1 docs.clean docs.html
@@ -120,7 +112,7 @@ jobs:
         FOLDER: dist/docs
         CLEAN: true # Automatically remove deleted files from the deploy branch
         SINGLE_COMMIT: True
-        COMMIT_MESSAGE: build from commit ${{ github.sha }}
+        COMMIT_MESSAGE: '[doc] build from commit ${{ github.sha }}'
 
   babel:
     name: Update translations branch
@@ -134,14 +126,14 @@ jobs:
       contents: write  # for make V=1 weblate.push.translations
     steps:
     - name: Checkout
-      uses: actions/checkout@v2
+      uses: actions/checkout@v4
       with:
         fetch-depth: '0'
         token: ${{ secrets.WEBLATE_GITHUB_TOKEN }}
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
-        python-version: '3.9'
+        python-version: '3.12'
         architecture: 'x64'
     - name: Cache Python dependencies
       id: cache-python
@@ -151,7 +143,7 @@ jobs:
           ./local
           ./.nvm
           ./node_modules
-        key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py','.nvmrc', 'package.json') }}
+        key: python-ubuntu-20.04-3.12-${{ hashFiles('requirements*.txt', 'setup.py','.nvmrc', 'package.json') }}
     - name: weblate & git setup
       env:
         WEBLATE_CONFIG: ${{ secrets.WEBLATE_CONFIG }}
@@ -163,7 +155,6 @@ jobs:
     - name: Update transations
       id: update
       run: |
-        git restore utils/brand.env
         make V=1 weblate.push.translations
 
   dockers:
@@ -179,14 +170,14 @@ jobs:
     steps:
       - name: Checkout
         if: env.DOCKERHUB_USERNAME != null
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
         with:
           # make sure "make docker.push" can get the git history
           fetch-depth: '0'
       - name: Set up Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
-          python-version: '3.9'
+          python-version: '3.12'
           architecture: 'x64'
       - name: Cache Python dependencies
         id: cache-python
@@ -196,7 +187,7 @@ jobs:
             ./local
             ./.nvm
             ./node_modules
-          key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py','.nvmrc', 'package.json') }}
+          key: python-ubuntu-20.04-3.12-${{ hashFiles('requirements*.txt', 'setup.py','.nvmrc', 'package.json') }}
       - name: Set up QEMU
         if: env.DOCKERHUB_USERNAME != null
         uses: docker/setup-qemu-action@v1

.github/workflows/security.yml

@@ -10,7 +10,7 @@ jobs:
     runs-on: ubuntu-20.04
     steps:
       - name: Checkout
-        uses: actions/checkout@v2
+        uses: actions/checkout@v4
 
       - name: Run Trivy vulnerability scanner
         uses: aquasecurity/trivy-action@master

.github/workflows/translations-update.yml

@@ -6,19 +6,19 @@ on:
 
 jobs:
   babel:
-    name: "create PR for additons from weblate"
+    name: "create PR for additions from weblate"
     runs-on: ubuntu-20.04
     if: ${{ github.repository_owner == 'searxng' && github.ref == 'refs/heads/master' }}
     steps:
     - name: Checkout
-      uses: actions/checkout@v2
+      uses: actions/checkout@v4
       with:
         fetch-depth: '0'
         token: ${{ secrets.WEBLATE_GITHUB_TOKEN }}
     - name: Set up Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
-        python-version: '3.9'
+        python-version: '3.12'
         architecture: 'x64'
     - name: Cache Python dependencies
       id: cache-python
@@ -28,7 +28,7 @@ jobs:
           ./local
           ./.nvm
           ./node_modules
-        key: python-ubuntu-20.04-3.9-${{ hashFiles('requirements*.txt', 'setup.py','.nvmrc', 'package.json') }}
+        key: python-ubuntu-20.04-3.12-${{ hashFiles('requirements*.txt', 'setup.py','.nvmrc', 'package.json') }}
     - name: weblate & git setup
       env:
         WEBLATE_CONFIG: ${{ secrets.WEBLATE_CONFIG }}
@@ -45,15 +45,15 @@ jobs:
       uses: peter-evans/create-pull-request@v3
       with:
         token: ${{ secrets.WEBLATE_GITHUB_TOKEN }}
-        commit-message: Update translations
+        commit-message: '[l10n] update translations from Weblate'
         committer: searxng-bot <searxng-bot@users.noreply.github.com>
         author: ${{ github.actor }} <${{ github.actor }}@users.noreply.github.com>
         signoff: false
         branch: translations_update
         delete-branch: true
         draft: false
-        title: 'Update translations'
+        title: '[l10n] update translations from Weblate'
         body: |
-          Update translations
+          update translations from Weblate
         labels: |
           translation

.gitignore

@@ -23,3 +23,6 @@ gh-pages/
 .idea/
 
 searx/version_frozen.py
+.dir-locals.el
+
+.python-version

.nvmrc

@@ -1 +1 @@
-v16.15.1
+v20.10

.pylintrc

@@ -1,4 +1,4 @@
-# -*- coding: utf-8; mode: conf -*-
+# -*- coding: utf-8; mode: conf-unix -*-
 # lint Python modules using external checkers.
 #
 # This is the main checker controlling the other ones and the reports
@@ -27,7 +27,7 @@ ignore-patterns=
 #init-hook=
 
 # Use multiple processes to speed up Pylint.
-jobs=1
+jobs=0
 
 # List of plugins (as comma separated values of python modules names) to load,
 # usually to register additional checkers.
@@ -338,6 +338,7 @@ valid-metaclass-classmethod-first-arg=mcs
 
 # Maximum number of arguments for function / method
 max-args=8
+max-positional-arguments=14
 
 # Maximum number of attributes for a class (see R0902).
 max-attributes=20

.tool-versions

@@ -0,0 +1,2 @@
+python 3.12.0
+shellcheck 0.9.0

AUTHORS.rst

@@ -168,3 +168,9 @@ features or generally made searx better:
 - Milad Laly @Milad-Laly
 - @llmII
 - @blob42 `<https://blob42.xyz>`_
+- Paolo Basso `<https://github.com/paolobasso99>`
+- Bernie Huang `<https://github.com/BernieHuang2008>`
+- Austin Olacsi `<https://github.com/Austin-Olacsi>`
+- @micsthepick
+- Daniel Kukula `<https://github.com/dkuku>`
+- Patrick Evans `https://github.com/holysoles`

CONTRIBUTING.md

@@ -2,7 +2,7 @@
 
 ## Resources in the documentation
 
-* [Development quickstart](https://docs.searxng.org/dev/contribution_guide.html)
+* [Development quickstart](https://docs.searxng.org/dev/quickstart.html)
 * [Contribution guide](https://docs.searxng.org/dev/contribution_guide.html)
 
 ## Submitting PRs

Dockerfile

@@ -1,4 +1,4 @@
-FROM alpine:3.18
+FROM alpine:3.20
 ENTRYPOINT ["/sbin/tini","--","/usr/local/searxng/dockerfiles/docker-entrypoint.sh"]
 EXPOSE 8080
 VOLUME /etc/searxng
@@ -15,7 +15,9 @@ ENV INSTANCE_NAME=searxng \
     MORTY_KEY= \
     MORTY_URL= \
     SEARXNG_SETTINGS_PATH=/etc/searxng/settings.yml \
-    UWSGI_SETTINGS_PATH=/etc/searxng/uwsgi.ini
+    UWSGI_SETTINGS_PATH=/etc/searxng/uwsgi.ini \
+    UWSGI_WORKERS=%k \
+    UWSGI_THREADS=4
 
 WORKDIR /usr/local/searxng
 
@@ -33,7 +35,6 @@ RUN apk add --no-cache -t build-dependencies \
     git \
  && apk add --no-cache \
     ca-certificates \
-    su-exec \
     python3 \
     py3-pip \
     libxml2 \
@@ -43,7 +44,7 @@ RUN apk add --no-cache -t build-dependencies \
     uwsgi \
     uwsgi-python3 \
     brotli \
- && pip3 install --no-cache -r requirements.txt \
+ && pip3 install --break-system-packages --no-cache -r requirements.txt \
  && apk del build-dependencies \
  && rm -rf /root/.cache
 

Makefile

@@ -44,10 +44,10 @@ lxc.clean:
 
 PHONY += search.checker search.checker.%
 search.checker: install
-	$(Q)./manage pyenv.cmd searx-checker -v
+	$(Q)./manage pyenv.cmd searxng-checker -v
 
 search.checker.%: install
-	$(Q)./manage pyenv.cmd searx-checker -v "$(subst _, ,$(patsubst search.checker.%,%,$@))"
+	$(Q)./manage pyenv.cmd searxng-checker -v "$(subst _, ,$(patsubst search.checker.%,%,$@))"
 
 PHONY += test ci.test test.shell
 ci.test: test.yamllint test.black test.pyright test.pylint test.unit test.robot test.rst test.pybabel
@@ -56,34 +56,32 @@ test.shell:
 	$(Q)shellcheck -x -s dash \
 		dockerfiles/docker-entrypoint.sh
 	$(Q)shellcheck -x -s bash \
-		utils/brand.env \
+		utils/brand.sh \
 		$(MTOOLS) \
 		utils/lib.sh \
-		utils/lib_nvm.sh \
-		utils/lib_static.sh \
+		utils/lib_sxng*.sh \
 		utils/lib_go.sh \
+		utils/lib_nvm.sh \
 		utils/lib_redis.sh \
-		utils/filtron.sh \
-		utils/searx.sh \
 		utils/searxng.sh \
-		utils/morty.sh \
 		utils/lxc.sh \
-		utils/lxc-searxng.env
+		utils/lxc-searxng.env \
+		utils/searx.sh \
+		utils/filtron.sh \
+		utils/morty.sh
 	$(Q)$(MTOOLS) build_msg TEST "$@ OK"
 
 
 # wrap ./manage script
 
-MANAGE += buildenv
 MANAGE += weblate.translations.commit weblate.push.translations
-MANAGE += data.all data.traits data.useragents
+MANAGE += data.all data.traits data.useragents data.locales
 MANAGE += docs.html docs.live docs.gh-pages docs.prebuild docs.clean
 MANAGE += docker.build docker.push docker.buildx
 MANAGE += gecko.driver
 MANAGE += node.env node.env.dev node.clean
 MANAGE += py.build py.clean
 MANAGE += pyenv pyenv.install pyenv.uninstall
-MANAGE += pypi.upload pypi.upload.test
 MANAGE += format.python
 MANAGE += test.yamllint test.pylint test.pyright test.black test.pybabel test.unit test.coverage test.robot test.rst test.clean
 MANAGE += themes.all themes.simple themes.simple.test pygments.less

PULL_REQUEST_TEMPLATE.md

@@ -12,11 +12,11 @@
 
 ## How to test this PR locally?
 
-<!-- commands to run the tests or instructions to test the changes-->
+<!-- commands to run the tests or instructions to test the changes -->
 
 ## Author's checklist
 
-<!-- additional notes for reviewiers -->
+<!-- additional notes for reviewers -->
 
 ## Related issues
 

README.rst

@@ -66,7 +66,7 @@ A user_, admin_ and developer_ handbook is available on the homepage_.
 Contact
 =======
 
-Ask questions or just chat about SearXNG on
+Ask questions or chat with the SearXNG community (this not a chatbot) on
 
 IRC
   `#searxng on libera.chat <https://web.libera.chat/?channel=#searxng>`_
@@ -75,76 +75,42 @@ IRC
 Matrix
   `#searxng:matrix.org <https://matrix.to/#/#searxng:matrix.org>`_
 
-Differences to searx
-====================
-
-SearXNG is a fork of `searx`_, with notable changes:
-
-.. _searx: https://github.com/searx/searx
-
-
-User experience
----------------
-
-- Reworked (and still simple) theme:
-
-  * Usable on desktop, tablet and mobile.
-  * Light and dark versions (available in the preferences).
-  * Right-to-left language support.
-  * `Screenshots <https://dev.searxng.org/screenshots.html>`_
-
-- The translations are up to date, you can contribute on `Weblate`_
-- The preferences page has been updated:
-
-  * Browse which engines are reliable or not.
-  * Engines are grouped inside each tab.
-  * Each engine has a description.
-
-- Thanks to the anonymous metrics, it is easier to report malfunctioning engines,
-  so they get fixed quicker
-
-  - `Turn off metrics on the server
-    <https://docs.searxng.org/admin/engines/settings.html#general>`_ if you don't want them recorded.
-
-- Administrators can `block and/or replace the URLs in the search results
-  <https://github.com/searxng/searxng/blob/5c1c0817c3996c5670a545d05831d234d21e6217/searx/settings.yml#L191-L199>`_
-
 
 Setup
------
+=====
 
-- No need for `Morty`_ to proxy images, even on a public instance.
-- No need for `Filtron`_ to block bots, as there is now a built-in `limiter`_.
-- A well maintained `Docker image`_, now also built for ARM64 and ARM/v7 architectures.
-  (Alternatively there are up to date installation scripts.)
+- A well maintained `Docker image`_, also built for ARM64 and ARM/v7
+  architectures.
+- Alternatively there are *up to date* `installation scripts`_.
+- For individual setup consult our detailed `Step by step`_ instructions.
+- To fine-tune your instance, take a look at the `Administrator documentation`_.
 
+.. _Administrator documentation: https://docs.searxng.org/admin/index.html
+.. _Step by step: https://docs.searxng.org/admin/installation-searxng.html
+.. _installation scripts: https://docs.searxng.org/admin/installation-scripts.html
 .. _Docker image: https://github.com/searxng/searxng-docker
 
-
-Contributing
-------------
-
-- Readable debug log.
-- Contributing is easier, thanks to the `Development Quickstart`_ guide.
-- A lot of code cleanup and bugfixes.
-- Up to date list dependencies.
-
-.. _Morty: https://github.com/asciimoo/morty
-.. _Filtron: https://github.com/searxng/filtron
-.. _limiter: https://docs.searxng.org/src/searx.plugins.limiter.html
-.. _Weblate: https://translate.codeberg.org/projects/searxng/searxng/
-.. _Development Quickstart: https://docs.searxng.org/dev/quickstart.html
-
-
 Translations
 ============
 
+.. _Weblate: https://translate.codeberg.org/projects/searxng/searxng/
+
 Help translate SearXNG at `Weblate`_
 
 .. figure:: https://translate.codeberg.org/widgets/searxng/-/multi-auto.svg
    :target: https://translate.codeberg.org/projects/searxng/
 
 
+Contributing
+============
+
+.. _development quickstart: https://docs.searxng.org/dev/quickstart.html
+.. _developer documentation: https://docs.searxng.org/dev/index.html
+
+Are you a developer?  Have a look at our `development quickstart`_ guide, it's
+very easy to contribute.  Additionally we have a `developer documentation`_.
+
+
 Codespaces
 ==========
 
@@ -155,9 +121,9 @@ You can contribute from your browser using `GitHub Codespaces`_:
 - Click on the ``Codespaces`` tab instead of ``Local``
 - Click on ``Create codespace on master``
 - VSCode is going to start in the browser
-- Wait for ``git pull && make install`` to appears and then to disapear
+- Wait for ``git pull && make install`` to appear and then disappear
 - You have `120 hours per month`_ (see also your `list of existing Codespaces`_)
-- You can start SearXNG using ``make run`` in the terminal or by pressing ``Ctrl+Shift+B``.
+- You can start SearXNG using ``make run`` in the terminal or by pressing ``Ctrl+Shift+B``
 
 .. _GitHub Codespaces: https://docs.github.com/en/codespaces/overview
 .. _120 hours per month: https://github.com/settings/billing

dockerfiles/docker-entrypoint.sh

@@ -175,4 +175,4 @@ unset MORTY_KEY
 
 # Start uwsgi
 printf 'Listen on %s\n' "${BIND_ADDRESS}"
-exec su-exec searxng:searxng uwsgi --master --http-socket "${BIND_ADDRESS}" "${UWSGI_SETTINGS_PATH}"
+exec uwsgi --master --uid searxng --gid searxng --http-socket "${BIND_ADDRESS}" "${UWSGI_SETTINGS_PATH}"

dockerfiles/uwsgi.ini

@@ -4,8 +4,12 @@ uid = searxng
 gid = searxng
 
 # Number of workers (usually CPU count)
-workers = %k
-threads = 4
+# default value: %k (= number of CPU core, see Dockerfile)
+workers = $(UWSGI_WORKERS)
+
+# Number of threads per worker
+# default value: 4 (see Dockerfile)
+threads = $(UWSGI_THREADS)
 
 # The right granted on the created socket
 chmod-socket = 666
@@ -38,12 +42,13 @@ buffer-size = 8192
 # See https://github.com/searx/searx-docker/issues/24
 add-header = Connection: close
 
+# Follow SIGTERM convention
+# See https://github.com/searxng/searxng/issues/3427
+die-on-term
+
 # uwsgi serves the static files
-# expires set to one year since there are hashes
 static-map = /static=/usr/local/searxng/searx/static
-static-expires = /* 31557600
+# expires set to one day
+static-expires = /* 86400
 static-gzip-all = True
 offload-threads = %k
-
-# Cache
-cache2 = name=searxngcache,items=2000,blocks=2000,blocksize=4096,bitmap=1

docs/_themes/searxng/theme.conf

@@ -4,3 +4,4 @@ stylesheet = searxng.css
 
 [options]
 touch_icon =
+globaltoc_maxdepth = 5

docs/admin/api.rst

@@ -69,10 +69,6 @@ Sample response
        {
          "enabled": true,
          "name": "HTTPS rewrite"
-       },
-       {
-         "enabled": false,
-         "name": "Vim-like hotkeys"
        }
      ],
      "safe_search": 0
@@ -88,9 +84,9 @@ HTML of the site.  URL of the SearXNG instance and values are customizable.
 .. code:: html
 
    <form method="post" action="https://example.org/">
-     <!-- search      --> <input type="text" name="q" />
-     <!-- categories  --> <input type="hidden" name="categories" value="general,social media" />
-     <!-- language    --> <input type="hidden" name="lang" value="all" />
-     <!-- locale      --> <input type="hidden" name="locale" value="en" />
-     <!-- date filter --> <input type="hidden" name="time_range" value="month" />
+     <!-- search      --> <input type="text" name="q">
+     <!-- categories  --> <input type="hidden" name="categories" value="general,social media">
+     <!-- language    --> <input type="hidden" name="lang" value="all">
+     <!-- locale      --> <input type="hidden" name="locale" value="en">
+     <!-- date filter --> <input type="hidden" name="time_range" value="month">
    </form>

docs/admin/buildhosts.rst

@@ -4,28 +4,36 @@
 Buildhosts
 ==========
 
-.. sidebar:: This article needs some work
-
-   If you have any contribution send us your :pull:`PR <../pulls>`, see
-   :ref:`how to contribute`.
-
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry
 
-To get best results from build, it's recommend to install additional packages
-on build hosts (see :ref:`searxng.sh`).::
+To get best results from build, it's recommend to install additional packages on
+build hosts (see :ref:`searxng.sh`).
 
-  sudo -H ./utils/searxng.sh install buildhost
+.. _searxng.sh install buildhost:
 
-This will install packages needed by searx:
+Build and Development tools
+===========================
+
+To Install tools used by build and development tasks in once:
+
+.. tabs::
+
+  .. group-tab:: SearXNG's development tools
+
+     .. code:: sh
+
+        $ sudo -H ./utils/searxng.sh install buildhost
+
+This will install packages needed by SearXNG:
 
 .. kernel-include:: $DOCS_BUILD/includes/searxng.rst
    :start-after: START distro-packages
    :end-before: END distro-packages
 
-and packages needed to build docuemtation and run tests:
+and packages needed to build documentation and run tests:
 
 .. kernel-include:: $DOCS_BUILD/includes/searxng.rst
    :start-after: START build-packages
@@ -73,7 +81,7 @@ If your docs build (``make docs.html``) shows warnings like this::
             display), check the imgmath_latex setting
 
 you need to install additional packages on your build host, to get better HTML
-output.
+output (:ref:`install buildhost <searxng.sh install buildhost>`).
 
 .. tabs::
 
@@ -93,7 +101,7 @@ output.
 
       .. code-block:: sh
 
-         $ sudo dnf install graphviz graphviz-gd texlive-xetex-bin librsvg2-tools
+         $ sudo dnf install graphviz graphviz-gd ImageMagick texlive-xetex-bin librsvg2-tools
 
 
 For PDF output you also need:
@@ -117,9 +125,8 @@ For PDF output you also need:
       .. code:: sh
 
       	 $ sudo dnf install \
-	        texlive-collection-fontsrecommended texlive-collection-latex \
-		dejavu-sans-fonts dejavu-serif-fonts dejavu-sans-mono-fonts \
-		ImageMagick
+             texlive-collection-fontsrecommended texlive-collection-latex \
+             dejavu-sans-fonts dejavu-serif-fonts dejavu-sans-mono-fonts
 
 .. _sh lint:
 
@@ -128,7 +135,8 @@ Lint shell scripts
 
 .. _ShellCheck: https://github.com/koalaman/shellcheck
 
-To lint shell scripts, we use ShellCheck_ - a shell script static analysis tool.
+To lint shell scripts we use ShellCheck_ - a shell script static analysis tool
+(:ref:`install buildhost <searxng.sh install buildhost>`).
 
 .. SNIP sh lint requirements
 

docs/admin/engines/command-line-engines.rst

@@ -1,79 +0,0 @@
-.. _engine command:
-
-====================
-Command Line Engines
-====================
-
-.. sidebar:: info
-
-   - :origin:`command.py <searx/engines/command.py>`
-   - :ref:`offline engines`
-
-With *command engines* administrators can run engines to integrate arbitrary
-shell commands.
-
-When creating and enabling a ``command`` engine on a public instance, you must
-be careful to avoid leaking private data.  The easiest solution is to limit the
-access by setting ``tokens`` as described in section :ref:`private engines`.
-
-The engine base is flexible.  Only your imagination can limit the power of this
-engine (and maybe security concerns).  The following options are available:
-
-``command``:
-  A comma separated list of the elements of the command.  A special token
-  ``{{QUERY}}`` tells where to put the search terms of the user. Example:
-
-  .. code:: yaml
-
-     ['ls', '-l', '-h', '{{QUERY}}']
-
-``delimiter``:
-  A mapping containing a delimiter ``char`` and the *titles* of each element in
-  ``keys``.
-
-``parse_regex``:
-  A dict containing the regular expressions for each result key.
-
-``query_type``:
-
-  The expected type of user search terms.  Possible values: ``path`` and
-  ``enum``.
-
-  ``path``:
-    Checks if the user provided path is inside the working directory.  If not,
-    the query is not executed.
-
-  ``enum``:
-    Is a list of allowed search terms.  If the user submits something which is
-    not included in the list, the query returns an error.
-
-``query_enum``:
-  A list containing allowed search terms if ``query_type`` is set to ``enum``.
-
-``working_dir``:
-
-  The directory where the command has to be executed.  Default: ``./``
-
-``result_separator``:
-  The character that separates results. Default: ``\n``
-
-The example engine below can be used to find files with a specific name in the
-configured working directory:
-
-.. code:: yaml
-
-  - name: find
-    engine: command
-    command: ['find', '.', '-name', '{{QUERY}}']
-    query_type: path
-    shortcut: fnd
-    delimiter:
-        chars: ' '
-        keys: ['line']
-
-
-Acknowledgment
-==============
-
-This development was sponsored by `Search and Discovery Fund
-<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.

docs/admin/engines/index.rst

@@ -1,26 +0,0 @@
-.. _engines and settings:
-
-==================
-Engines & Settings
-==================
-
-.. sidebar:: Further reading ..
-
-   - :ref:`settings engine`
-   - :ref:`engine settings` & :ref:`engine file`
-
-.. toctree::
-   :maxdepth: 3
-
-   settings
-
-.. toctree::
-   :maxdepth: 1
-
-   private-engines
-   recoll
-   sql-engines
-   nosql-engines
-   search-indexer-engines
-   command-line-engines
-   searx.engines.xpath

docs/admin/engines/private-engines.rst

@@ -1,49 +0,0 @@
-.. _private engines:
-
-============================
-Private Engines (``tokens``)
-============================
-
-Administrators might find themselves wanting to limit access to some of the
-enabled engines on their instances. It might be because they do not want to
-expose some private information through :ref:`offline engines`.  Or they would
-rather share engines only with their trusted friends or colleagues.
-
-To solve this issue the concept of *private engines* exists.
-
-
-A new option was added to engines named `tokens`. It expects a list of
-strings. If the user making a request presents one of the tokens of an engine,
-they can access information about the engine and make search requests.
-
-Example configuration to restrict access to the Arch Linux Wiki engine:
-
-.. code:: yaml
-
-  - name: arch linux wiki
-    engine: archlinux
-    shortcut: al
-    tokens: [ 'my-secret-token' ]
-
-
-Unless a user has configured the right token, the engine is going
-to be hidden from him/her. It is not going to be included in the
-list of engines on the Preferences page and in the output of
-`/config` REST API call.
-
-Tokens can be added to one's configuration on the Preferences page
-under "Engine tokens". The input expects a comma separated list of
-strings.
-
-The distribution of the tokens from the administrator to the users
-is not carved in stone. As providing access to such engines
-implies that the admin knows and trusts the user, we do not see
-necessary to come up with a strict process. Instead,
-we would like to add guidelines to the documentation of the feature.
-
-
-Acknowledgment
-==============
-
-This development was sponsored by `Search and Discovery Fund
-<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.

docs/admin/engines/recoll.rst

@@ -1,50 +0,0 @@
-.. _engine recoll:
-
-=============
-Recoll Engine
-=============
-
-.. sidebar:: info
-
-   - `Recoll <https://www.lesbonscomptes.com/recoll/>`_
-   - `recoll-webui <https://framagit.org/medoc92/recollwebui.git>`_
-   - :origin:`searx/engines/recoll.py`
-
-Recoll_ is a desktop full-text search tool based on Xapian.  By itself Recoll_
-does not offer WEB or API access, this can be achieved using recoll-webui_
-
-
-Configuration
-=============
-
-You must configure the following settings:
-
-``base_url``:
-  Location where recoll-webui can be reached.
-
-``mount_prefix``:
-  Location where the file hierarchy is mounted on your *local* filesystem.
-
-``dl_prefix``:
-  Location where the file hierarchy as indexed by recoll can be reached.
-
-``search_dir``:
-  Part of the indexed file hierarchy to be search, if empty the full domain is
-  searched.
-
-
-Example
-=======
-
-Scenario:
-
-#. Recoll indexes a local filesystem mounted in ``/export/documents/reference``,
-#. the Recoll search interface can be reached at https://recoll.example.org/ and
-#. the contents of this filesystem can be reached though https://download.example.org/reference
-
-.. code:: yaml
-
-   base_url: https://recoll.example.org/
-   mount_prefix: /export/documents
-   dl_prefix: https://download.example.org
-   search_dir: ''

docs/admin/engines/search-indexer-engines.rst

@@ -1,136 +0,0 @@
-====================
-Local Search Engines
-====================
-
-.. sidebar:: further read
-
-   - `Comparison to alternatives
-     <https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html>`_
-
-Administrators might find themselves wanting to integrate locally running search
-engines.  The following ones are supported for now:
-
-* `Elasticsearch`_
-* `Meilisearch`_
-* `Solr`_
-
-Each search engine is powerful, capable of full-text search.  All of the engines
-above are added to ``settings.yml`` just commented out, as you have to
-``base_url`` for all them.
-
-Please note that if you are not using HTTPS to access these engines, you have to enable
-HTTP requests by setting ``enable_http`` to ``True``.
-
-Furthermore, if you do not want to expose these engines on a public instance, you
-can still add them and limit the access by setting ``tokens`` as described in
-section :ref:`private engines`.
-
-.. _engine meilisearch:
-
-MeiliSearch
-===========
-
-.. sidebar:: info
-
-   - :origin:`meilisearch.py <searx/engines/meilisearch.py>`
-   - `MeiliSearch <https://www.meilisearch.com>`_
-   - `MeiliSearch Documentation <https://docs.meilisearch.com/>`_
-   - `Install MeiliSearch
-     <https://docs.meilisearch.com/learn/getting_started/installation.html>`_
-
-MeiliSearch_ is aimed at individuals and small companies.  It is designed for
-small-scale (less than 10 million documents) data collections.  E.g. it is great
-for storing web pages you have visited and searching in the contents later.
-
-The engine supports faceted search, so you can search in a subset of documents
-of the collection.  Furthermore, you can search in MeiliSearch_ instances that
-require authentication by setting ``auth_token``.
-
-Here is a simple example to query a Meilisearch instance:
-
-.. code:: yaml
-
-  - name: meilisearch
-    engine: meilisearch
-    shortcut: mes
-    base_url: http://localhost:7700
-    index: my-index
-    enable_http: true
-
-
-.. _engine elasticsearch:
-
-Elasticsearch
-=============
-
-.. sidebar:: info
-
-   - :origin:`elasticsearch.py <searx/engines/elasticsearch.py>`
-   - `Elasticsearch <https://www.elastic.co/elasticsearch/>`_
-   - `Elasticsearch Guide
-     <https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html>`_
-   - `Install Elasticsearch
-     <https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html>`_
-
-Elasticsearch_ supports numerous ways to query the data it is storing.  At the
-moment the engine supports the most popular search methods (``query_type``):
-
-- ``match``,
-- ``simple_query_string``,
-- ``term`` and
-- ``terms``.
-
-If none of the methods fit your use case, you can select ``custom`` query type
-and provide the JSON payload to submit to Elasticsearch in
-``custom_query_json``.
-
-The following is an example configuration for an Elasticsearch_ instance with
-authentication configured to read from ``my-index`` index.
-
-.. code:: yaml
-
-  - name: elasticsearch
-    shortcut: es
-    engine: elasticsearch
-    base_url: http://localhost:9200
-    username: elastic
-    password: changeme
-    index: my-index
-    query_type: match
-    # custom_query_json: '{ ... }'
-    enable_http: true
-
-.. _engine solr:
-
-Solr
-====
-
-.. sidebar:: info
-
-   - :origin:`solr.py <searx/engines/solr.py>`
-   - `Solr <https://solr.apache.org>`_
-   - `Solr Resources <https://solr.apache.org/resources.html>`_
-   - `Install Solr <https://solr.apache.org/guide/installing-solr.html>`_
-
-Solr_ is a popular search engine based on Lucene, just like Elasticsearch_.  But
-instead of searching in indices, you can search in collections.
-
-This is an example configuration for searching in the collection
-``my-collection`` and get the results in ascending order.
-
-.. code:: yaml
-
-  - name: solr
-    engine: solr
-    shortcut: slr
-    base_url: http://localhost:8983
-    collection: my-collection
-    sort: asc
-    enable_http: true
-
-
-Acknowledgment
-==============
-
-This development was sponsored by `Search and Discovery Fund
-<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.

docs/admin/engines/settings.rst

@@ -1,732 +0,0 @@
-.. _settings.yml:
-
-================
-``settings.yml``
-================
-
-This page describe the options possibilities of the :origin:`searx/settings.yml`
-file.
-
-.. sidebar:: Further reading ..
-
-   - :ref:`use_default_settings.yml`
-   - :ref:`search API`
-
-.. contents:: Contents
-   :depth: 2
-   :local:
-   :backlinks: entry
-
-.. _settings location:
-
-settings.yml location
-=====================
-
-The initial ``settings.yml`` we be load from these locations:
-
-1. the full path specified in the ``SEARXNG_SETTINGS_PATH`` environment variable.
-2. ``/etc/searxng/settings.yml``
-
-If these files don't exist (or are empty or can't be read), SearXNG uses the
-:origin:`searx/settings.yml` file.  Read :ref:`settings use_default_settings` to
-see how you can simplify your *user defined* ``settings.yml``.
-
-
-.. _settings global:
-
-Global Settings
-===============
-
-.. _settings brand:
-
-``brand:``
-----------
-
-.. code:: yaml
-
-   brand:
-     issue_url: https://github.com/searxng/searxng/issues
-     docs_url: https://docs.searxng.org
-     public_instances: https://searx.space
-     wiki_url: https://github.com/searxng/searxng/wiki
-
-``issue_url`` :
-  If you host your own issue tracker change this URL.
-
-``docs_url`` :
-  If you host your own documentation change this URL.
-
-``public_instances`` :
-  If you host your own https://searx.space change this URL.
-
-``wiki_url`` :
-  Link to your wiki (or ``false``)
-
-.. _settings general:
-
-``general:``
-------------
-
-.. code:: yaml
-
-   general:
-     debug: false
-     instance_name:  "SearXNG"
-     privacypolicy_url: false
-     donation_url: false
-     contact_url: false
-     enable_metrics: true
-
-``debug`` : ``$SEARXNG_DEBUG``
-  Allow a more detailed log if you run SearXNG directly. Display *detailed* error
-  messages in the browser too, so this must be deactivated in production.
-
-``donation_url`` :
-  Set value to ``true`` to use your own donation page written in the
-  :ref:`searx/info/en/donate.md <searx.infopage>` and use ``false`` to disable
-  the donation link altogether.
-
-``privacypolicy_url``:
-  Link to privacy policy.
-
-``contact_url``:
-  Contact ``mailto:`` address or WEB form.
-
-``enable_metrics``:
-  Enabled by default. Record various anonymous metrics availabled at ``/stats``,
-  ``/stats/errors`` and ``/preferences``.
-
-.. _settings search:
-
-``search:``
------------
-
-.. code:: yaml
-
-   search:
-     safe_search: 0
-     autocomplete: ""
-     default_lang: ""
-     ban_time_on_fail: 5
-     max_ban_time_on_fail: 120
-     suspended_times:
-       SearxEngineAccessDenied: 86400
-       SearxEngineCaptcha: 86400
-       SearxEngineTooManyRequests: 3600
-       cf_SearxEngineCaptcha: 1296000
-       cf_SearxEngineAccessDenied: 86400
-       recaptcha_SearxEngineCaptcha: 604800
-     formats:
-       - html
-
-``safe_search``:
-  Filter results.
-
-  - ``0``: None
-  - ``1``: Moderate
-  - ``2``: Strict
-
-``autocomplete``:
-  Existing autocomplete backends, leave blank to turn it off.
-
-  - ``dbpedia``
-  - ``duckduckgo``
-  - ``google``
-  - ``startpage``
-  - ``swisscows``
-  - ``qwant``
-  - ``wikipedia``
-
-``default_lang``:
-  Default search language - leave blank to detect from browser information or
-  use codes from :origin:`searx/languages.py`.
-
-``languages``:
-  List of available languages - leave unset to use all codes from
-  :origin:`searx/languages.py`.  Otherwise list codes of available languages.
-  The ``all`` value is shown as the ``Default language`` in the user interface
-  (in most cases, it is meant to send the query without a language parameter ;
-  in some cases, it means the English language) Example:
-
-  .. code:: yaml
-
-     languages:
-       - all
-       - en
-       - en-US
-       - de
-       - it-IT
-       - fr
-       - fr-BE
-
-``ban_time_on_fail``:
-  Ban time in seconds after engine errors.
-
-``max_ban_time_on_fail``:
-  Max ban time in seconds after engine errors.
-
-``suspended_times``:
-  Engine suspension time after error (in seconds; set to 0 to disable)
-
-  ``SearxEngineAccessDenied``: 86400
-    For error "Access denied" and "HTTP error [402, 403]"
-
-  ``SearxEngineCaptcha``: 86400
-    For error "CAPTCHA"
-
-  ``SearxEngineTooManyRequests``: 3600
-    For error "Too many request" and "HTTP error 429"
-
-  Cloudflare CAPTCHA:
-     - ``cf_SearxEngineCaptcha``: 1296000
-     - ``cf_SearxEngineAccessDenied``: 86400
-
-  Google CAPTCHA:
-    - ``recaptcha_SearxEngineCaptcha``: 604800
-
-``formats``:
-  Result formats available from web, remove format to deny access (use lower
-  case).
-
-  - ``html``
-  - ``csv``
-  - ``json``
-  - ``rss``
-
-
-.. _settings server:
-
-``server:``
------------
-
-.. code:: yaml
-
-   server:
-       base_url: http://example.org/location  # change this!
-       port: 8888
-       bind_address: "127.0.0.1"
-       secret_key: "ultrasecretkey"           # change this!
-       limiter: false
-       image_proxy: false
-       default_http_headers:
-         X-Content-Type-Options : nosniff
-         X-XSS-Protection : 1; mode=block
-         X-Download-Options : noopen
-         X-Robots-Tag : noindex, nofollow
-         Referrer-Policy : no-referrer
-
-
-``base_url`` : ``$SEARXNG_URL`` :ref:`buildenv <make buildenv>`
-  The base URL where SearXNG is deployed.  Used to create correct inbound links.
-  If you change the value, don't forget to rebuild instance's environment
-  (:ref:`utils/brand.env <make buildenv>`)
-
-``port`` & ``bind_address``: ``$SEARXNG_PORT`` & ``$SEARXNG_BIND_ADDRESS`` :ref:`buildenv <make buildenv>`
-  Port number and *bind address* of the SearXNG web application if you run it
-  directly using ``python searx/webapp.py``.  Doesn't apply to a SearXNG
-  services running behind a proxy and using socket communications.  If you
-  change the value, don't forget to rebuild instance's environment
-  (:ref:`utils/brand.env <make buildenv>`)
-
-``secret_key`` : ``$SEARXNG_SECRET``
-  Used for cryptography purpose.
-
-.. _limiter:
-
-``limiter`` :
-  Rate limit the number of request on the instance, block some bots.  The
-  :ref:`limiter src` requires a :ref:`settings redis` database.
-
-.. _image_proxy:
-
-``image_proxy`` :
-  Allow your instance of SearXNG of being able to proxy images.  Uses memory space.
-
-.. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
-
-``default_http_headers`` :
-  Set additional HTTP headers, see `#755 <https://github.com/searx/searx/issues/715>`__
-
-
-.. _settings ui:
-
-``ui:``
--------
-
-.. _cache busting:
-   https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#caching_static_assets_with_cache_busting
-
-.. code:: yaml
-
-   ui:
-     static_use_hash: false
-     default_locale: ""
-     query_in_title: false
-     infinite_scroll: false
-     center_alignment: false
-     cache_url: https://web.archive.org/web/
-     default_theme: simple
-     theme_args:
-       simple_style: auto
-
-.. _static_use_hash:
-
-``static_use_hash`` :
-  Enables `cache busting`_ of static files.
-
-``default_locale`` :
-  SearXNG interface language.  If blank, the locale is detected by using the
-  browser language.  If it doesn't work, or you are deploying a language
-  specific instance of searx, a locale can be defined using an ISO language
-  code, like ``fr``, ``en``, ``de``.
-
-``query_in_title`` :
-  When true, the result page's titles contains the query it decreases the
-  privacy, since the browser can records the page titles.
-
-``infinite_scroll``:
-  When true, automatically loads the next page when scrolling to bottom of the current page.
-
-``center_alignment`` : default ``false``
-  When enabled, the results are centered instead of being in the left (or RTL)
-  side of the screen.  This setting only affects the *desktop layout*
-  (:origin:`min-width: @tablet <searx/static/themes/simple/src/less/definitions.less>`)
-
-.. cache_url:
-
-``cache_url`` : ``https://web.archive.org/web/``
-  URL prefix of the internet archive or cache, don't forgett trailing slash (if
-  needed).  The default is https://web.archive.org/web/ alternatives are:
-
-  - https://webcache.googleusercontent.com/search?q=cache:
-  - https://archive.today/
-
-``default_theme`` :
-  Name of the theme you want to use by default on your SearXNG instance.
-
-``theme_args.simple_style``:
-  Style of simple theme: ``auto``, ``light``, ``dark``
-
-``results_on_new_tab``:
-  Open result links in a new tab by default.
-
-.. _settings redis:
-
-``redis:``
-----------
-
-.. _Redis.from_url(url): https://redis-py.readthedocs.io/en/stable/connections.html#redis.client.Redis.from_url
-
-A redis DB can be connected by an URL, in :py:obj:`searx.redisdb` you
-will find a description to test your redis connection in SerXNG.  When using
-sockets, don't forget to check the access rights on the socket::
-
-  ls -la /usr/local/searxng-redis/run/redis.sock
-  srwxrwx--- 1 searxng-redis searxng-redis ... /usr/local/searxng-redis/run/redis.sock
-
-In this example read/write access is given to the *searxng-redis* group.  To get
-access rights to redis instance (the socket), your SearXNG (or even your
-developer) account needs to be added to the *searxng-redis* group.
-
-``url``
-  URL to connect redis database, see `Redis.from_url(url)`_ & :ref:`redis db`::
-
-    redis://[[username]:[password]]@localhost:6379/0
-    rediss://[[username]:[password]]@localhost:6379/0
-    unix://[[username]:[password]]@/path/to/socket.sock?db=0
-
-.. admonition:: Tip for developers
-
-   To set up a local redis instance, first set the socket path of the Redis DB
-   in your YAML setting:
-
-   .. code:: yaml
-
-      redis:
-        url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
-
-   Then use the following commands to install the redis instance ::
-
-     $ ./manage redis.build
-     $ sudo -H ./manage redis.install
-     $ sudo -H ./manage redis.addgrp "${USER}"
-     # don't forget to logout & login to get member of group
-
-
-.. _settings outgoing:
-
-``outgoing:``
--------------
-
-Communication with search engines.
-
-.. code:: yaml
-
-   outgoing:
-     request_timeout: 2.0       # default timeout in seconds, can be override by engine
-     max_request_timeout: 10.0  # the maximum timeout in seconds
-     useragent_suffix: ""       # information like an email address to the administrator
-     pool_connections: 100      # Maximum number of allowable connections, or null
-                                # for no limits. The default is 100.
-     pool_maxsize: 10           # Number of allowable keep-alive connections, or null
-                                # to always allow. The default is 10.
-     enable_http2: true         # See https://www.python-httpx.org/http2/
-     # uncomment below section if you want to use a custom server certificate
-     # see https://www.python-httpx.org/advanced/#changing-the-verification-defaults
-     # and https://www.python-httpx.org/compatibility/#ssl-configuration
-     #  verify: ~/.mitmproxy/mitmproxy-ca-cert.cer
-     #
-     # uncomment below section if you want to use a proxyq see: SOCKS proxies
-     #   https://2.python-requests.org/en/latest/user/advanced/#proxies
-     # are also supported: see
-     #   https://2.python-requests.org/en/latest/user/advanced/#socks
-     #
-     #  proxies:
-     #    all://:
-     #      - http://proxy1:8080
-     #      - http://proxy2:8080
-     #
-     #  using_tor_proxy: true
-     #
-     # Extra seconds to add in order to account for the time taken by the proxy
-     #
-     #  extra_proxy_timeout: 10.0
-     #
-
-``request_timeout`` :
-  Global timeout of the requests made to others engines in seconds.  A bigger
-  timeout will allow to wait for answers from slow engines, but in consequence
-  will slow SearXNG reactivity (the result page may take the time specified in the
-  timeout to load). Can be override by :ref:`settings engine`
-
-``useragent_suffix`` :
-  Suffix to the user-agent SearXNG uses to send requests to others engines.  If an
-  engine wish to block you, a contact info here may be useful to avoid that.
-
-``keepalive_expiry`` :
-  Number of seconds to keep a connection in the pool. By default 5.0 seconds.
-
-.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying
-
-``proxies`` :
-  Define one or more proxies you wish to use, see `httpx proxies`_.
-  If there are more than one proxy for one protocol (http, https),
-  requests to the engines are distributed in a round-robin fashion.
-
-``source_ips`` :
-  If you use multiple network interfaces, define from which IP the requests must
-  be made. Example:
-
-  * ``0.0.0.0`` any local IPv4 address.
-  * ``::`` any local IPv6 address.
-  * ``192.168.0.1``
-  * ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses
-  * ``fe80::60a2:1691:e5a2:ee1f``
-  * ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network.
-  * ``[ 192.168.0.1, fe80::/126 ]``
-
-``retries`` :
-  Number of retry in case of an HTTP error.  On each retry, SearXNG uses an
-  different proxy and source ip.
-
-``retry_on_http_error`` :
-  Retry request on some HTTP status code.
-
-  Example:
-
-  * ``true`` : on HTTP status code between 400 and 599.
-  * ``403`` : on HTTP status code 403.
-  * ``[403, 429]``: on HTTP status code 403 and 429.
-
-``enable_http2`` :
-  Enable by default. Set to ``false`` to disable HTTP/2.
-
-.. _httpx verification defaults: https://www.python-httpx.org/advanced/#changing-the-verification-defaults
-.. _httpx ssl configuration: https://www.python-httpx.org/compatibility/#ssl-configuration
-
-``verify``: : ``$SSL_CERT_FILE``, ``$SSL_CERT_DIR``
-  Allow to specify a path to certificate.
-  see `httpx verification defaults`_.
-
-  In addition to ``verify``, SearXNG supports the ``$SSL_CERT_FILE`` (for a file) and
-  ``$SSL_CERT_DIR`` (for a directory) OpenSSL variables.
-  see `httpx ssl configuration`_.
-
-``max_redirects`` :
-  30 by default. Maximum redirect before it is an error.
-
-
-.. _settings categories_as_tabs:
-
-``categories_as_tabs:``
------------------------
-
-A list of the categories that are displayed as tabs in the user interface.
-Categories not listed here can still be searched with the :ref:`search-syntax`.
-
-.. code-block:: yaml
-
-  categories_as_tabs:
-    general:
-    images:
-    videos:
-    news:
-    map:
-    music:
-    it:
-    science:
-    files:
-    social media:
-
-Engines are added to ``categories:`` (compare :ref:`engine categories`), the
-categories listed in ``categories_as_tabs`` are shown as tabs in the UI.  If
-there are no active engines in a category, the tab is not displayed (e.g. if a
-user disables all engines in a category).
-
-On the preferences page (``/preferences``) -- under *engines* -- there is an
-additional tab, called *other*.  In this tab are all engines listed that are not
-in one of the UI tabs (not included in ``categories_as_tabs``).
-
-.. _settings engine:
-
-Engine settings
-===============
-
-.. sidebar:: Further reading ..
-
-   - :ref:`configured engines`
-   - :ref:`engines-dev`
-
-In the code example below a *full fledged* example of a YAML setup from a dummy
-engine is shown.  Most of the options have a default value or even are optional.
-
-.. code:: yaml
-
-   - name: example engine
-     engine: example
-     shortcut: demo
-     base_url: 'https://{language}.example.com/'
-     send_accept_language_header: false
-     categories: general
-     timeout: 3.0
-     api_key: 'apikey'
-     disabled: false
-     language: en_US
-     tokens: [ 'my-secret-token' ]
-     weight: 1
-     display_error_messages: true
-     about:
-        website: https://example.com
-        wikidata_id: Q306656
-        official_api_documentation: https://example.com/api-doc
-        use_official_api: true
-        require_api_key: true
-        results: HTML
-     enable_http: false
-     enable_http2: false
-     retries: 1
-     retry_on_http_error: true # or 403 or [404, 429]
-     max_connections: 100
-     max_keepalive_connections: 10
-     keepalive_expiry: 5.0
-     proxies:
-       http:
-         - http://proxy1:8080
-         - http://proxy2:8080
-       https:
-         - http://proxy1:8080
-         - http://proxy2:8080
-         - socks5://user:password@proxy3:1080
-         - socks5h://user:password@proxy4:1080
-
-``name`` :
-  Name that will be used across SearXNG to define this engine.  In settings, on
-  the result page...
-
-``engine`` :
-  Name of the python file used to handle requests and responses to and from this
-  search engine.
-
-``shortcut`` :
-  Code used to execute bang requests (in this case using ``!bi``)
-
-``base_url`` : optional
-  Part of the URL that should be stable across every request.  Can be useful to
-  use multiple sites using only one engine, or updating the site URL without
-  touching at the code.
-
-``send_accept_language_header`` :
-  Several engines that support languages (or regions) deal with the HTTP header
-  ``Accept-Language`` to build a response that fits to the locale.  When this
-  option is activated, the language (locale) that is selected by the user is used
-  to build and send a ``Accept-Language`` header in the request to the origin
-  search engine.
-
-.. _engine categories:
-
-``categories`` : optional
-  Specifies to which categories the engine should be added.  Engines can be
-  assigned to multiple categories.
-
-  Categories can be shown as tabs (:ref:`settings categories_as_tabs`) in the
-  UI.  A search in a tab (in the UI) will query all engines that are active in
-  this tab.  In the preferences page (``/preferences``) -- under *engines* --
-  users can select what engine should be active when querying in this tab.
-
-  Alternatively, :ref:`\!bang <search-syntax>` can be used to search all engines
-  in a category, regardless of whether they are active or not, or whether they
-  are in a tab of the UI or not.  For example, ``!dictionaries`` can be used to
-  query all search engines in that category (group).
-
-``timeout`` : optional
-  Timeout of the search with the current search engine.  **Be careful, it will
-  modify the global timeout of SearXNG.**
-
-``api_key`` : optional
-  In a few cases, using an API needs the use of a secret key.  How to obtain them
-  is described in the file.
-
-``disabled`` : optional
-  To disable by default the engine, but not deleting it.  It will allow the user
-  to manually activate it in the settings.
-
-``inactive``: optional
-  Remove the engine from the settings (*disabled & removed*).
-
-``language`` : optional
-  If you want to use another language for a specific engine, you can define it
-  by using the ISO code of language (and region), like ``fr``, ``en-US``,
-  ``de-DE``.
-
-``tokens`` : optional
-  A list of secret tokens to make this engine *private*, more details see
-  :ref:`private engines`.
-
-``weight`` : default ``1``
-  Weighting of the results of this engine.
-
-``display_error_messages`` : default ``true``
-  When an engine returns an error, the message is displayed on the user interface.
-
-``network`` : optional
-  Use the network configuration from another engine.
-  In addition, there are two default networks:
-
-  - ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses)
-  - ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses)
-
-.. note::
-
-   A few more options are possible, but they are pretty specific to some
-   engines, and so won't be described here.
-
-
-Example: Multilingual Search
-----------------------------
-
-SearXNG does not support true multilingual search.  You have to use the language
-prefix in your search query when searching in a different language.
-
-But there is a workaround: By adding a new search engine with a different
-language, SearXNG will search in your default and other language.
-
-Example configuration in settings.yml for a German and English speaker:
-
-.. code-block:: yaml
-
-    search:
-        default_lang : "de"
-        ...
-
-    engines:
-      - name : google english
-        engine : google
-        language : en
-        ...
-
-When searching, the default google engine will return German results and
-"google english" will return English results.
-
-
-.. _settings use_default_settings:
-
-use_default_settings
-====================
-
-.. sidebar:: ``use_default_settings: true``
-
-   - :ref:`settings location`
-   - :ref:`use_default_settings.yml`
-   - :origin:`/etc/searxng/settings.yml <utils/templates/etc/searxng/settings.yml>`
-
-The user defined ``settings.yml`` is loaded from the :ref:`settings location`
-and can relied on the default configuration :origin:`searx/settings.yml` using:
-
- ``use_default_settings: true``
-
-``server:``
-  In the following example, the actual settings are the default settings defined
-  in :origin:`searx/settings.yml` with the exception of the ``secret_key`` and
-  the ``bind_address``:
-
-  .. code-block:: yaml
-
-    use_default_settings: true
-    server:
-        secret_key: "ultrasecretkey"   # change this!
-        bind_address: "0.0.0.0"
-
-``engines:``
-  With ``use_default_settings: true``, each settings can be override in a
-  similar way, the ``engines`` section is merged according to the engine
-  ``name``.  In this example, SearXNG will load all the default engines, will
-  enable the ``bing`` engine and define a :ref:`token <private engines>` for
-  the arch linux engine:
-
-  .. code-block:: yaml
-
-    use_default_settings: true
-    server:
-      secret_key: "ultrasecretkey"   # change this!
-    engines:
-      - name: arch linux wiki
-        tokens: ['$ecretValue']
-      - name: bing
-        disabled: false
-
-
-``engines:`` / ``remove:``
-  It is possible to remove some engines from the default settings. The following
-  example is similar to the above one, but SearXNG doesn't load the the google
-  engine:
-
-  .. code-block:: yaml
-
-    use_default_settings:
-      engines:
-        remove:
-          - google
-    server:
-      secret_key: "ultrasecretkey"   # change this!
-    engines:
-      - name: arch linux wiki
-        tokens: ['$ecretValue']
-
-``engines:`` / ``keep_only:``
-  As an alternative, it is possible to specify the engines to keep. In the
-  following example, SearXNG has only two engines:
-
-  .. code-block:: yaml
-
-    use_default_settings:
-      engines:
-        keep_only:
-          - google
-          - duckduckgo
-    server:
-      secret_key: "ultrasecretkey"   # change this!
-    engines:
-      - name: google
-        tokens: ['$ecretValue']
-      - name: duckduckgo
-        tokens: ['$ecretValue']

docs/admin/index.rst

@@ -4,8 +4,8 @@ Administrator documentation
 
 .. toctree::
    :maxdepth: 2
-   :caption: Contents
 
+   settings/index
    installation
    installation-docker
    installation-scripts
@@ -15,7 +15,8 @@ Administrator documentation
    installation-apache
    update-searxng
    answer-captcha
-   engines/index
+   searx.favicons
+   searx.limiter
    api
    architecture
    plugins

docs/admin/installation-apache.rst

@@ -61,7 +61,7 @@ section might give you some guidance.
    - `Apache Fedora`_
    - `Apache directives`_
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry
@@ -190,7 +190,7 @@ Debian:
 Apache modules
 --------------
 
-To load additional modules, in most distributions you have to un-comment the
+To load additional modules, in most distributions you have to uncomment the
 lines with the corresponding LoadModule_ directive, except in :ref:`Debian's
 Apache layout`.
 

docs/admin/installation-docker.rst

@@ -43,7 +43,7 @@ of this container:
 
 - enables :ref:`limiter <limiter>` to protect against bots
 - enables :ref:`image proxy <image_proxy>` for better privacy
-- enables :ref:`cache busting <static_use_hash>` to save bandwith
+- enables :ref:`cache busting <static_use_hash>` to save bandwidth
 
 ----
 
@@ -92,6 +92,9 @@ instance using `docker run <https://docs.docker.com/engine/reference/run/>`_:
                 searxng/searxng
    2f998.... # container's ID
 
+The environment variables UWSGI_WORKERS and UWSGI_THREADS overwrite the default
+number of UWSGI processes and UWSGI threads specified in `/etc/searxng/uwsgi.ini`.
+
 Open your WEB browser and visit the URL:
 
 .. code:: sh
@@ -108,7 +111,7 @@ can modify these files according to your needs and restart the Docker image.
 Use command ``container ls`` to list running containers, add flag `-a
 <https://docs.docker.com/engine/reference/commandline/container_ls>`__ to list
 exited containers also.  With ``container stop`` a running container can be
-stoped.  To get rid of a container use ``container rm``:
+stopped.  To get rid of a container use ``container rm``:
 
 .. code:: sh
 

docs/admin/installation-nginx.rst

@@ -41,7 +41,7 @@ section might give you some guidance.
    - `uWSGI support from nginx`_
 
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry

docs/admin/installation-searxng.rst

@@ -4,7 +4,7 @@
 Step by step installation
 =========================
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry
@@ -73,7 +73,7 @@ Configuration
 
 .. sidebar:: ``use_default_settings: True``
 
-   - :ref:`settings global`
+   - :ref:`settings.yml`
    - :ref:`settings location`
    - :ref:`settings use_default_settings`
    - :origin:`/etc/searxng/settings.yml <utils/templates/etc/searxng/settings.yml>`
@@ -86,7 +86,7 @@ below. This setup:
 
 - enables :ref:`limiter <limiter>` to protect against bots
 - enables :ref:`image proxy <image_proxy>` for better privacy
-- enables :ref:`cache busting <static_use_hash>` to save bandwith
+- enables :ref:`cache busting <static_use_hash>` to save bandwidth
 
 Modify the ``/etc/searxng/settings.yml`` to your needs:
 
@@ -96,7 +96,7 @@ Modify the ``/etc/searxng/settings.yml`` to your needs:
 
      .. literalinclude:: ../../utils/templates/etc/searxng/settings.yml
         :language: yaml
-        :end-before: # hostname_replace:
+        :end-before: # hostnames:
 
      To see the entire file jump to :origin:`utils/templates/etc/searxng/settings.yml`
 
@@ -104,7 +104,7 @@ Modify the ``/etc/searxng/settings.yml`` to your needs:
 
      .. literalinclude:: ../../searx/settings.yml
         :language: yaml
-        :end-before: # hostname_replace:
+        :end-before: # hostnames:
 
      To see the entire file jump to :origin:`searx/settings.yml`
 

docs/admin/installation-uwsgi.rst

@@ -9,7 +9,7 @@ uWSGI
    - `systemd.unit`_
    - `uWSGI Emperor`_
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry

docs/admin/searx.favicons.rst

@@ -0,0 +1,251 @@
+.. _favicons:
+
+========
+Favicons
+========
+
+.. sidebar:: warning
+
+   Don't activate the favicons before reading the documentation.
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+Activating the favicons in SearXNG is very easy, but this **generates a
+significantly higher load** in the client/server communication and increases
+resources needed on the server.
+
+To mitigate these disadvantages, various methods have been implemented,
+including a *cache*.  The cache must be parameterized according to your own
+requirements and maintained regularly.
+
+To activate favicons in SearXNG's result list, set a default
+``favicon_resolver`` in the :ref:`search <settings search>` settings:
+
+.. code:: yaml
+
+   search:
+     favicon_resolver: "duckduckgo"
+
+By default and without any extensions, SearXNG serves these resolvers:
+
+- ``duckduckgo``
+- ``allesedv``
+- ``google``
+- ``yandex``
+
+With the above setting favicons are displayed, the user has the option to
+deactivate this feature in his settings.  If the user is to have the option of
+selecting from several *resolvers*, a further setting is required / but this
+setting will be discussed :ref:`later <register resolvers>` in this article,
+first we have to setup the favicons cache.
+
+Infrastructure
+==============
+
+The infrastructure for providing the favicons essentially consists of three
+parts:
+
+- :py:obj:`Favicons-Proxy <.favicons.proxy>` (aka *proxy*)
+- :py:obj:`Favicons-Resolvers <.favicons.resolvers>` (aka *resolver*)
+- :py:obj:`Favicons-Cache <.favicons.cache>` (aka *cache*)
+
+To protect the privacy of users, the favicons are provided via a *proxy*.  This
+*proxy* is automatically activated with the above activation of a *resolver*.
+Additional requests are required to provide the favicons: firstly, the *proxy*
+must process the incoming requests and secondly, the *resolver* must make
+outgoing requests to obtain the favicons from external sources.
+
+A *cache* has been developed to massively reduce both, incoming and outgoing
+requests.  This *cache* is also activated automatically with the above
+activation of a *resolver*.  In its defaults, however, the *cache* is minimal
+and not well suitable for a production environment!
+
+.. _favicon cache setup:
+
+Setting up the cache
+====================
+
+To parameterize the *cache* and more settings of the favicons infrastructure, a
+TOML_ configuration is created in the file ``/etc/searxng/favicons.toml``.
+
+.. code:: toml
+
+   [favicons]
+
+   cfg_schema = 1   # config's schema version no.
+
+   [favicons.cache]
+
+   db_url = "/var/cache/searxng/faviconcache.db"  # default: "/tmp/faviconcache.db"
+   LIMIT_TOTAL_BYTES = 2147483648                 # 2 GB / default: 50 MB
+   # HOLD_TIME = 5184000                            # 60 days / default: 30 days
+   # BLOB_MAX_BYTES = 40960                         # 40 KB / default 20 KB
+   # MAINTENANCE_MODE = "off"                       # default: "auto"
+   # MAINTENANCE_PERIOD = 600                       # 10min / default: 1h
+
+:py:obj:`cfg_schema <.FaviconConfig.cfg_schema>`:
+  Is required to trigger any processes required for future upgrades / don't
+  change it.
+
+:py:obj:`cache.db_url <.FaviconCacheConfig.db_url>`:
+  The path to the (SQLite_) database file.  The default path is in the `/tmp`_
+  folder, which is deleted on every reboot and is therefore unsuitable for a
+  production environment.  The FHS_ provides the folder  for the
+  application cache
+
+  The FHS_ provides the folder `/var/cache`_ for the cache of applications, so a
+  suitable storage location of SearXNG's caches is folder ``/var/cache/searxng``.
+  In container systems, a volume should be mounted for this folder and in a
+  standard installation (compare :ref:`create searxng user`), the folder must be
+  created and the user under which the SearXNG process is running must be given
+  write permission to this folder.
+
+  .. code:: bash
+
+     $ sudo mkdir /var/cache/searxng
+     $ sudo chown root:searxng /var/cache/searxng/
+     $ sudo chmod g+w /var/cache/searxng/
+
+:py:obj:`cache.LIMIT_TOTAL_BYTES <.FaviconCacheConfig.LIMIT_TOTAL_BYTES>`:
+  Maximum of bytes stored in the cache of all blobs.  The limit is only reached
+  at each maintenance interval after which the oldest BLOBs are deleted; the
+  limit is exceeded during the maintenance period.
+
+  .. attention::
+
+     If the maintenance period is too long or maintenance is switched
+     off completely, the cache grows uncontrollably.
+
+SearXNG hosters can change other parameters of the cache as required:
+
+- :py:obj:`cache.HOLD_TIME <.FaviconCacheConfig.HOLD_TIME>`
+- :py:obj:`cache.BLOB_MAX_BYTES <.FaviconCacheConfig.BLOB_MAX_BYTES>`
+
+
+Maintenance of the cache
+------------------------
+
+Regular maintenance of the cache is required!  By default, regular maintenance
+is triggered automatically as part of the client requests:
+
+- :py:obj:`cache.MAINTENANCE_MODE <.FaviconCacheConfig.MAINTENANCE_MODE>` (default ``auto``)
+- :py:obj:`cache.MAINTENANCE_PERIOD <.FaviconCacheConfig.MAINTENANCE_PERIOD>` (default ``6000`` / 1h)
+
+As an alternative to maintenance as part of the client request process, it is
+also possible to carry out maintenance using an external process. For example,
+by creating a :man:`crontab` entry for maintenance:
+
+.. code:: bash
+
+   $ python -m searx.favicons cache maintenance
+
+The following command can be used to display the state of the cache:
+
+.. code:: bash
+
+   $ python -m searx.favicons cache state
+
+
+.. _favicon proxy setup:
+
+Proxy configuration
+===================
+
+Most of the options of the :py:obj:`Favicons-Proxy <.favicons.proxy>` are
+already set sensibly with settings from the :ref:`settings.yml <searxng
+settings.yml>` and should not normally be adjusted.
+
+.. code:: toml
+
+   [favicons.proxy]
+
+   max_age = 5184000             # 60 days / default: 7 days (604800 sec)
+
+
+:py:obj:`max_age <.FaviconProxyConfig.max_age>`:
+  The `HTTP Cache-Control max-age`_ response directive indicates that the
+  response remains fresh until N seconds after the response is generated.  This
+  setting therefore determines how long a favicon remains in the client's cache.
+  As a rule, in the favicons infrastructure of SearXNG's this setting only
+  affects favicons whose byte size exceeds :ref:`BLOB_MAX_BYTES <favicon cache
+  setup>` (the other favicons that are already in the cache are embedded as
+  `data URL`_ in the :py:obj:`generated HTML <.favicons.proxy.favicon_url>`,
+  which can greatly reduce the number of additional requests).
+
+.. _register resolvers:
+
+Register resolvers
+------------------
+
+A :py:obj:`resolver <.favicon.resolvers>` is a function that obtains the favicon
+from an external source.  The resolver functions available to the user are
+registered with their fully qualified name (FQN_) in a ``resolver_map``.
+
+If no ``resolver_map`` is defined in the ``favicon.toml``, the favicon
+infrastructure of SearXNG generates this ``resolver_map`` automatically
+depending on the ``settings.yml``.  SearXNG would automatically generate the
+following TOML configuration from the following YAML configuration:
+
+.. code:: yaml
+
+   search:
+     favicon_resolver: "duckduckgo"
+
+.. code:: toml
+
+   [favicons.proxy.resolver_map]
+
+   "duckduckgo" = "searx.favicons.resolvers.duckduckgo"
+
+If this automatism is not desired, then (and only then) a separate
+``resolver_map`` must be created.  For example, to give the user two resolvers to
+choose from, the following configuration could be used:
+
+.. code:: toml
+
+   [favicons.proxy.resolver_map]
+
+   "duckduckgo" = "searx.favicons.resolvers.duckduckgo"
+   "allesedv" = "searx.favicons.resolvers.allesedv"
+   # "google" = "searx.favicons.resolvers.google"
+   # "yandex" = "searx.favicons.resolvers.yandex"
+
+.. note::
+
+   With each resolver, the resource requirement increases significantly.
+
+The number of resolvers increases:
+
+- the number of incoming/outgoing requests and
+- the number of favicons to be stored in the cache.
+
+In the following we list the resolvers available in the core of SearXNG, but via
+the FQN_ it is also possible to implement your own resolvers and integrate them
+into the *proxy*:
+
+- :py:obj:`searx.favicons.resolvers.duckduckgo`
+- :py:obj:`searx.favicons.resolvers.allesedv`
+- :py:obj:`searx.favicons.resolvers.google`
+- :py:obj:`searx.favicons.resolvers.yandex`
+
+
+
+.. _SQLite:
+   https://www.sqlite.org/
+.. _FHS:
+   https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html
+.. _`/var/cache`:
+   https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s05.html
+.. _`/tmp`:
+   https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s18.html
+.. _TOML:
+    https://toml.io/en/
+.. _HTTP Cache-Control max-age:
+   https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#response_directives
+.. _data URL:
+   https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs
+.. _FQN: https://en.wikipedia.org/wiki/Fully_qualified_name
+

docs/admin/searx.limiter.rst

@@ -0,0 +1,17 @@
+.. _limiter:
+
+=======
+Limiter
+=======
+
+.. sidebar:: info
+
+   The limiter requires a :ref:`Redis <settings redis>` database.
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.limiter
+   :members:

docs/admin/settings/index.rst

@@ -0,0 +1,27 @@
+.. _searxng settings.yml:
+
+========
+Settings
+========
+
+.. sidebar:: Further reading ..
+
+   - :ref:`engine settings`
+   - :ref:`engine file`
+
+.. toctree::
+   :maxdepth: 2
+
+   settings
+   settings_engine
+   settings_brand
+   settings_general
+   settings_search
+   settings_server
+   settings_ui
+   settings_redis
+   settings_outgoing
+   settings_categories_as_tabs
+
+
+

docs/admin/settings/settings.rst

@@ -0,0 +1,117 @@
+.. _settings.yml:
+
+================
+``settings.yml``
+================
+
+This page describe the options possibilities of the :origin:`searx/settings.yml`
+file.
+
+.. sidebar:: Further reading ..
+
+   - :ref:`use_default_settings.yml`
+   - :ref:`search API`
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. _settings location:
+
+settings.yml location
+=====================
+
+The initial ``settings.yml`` we be load from these locations:
+
+1. the full path specified in the ``SEARXNG_SETTINGS_PATH`` environment variable.
+2. ``/etc/searxng/settings.yml``
+
+If these files don't exist (or are empty or can't be read), SearXNG uses the
+:origin:`searx/settings.yml` file.  Read :ref:`settings use_default_settings` to
+see how you can simplify your *user defined* ``settings.yml``.
+
+
+
+.. _settings use_default_settings:
+
+use_default_settings
+====================
+
+.. sidebar:: ``use_default_settings: true``
+
+   - :ref:`settings location`
+   - :ref:`use_default_settings.yml`
+   - :origin:`/etc/searxng/settings.yml <utils/templates/etc/searxng/settings.yml>`
+
+The user defined ``settings.yml`` is loaded from the :ref:`settings location`
+and can relied on the default configuration :origin:`searx/settings.yml` using:
+
+ ``use_default_settings: true``
+
+``server:``
+  In the following example, the actual settings are the default settings defined
+  in :origin:`searx/settings.yml` with the exception of the ``secret_key`` and
+  the ``bind_address``:
+
+  .. code:: yaml
+
+    use_default_settings: true
+    server:
+        secret_key: "ultrasecretkey"   # change this!
+        bind_address: "0.0.0.0"
+
+``engines:``
+  With ``use_default_settings: true``, each settings can be override in a
+  similar way, the ``engines`` section is merged according to the engine
+  ``name``.  In this example, SearXNG will load all the default engines, will
+  enable the ``bing`` engine and define a :ref:`token <private engines>` for
+  the arch linux engine:
+
+  .. code:: yaml
+
+    use_default_settings: true
+    server:
+      secret_key: "ultrasecretkey"   # change this!
+    engines:
+      - name: arch linux wiki
+        tokens: ['$ecretValue']
+      - name: bing
+        disabled: false
+
+
+``engines:`` / ``remove:``
+  It is possible to remove some engines from the default settings. The following
+  example is similar to the above one, but SearXNG doesn't load the the google
+  engine:
+
+  .. code:: yaml
+
+    use_default_settings:
+      engines:
+        remove:
+          - google
+    server:
+      secret_key: "ultrasecretkey"   # change this!
+    engines:
+      - name: arch linux wiki
+        tokens: ['$ecretValue']
+
+``engines:`` / ``keep_only:``
+  As an alternative, it is possible to specify the engines to keep. In the
+  following example, SearXNG has only two engines:
+
+  .. code:: yaml
+
+    use_default_settings:
+      engines:
+        keep_only:
+          - google
+          - duckduckgo
+    server:
+      secret_key: "ultrasecretkey"   # change this!
+    engines:
+      - name: google
+        tokens: ['$ecretValue']
+      - name: duckduckgo
+        tokens: ['$ecretValue']

docs/admin/settings/settings_brand.rst

@@ -0,0 +1,25 @@
+.. _settings brand:
+
+==========
+``brand:``
+==========
+
+.. code:: yaml
+
+   brand:
+     issue_url: https://github.com/searxng/searxng/issues
+     docs_url: https://docs.searxng.org
+     public_instances: https://searx.space
+     wiki_url: https://github.com/searxng/searxng/wiki
+
+``issue_url`` :
+  If you host your own issue tracker change this URL.
+
+``docs_url`` :
+  If you host your own documentation change this URL.
+
+``public_instances`` :
+  If you host your own https://searx.space change this URL.
+
+``wiki_url`` :
+  Link to your wiki (or ``false``)

docs/admin/settings/settings_categories_as_tabs.rst

@@ -0,0 +1,31 @@
+.. _settings categories_as_tabs:
+
+=======================
+``categories_as_tabs:``
+=======================
+
+A list of the categories that are displayed as tabs in the user interface.
+Categories not listed here can still be searched with the :ref:`search-syntax`.
+
+.. code:: yaml
+
+  categories_as_tabs:
+    general:
+    images:
+    videos:
+    news:
+    map:
+    music:
+    it:
+    science:
+    files:
+    social media:
+
+Engines are added to ``categories:`` (compare :ref:`engine categories`), the
+categories listed in ``categories_as_tabs`` are shown as tabs in the UI.  If
+there are no active engines in a category, the tab is not displayed (e.g. if a
+user disables all engines in a category).
+
+On the preferences page (``/preferences``) -- under *engines* -- there is an
+additional tab, called *other*.  In this tab are all engines listed that are not
+in one of the UI tabs (not included in ``categories_as_tabs``).

docs/admin/settings/settings_engine.rst

@@ -0,0 +1,244 @@
+.. _settings engine:
+
+===========
+``engine:``
+===========
+
+.. sidebar:: Further reading ..
+
+   - :ref:`configured engines`
+   - :ref:`engines-dev`
+
+In the code example below a *full fledged* example of a YAML setup from a dummy
+engine is shown.  Most of the options have a default value or even are optional.
+
+.. hint::
+
+   A few more options are possible, but they are pretty specific to some
+   engines (:ref:`engine implementations`).
+
+.. code:: yaml
+
+   - name: example engine
+     engine: example
+     shortcut: demo
+     base_url: 'https://{language}.example.com/'
+     send_accept_language_header: false
+     categories: general
+     timeout: 3.0
+     api_key: 'apikey'
+     disabled: false
+     language: en_US
+     tokens: [ 'my-secret-token' ]
+     weight: 1
+     display_error_messages: true
+     about:
+        website: https://example.com
+        wikidata_id: Q306656
+        official_api_documentation: https://example.com/api-doc
+        use_official_api: true
+        require_api_key: true
+        results: HTML
+
+     # overwrite values from section 'outgoing:'
+     enable_http2: false
+     retries: 1
+     max_connections: 100
+     max_keepalive_connections: 10
+     keepalive_expiry: 5.0
+     using_tor_proxy: false
+     proxies:
+       http:
+         - http://proxy1:8080
+         - http://proxy2:8080
+       https:
+         - http://proxy1:8080
+         - http://proxy2:8080
+         - socks5://user:password@proxy3:1080
+         - socks5h://user:password@proxy4:1080
+
+     # other network settings
+     enable_http: false
+     retry_on_http_error: true # or 403 or [404, 429]
+
+
+``name`` :
+  Name that will be used across SearXNG to define this engine.  In settings, on
+  the result page...
+
+``engine`` :
+  Name of the python file used to handle requests and responses to and from this
+  search engine.
+
+``shortcut`` :
+  Code used to execute bang requests (in this case using ``!bi``)
+
+``base_url`` : optional
+  Part of the URL that should be stable across every request.  Can be useful to
+  use multiple sites using only one engine, or updating the site URL without
+  touching at the code.
+
+``send_accept_language_header`` :
+  Several engines that support languages (or regions) deal with the HTTP header
+  ``Accept-Language`` to build a response that fits to the locale.  When this
+  option is activated, the language (locale) that is selected by the user is used
+  to build and send a ``Accept-Language`` header in the request to the origin
+  search engine.
+
+.. _engine categories:
+
+``categories`` : optional
+  Specifies to which categories the engine should be added.  Engines can be
+  assigned to multiple categories.
+
+  Categories can be shown as tabs (:ref:`settings categories_as_tabs`) in the
+  UI.  A search in a tab (in the UI) will query all engines that are active in
+  this tab.  In the preferences page (``/preferences``) -- under *engines* --
+  users can select what engine should be active when querying in this tab.
+
+  Alternatively, :ref:`\!bang <search-syntax>` can be used to search all engines
+  in a category, regardless of whether they are active or not, or whether they
+  are in a tab of the UI or not.  For example, ``!dictionaries`` can be used to
+  query all search engines in that category (group).
+
+``timeout`` : optional
+  Timeout of the search with the current search engine.  Overwrites
+  ``request_timeout`` from :ref:`settings outgoing`.  **Be careful, it will
+  modify the global timeout of SearXNG.**
+
+``api_key`` : optional
+  In a few cases, using an API needs the use of a secret key.  How to obtain them
+  is described in the file.
+
+``disabled`` : optional
+  To disable by default the engine, but not deleting it.  It will allow the user
+  to manually activate it in the settings.
+
+``inactive``: optional
+  Remove the engine from the settings (*disabled & removed*).
+
+``language`` : optional
+  If you want to use another language for a specific engine, you can define it
+  by using the ISO code of language (and region), like ``fr``, ``en-US``,
+  ``de-DE``.
+
+``tokens`` : optional
+  A list of secret tokens to make this engine *private*, more details see
+  :ref:`private engines`.
+
+``weight`` : default ``1``
+  Weighting of the results of this engine.
+
+``display_error_messages`` : default ``true``
+  When an engine returns an error, the message is displayed on the user interface.
+
+``network`` : optional
+  Use the network configuration from another engine.
+  In addition, there are two default networks:
+
+  - ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses)
+  - ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses)
+
+``enable_http`` : optional
+  Enable HTTP for this engine (by default only HTTPS is enabled).
+
+``retry_on_http_error`` : optional
+  Retry request on some HTTP status code.
+
+  Example:
+
+  * ``true`` : on HTTP status code between 400 and 599.
+  * ``403`` : on HTTP status code 403.
+  * ``[403, 429]``: on HTTP status code 403 and 429.
+
+``proxies`` :
+  Overwrites proxy settings from :ref:`settings outgoing`.
+
+``using_tor_proxy`` :
+  Using tor proxy (``true``) or not (``false``) for this engine.  The default is
+  taken from ``using_tor_proxy`` of the :ref:`settings outgoing`.
+
+.. _Pool limit configuration: https://www.python-httpx.org/advanced/#pool-limit-configuration
+
+``max_keepalive_connection#s`` :
+  `Pool limit configuration`_, overwrites value ``pool_maxsize`` from
+   :ref:`settings outgoing` for this engine.
+
+``max_connections`` :
+  `Pool limit configuration`_, overwrites value ``pool_connections`` from
+  :ref:`settings outgoing` for this engine.
+
+``keepalive_expiry`` :
+  `Pool limit configuration`_, overwrites value ``keepalive_expiry`` from
+  :ref:`settings outgoing` for this engine.
+
+
+.. _private engines:
+
+Private Engines (``tokens``)
+============================
+
+Administrators might find themselves wanting to limit access to some of the
+enabled engines on their instances.  It might be because they do not want to
+expose some private information through :ref:`offline engines`.  Or they would
+rather share engines only with their trusted friends or colleagues.
+
+.. sidebar:: info
+
+   Initial sponsored by `Search and Discovery Fund
+   <https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
+
+To solve this issue the concept of *private engines* exists.
+
+A new option was added to engines named `tokens`.  It expects a list of strings.
+If the user making a request presents one of the tokens of an engine, they can
+access information about the engine and make search requests.
+
+Example configuration to restrict access to the Arch Linux Wiki engine:
+
+.. code:: yaml
+
+  - name: arch linux wiki
+    engine: archlinux
+    shortcut: al
+    tokens: [ 'my-secret-token' ]
+
+Unless a user has configured the right token, the engine is going to be hidden
+from them.  It is not going to be included in the list of engines on the
+Preferences page and in the output of `/config` REST API call.
+
+Tokens can be added to one's configuration on the Preferences page under "Engine
+tokens".  The input expects a comma separated list of strings.
+
+The distribution of the tokens from the administrator to the users is not carved
+in stone.  As providing access to such engines implies that the admin knows and
+trusts the user, we do not see necessary to come up with a strict process.
+Instead, we would like to add guidelines to the documentation of the feature.
+
+
+Example: Multilingual Search
+============================
+
+SearXNG does not support true multilingual search.  You have to use the language
+prefix in your search query when searching in a different language.
+
+But there is a workaround: By adding a new search engine with a different
+language, SearXNG will search in your default and other language.
+
+Example configuration in settings.yml for a German and English speaker:
+
+.. code-block:: yaml
+
+    search:
+        default_lang : "de"
+        ...
+
+    engines:
+      - name : google english
+        engine : google
+        language : en
+        ...
+
+When searching, the default google engine will return German results and
+"google english" will return English results.
+

docs/admin/settings/settings_general.rst

@@ -0,0 +1,34 @@
+.. _settings general:
+
+============
+``general:``
+============
+
+.. code:: yaml
+
+   general:
+     debug: false
+     instance_name:  "SearXNG"
+     privacypolicy_url: false
+     donation_url: false
+     contact_url: false
+     enable_metrics: true
+
+``debug`` : ``$SEARXNG_DEBUG``
+  Allow a more detailed log if you run SearXNG directly. Display *detailed* error
+  messages in the browser too, so this must be deactivated in production.
+
+``donation_url`` :
+  Set value to ``true`` to use your own donation page written in the
+  :ref:`searx/info/en/donate.md <searx.infopage>` and use ``false`` to disable
+  the donation link altogether.
+
+``privacypolicy_url``:
+  Link to privacy policy.
+
+``contact_url``:
+  Contact ``mailto:`` address or WEB form.
+
+``enable_metrics``:
+  Enabled by default. Record various anonymous metrics available at ``/stats``,
+  ``/stats/errors`` and ``/preferences``.

docs/admin/settings/settings_outgoing.rst

@@ -0,0 +1,110 @@
+.. _settings outgoing:
+
+=============
+``outgoing:``
+=============
+
+Communication with search engines.
+
+.. code:: yaml
+
+   outgoing:
+     request_timeout: 2.0       # default timeout in seconds, can be override by engine
+     max_request_timeout: 10.0  # the maximum timeout in seconds
+     useragent_suffix: ""       # information like an email address to the administrator
+     pool_connections: 100      # Maximum number of allowable connections, or null
+                                # for no limits. The default is 100.
+     pool_maxsize: 10           # Number of allowable keep-alive connections, or null
+                                # to always allow. The default is 10.
+     enable_http2: true         # See https://www.python-httpx.org/http2/
+     # uncomment below section if you want to use a custom server certificate
+     # see https://www.python-httpx.org/advanced/#changing-the-verification-defaults
+     # and https://www.python-httpx.org/compatibility/#ssl-configuration
+     #  verify: ~/.mitmproxy/mitmproxy-ca-cert.cer
+     #
+     # uncomment below section if you want to use a proxyq see: SOCKS proxies
+     #   https://2.python-requests.org/en/latest/user/advanced/#proxies
+     # are also supported: see
+     #   https://2.python-requests.org/en/latest/user/advanced/#socks
+     #
+     #  proxies:
+     #    all://:
+     #      - http://proxy1:8080
+     #      - http://proxy2:8080
+     #
+     #  using_tor_proxy: true
+     #
+     # Extra seconds to add in order to account for the time taken by the proxy
+     #
+     #  extra_proxy_timeout: 10.0
+     #
+
+``request_timeout`` :
+  Global timeout of the requests made to others engines in seconds.  A bigger
+  timeout will allow to wait for answers from slow engines, but in consequence
+  will slow SearXNG reactivity (the result page may take the time specified in the
+  timeout to load).  Can be override by ``timeout`` in the :ref:`settings engine`.
+
+``useragent_suffix`` :
+  Suffix to the user-agent SearXNG uses to send requests to others engines.  If an
+  engine wish to block you, a contact info here may be useful to avoid that.
+
+.. _Pool limit configuration: https://www.python-httpx.org/advanced/#pool-limit-configuration
+
+``pool_maxsize``:
+  Number of allowable keep-alive connections, or ``null`` to always allow.  The
+  default is 10.  See ``max_keepalive_connections`` `Pool limit configuration`_.
+
+``pool_connections`` :
+  Maximum number of allowable connections, or ``null`` # for no limits.  The
+  default is 100.  See ``max_connections`` `Pool limit configuration`_.
+
+``keepalive_expiry`` :
+  Number of seconds to keep a connection in the pool.  By default 5.0 seconds.
+  See ``keepalive_expiry`` `Pool limit configuration`_.
+
+.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying
+
+``proxies`` :
+  Define one or more proxies you wish to use, see `httpx proxies`_.
+  If there are more than one proxy for one protocol (http, https),
+  requests to the engines are distributed in a round-robin fashion.
+
+``source_ips`` :
+  If you use multiple network interfaces, define from which IP the requests must
+  be made. Example:
+
+  * ``0.0.0.0`` any local IPv4 address.
+  * ``::`` any local IPv6 address.
+  * ``192.168.0.1``
+  * ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses
+  * ``fe80::60a2:1691:e5a2:ee1f``
+  * ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network.
+  * ``[ 192.168.0.1, fe80::/126 ]``
+
+``retries`` :
+  Number of retry in case of an HTTP error.  On each retry, SearXNG uses an
+  different proxy and source ip.
+
+``enable_http2`` :
+  Enable by default. Set to ``false`` to disable HTTP/2.
+
+.. _httpx verification defaults: https://www.python-httpx.org/advanced/#changing-the-verification-defaults
+.. _httpx ssl configuration: https://www.python-httpx.org/compatibility/#ssl-configuration
+
+``verify``: : ``$SSL_CERT_FILE``, ``$SSL_CERT_DIR``
+  Allow to specify a path to certificate.
+  see `httpx verification defaults`_.
+
+  In addition to ``verify``, SearXNG supports the ``$SSL_CERT_FILE`` (for a file) and
+  ``$SSL_CERT_DIR`` (for a directory) OpenSSL variables.
+  see `httpx ssl configuration`_.
+
+``max_redirects`` :
+  30 by default. Maximum redirect before it is an error.
+
+``using_tor_proxy`` :
+  Using tor proxy (``true``) or not (``false``) for all engines.  The default is
+  ``false`` and can be overwritten in the :ref:`settings engine`
+
+

docs/admin/settings/settings_redis.rst

@@ -0,0 +1,49 @@
+.. _settings redis:
+
+==========
+``redis:``
+==========
+
+.. _Redis.from_url(url): https://redis-py.readthedocs.io/en/stable/connections.html#redis.client.Redis.from_url
+
+A redis DB can be connected by an URL, in :py:obj:`searx.redisdb` you
+will find a description to test your redis connection in SearXNG.  When using
+sockets, don't forget to check the access rights on the socket::
+
+  ls -la /usr/local/searxng-redis/run/redis.sock
+  srwxrwx--- 1 searxng-redis searxng-redis ... /usr/local/searxng-redis/run/redis.sock
+
+In this example read/write access is given to the *searxng-redis* group.  To get
+access rights to redis instance (the socket), your SearXNG (or even your
+developer) account needs to be added to the *searxng-redis* group.
+
+``url`` : ``$SEARXNG_REDIS_URL``
+  URL to connect redis database, see `Redis.from_url(url)`_ & :ref:`redis db`::
+
+    redis://[[username]:[password]]@localhost:6379/0
+    rediss://[[username]:[password]]@localhost:6379/0
+    unix://[[username]:[password]]@/path/to/socket.sock?db=0
+
+.. _Redis Developer Notes:
+
+Redis Developer Notes
+=====================
+
+To set up a local redis instance, first set the socket path of the Redis DB
+in your YAML setting:
+
+.. code:: yaml
+
+   redis:
+     url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
+
+Then use the following commands to install the redis instance (:ref:`manage
+redis.help`):
+
+.. code:: sh
+
+   $ ./manage redis.build
+   $ sudo -H ./manage redis.install
+   $ sudo -H ./manage redis.addgrp "${USER}"
+   # don't forget to logout & login to get member of group
+

docs/admin/settings/settings_search.rst

@@ -0,0 +1,104 @@
+.. _settings search:
+
+===========
+``search:``
+===========
+
+.. code:: yaml
+
+   search:
+     safe_search: 0
+     autocomplete: ""
+     favicon_resolver: ""
+     default_lang: ""
+     ban_time_on_fail: 5
+     max_ban_time_on_fail: 120
+     suspended_times:
+       SearxEngineAccessDenied: 86400
+       SearxEngineCaptcha: 86400
+       SearxEngineTooManyRequests: 3600
+       cf_SearxEngineCaptcha: 1296000
+       cf_SearxEngineAccessDenied: 86400
+       recaptcha_SearxEngineCaptcha: 604800
+     formats:
+       - html
+
+``safe_search``:
+  Filter results.
+
+  - ``0``: None
+  - ``1``: Moderate
+  - ``2``: Strict
+
+``autocomplete``:
+  Existing autocomplete backends, leave blank to turn it off.
+
+  - ``dbpedia``
+  - ``duckduckgo``
+  - ``google``
+  - ``mwmbl``
+  - ``startpage``
+  - ``swisscows``
+  - ``qwant``
+  - ``wikipedia``
+
+``favicon_resolver``:
+  To activate favicons in SearXNG's result list select a default
+  favicon-resolver, leave blank to turn off the feature.  Don't activate the
+  favicons before reading the :ref:`Favicons documentation <favicons>`.
+
+``default_lang``:
+  Default search language - leave blank to detect from browser information or
+  use codes from :origin:`searx/languages.py`.
+
+``languages``:
+  List of available languages - leave unset to use all codes from
+  :origin:`searx/languages.py`.  Otherwise list codes of available languages.
+  The ``all`` value is shown as the ``Default language`` in the user interface
+  (in most cases, it is meant to send the query without a language parameter ;
+  in some cases, it means the English language) Example:
+
+  .. code:: yaml
+
+     languages:
+       - all
+       - en
+       - en-US
+       - de
+       - it-IT
+       - fr
+       - fr-BE
+
+``ban_time_on_fail``:
+  Ban time in seconds after engine errors.
+
+``max_ban_time_on_fail``:
+  Max ban time in seconds after engine errors.
+
+``suspended_times``:
+  Engine suspension time after error (in seconds; set to 0 to disable)
+
+  ``SearxEngineAccessDenied``: 86400
+    For error "Access denied" and "HTTP error [402, 403]"
+
+  ``SearxEngineCaptcha``: 86400
+    For error "CAPTCHA"
+
+  ``SearxEngineTooManyRequests``: 3600
+    For error "Too many request" and "HTTP error 429"
+
+  Cloudflare CAPTCHA:
+     - ``cf_SearxEngineCaptcha``: 1296000
+     - ``cf_SearxEngineAccessDenied``: 86400
+
+  Google CAPTCHA:
+    - ``recaptcha_SearxEngineCaptcha``: 604800
+
+``formats``:
+  Result formats available from web, remove format to deny access (use lower
+  case).
+
+  - ``html``
+  - ``csv``
+  - ``json``
+  - ``rss``

docs/admin/settings/settings_server.rst

@@ -0,0 +1,57 @@
+.. _settings server:
+
+===========
+``server:``
+===========
+
+.. code:: yaml
+
+   server:
+       base_url: http://example.org/location  # change this!
+       port: 8888
+       bind_address: "127.0.0.1"
+       secret_key: "ultrasecretkey"           # change this!
+       limiter: false
+       public_instance: false
+       image_proxy: false
+       default_http_headers:
+         X-Content-Type-Options : nosniff
+         X-Download-Options : noopen
+         X-Robots-Tag : noindex, nofollow
+         Referrer-Policy : no-referrer
+
+``base_url`` : ``$SEARXNG_URL``
+  The base URL where SearXNG is deployed.  Used to create correct inbound links.
+
+``port`` & ``bind_address``: ``$SEARXNG_PORT`` & ``$SEARXNG_BIND_ADDRESS``
+  Port number and *bind address* of the SearXNG web application if you run it
+  directly using ``python searx/webapp.py``.  Doesn't apply to a SearXNG
+  services running behind a proxy and using socket communications.
+
+``secret_key`` : ``$SEARXNG_SECRET``
+  Used for cryptography purpose.
+
+``limiter`` :  ``$SEARXNG_LIMITER``
+  Rate limit the number of request on the instance, block some bots.  The
+  :ref:`limiter` requires a :ref:`settings redis` database.
+
+.. _public_instance:
+
+``public_instance`` :  ``$SEARXNG_PUBLIC_INSTANCE``
+
+  Setting that allows to enable features specifically for public instances (not
+  needed for local usage).  By set to ``true`` the following features are
+  activated:
+
+  - :py:obj:`searx.botdetection.link_token` in the :ref:`limiter`
+
+.. _image_proxy:
+
+``image_proxy`` : ``$SEARXNG_IMAGE_PROXY``
+  Allow your instance of SearXNG of being able to proxy images.  Uses memory space.
+
+.. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
+
+``default_http_headers`` :
+  Set additional HTTP headers, see `#755 <https://github.com/searx/searx/issues/715>`__
+

docs/admin/settings/settings_ui.rst

@@ -0,0 +1,70 @@
+.. _settings ui:
+
+=======
+``ui:``
+=======
+
+.. _cache busting:
+   https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#caching_static_assets_with_cache_busting
+
+.. code:: yaml
+
+   ui:
+     static_use_hash: false
+     default_locale: ""
+     query_in_title: false
+     infinite_scroll: false
+     center_alignment: false
+     cache_url: https://web.archive.org/web/
+     default_theme: simple
+     theme_args:
+       simple_style: auto
+     search_on_category_select: true
+     hotkeys: default
+
+.. _static_use_hash:
+
+``static_use_hash`` : ``$SEARXNG_STATIC_USE_HASH``
+  Enables `cache busting`_ of static files.
+
+``default_locale`` :
+  SearXNG interface language.  If blank, the locale is detected by using the
+  browser language.  If it doesn't work, or you are deploying a language
+  specific instance of searx, a locale can be defined using an ISO language
+  code, like ``fr``, ``en``, ``de``.
+
+``query_in_title`` :
+  When true, the result page's titles contains the query it decreases the
+  privacy, since the browser can records the page titles.
+
+``infinite_scroll``:
+  When true, automatically loads the next page when scrolling to bottom of the current page.
+
+``center_alignment`` : default ``false``
+  When enabled, the results are centered instead of being in the left (or RTL)
+  side of the screen.  This setting only affects the *desktop layout*
+  (:origin:`min-width: @tablet <searx/static/themes/simple/src/less/definitions.less>`)
+
+.. cache_url:
+
+``cache_url`` : ``https://web.archive.org/web/``
+  URL prefix of the internet archive or cache, don't forget trailing slash (if
+  needed).  The default is https://web.archive.org/web/ alternatives are:
+
+  - https://webcache.googleusercontent.com/search?q=cache:
+  - https://archive.today/
+
+``default_theme`` :
+  Name of the theme you want to use by default on your SearXNG instance.
+
+``theme_args.simple_style``:
+  Style of simple theme: ``auto``, ``light``, ``dark``, ``black``
+
+``results_on_new_tab``:
+  Open result links in a new tab by default.
+
+``search_on_category_select``:
+  Perform search immediately if a category selected. Disable to select multiple categories.
+
+``hotkeys``:
+  Hotkeys to use in the search interface: ``default``, ``vim`` (Vim-like).

docs/admin/update-searxng.rst

@@ -9,7 +9,7 @@ SearXNG maintenance
    - :ref:`toolboxing`
    - :ref:`uWSGI maintenance`
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry
@@ -61,7 +61,7 @@ and then, to name just a few:
 - The image proxy morty is no longer needed, it has been replaced by the
   :ref:`image proxy <image_proxy>` from SearXNG.
 
-- To save bandwith :ref:`cache busting <static_use_hash>` has been implemented.
+- To save bandwidth :ref:`cache busting <static_use_hash>` has been implemented.
   To get in use, the ``static-expires`` needs to be set in the :ref:`uwsgi
   setup`.
 

docs/build-templates/filtron.rst

@@ -1,53 +0,0 @@
-.. START create user
-
-.. tabs::
-
-  .. group-tab:: bash
-
-    .. code-block:: sh
-
-      $ sudo -H useradd --shell /bin/bash --system \\
-          --home-dir \"$SERVICE_HOME\" \\
-          --comment \"Privacy-respecting metasearch engine\" $SERVICE_USER
-
-      $ sudo -H mkdir \"$SERVICE_HOME\"
-      $ sudo -H chown -R \"$SERVICE_GROUP:$SERVICE_GROUP\" \"$SERVICE_HOME\"
-
-.. END create user
-
-.. START install go
-
-.. tabs::
-
-  .. group-tab:: os: linux / arch: amd64
-
-
-    .. code-block:: bash
-
-       $ cat > \"$GO_ENV\" <<EOF
-       export GOPATH=${SERVICE_HOME}/go-apps
-       export PATH=\$PATH:${SERVICE_HOME}/local/go/bin:\$GOPATH/bin
-       EOF
-       $ sudo -i -u \"${SERVICE_USER}\"
-       (${SERVICE_USER}) $ echo 'source $GO_ENV' >> ~/.profile
-       (${SERVICE_USER}) $ mkdir ${SERVICE_HOME}/local
-       (${SERVICE_USER}) $ wget --progress=bar -O \"${GO_VERSION}.linux-amd64.tar.gz\" \\
-                   \"${GO_DL_URL}/${GO_VERSION}.linux-amd64.tar.gz\"
-       (${SERVICE_USER}) $ tar -C ${SERVICE_HOME}/local -xzf \"${GO_VERSION}.linux-amd64.tar.gz\"
-       (${SERVICE_USER}) $ which go
-       ${SERVICE_HOME}/local/go/bin/go
-
-.. END install go
-
-.. START install filtron
-
-.. tabs::
-
-  .. group-tab:: bash
-
-    .. code-block:: bash
-
-       $ sudo -i -u \"${SERVICE_USER}\"
-       (${SERVICE_USER}) $ go get -v -u github.com/searxng/filtron
-
-.. END install filtron

docs/build-templates/morty.rst

@@ -1,53 +0,0 @@
-.. START create user
-
-.. tabs::
-
-  .. group-tab:: bash
-
-    .. code-block:: sh
-
-      $ sudo -H useradd --shell /bin/bash --system \\
-          --home-dir \"$SERVICE_HOME\" \\
-          --comment \"Privacy-respecting metasearch engine\" $SERVICE_USER
-
-      $ sudo -H mkdir \"$SERVICE_HOME\"
-      $ sudo -H chown -R \"$SERVICE_GROUP:$SERVICE_GROUP\" \"$SERVICE_HOME\"
-
-.. END create user
-
-.. START install go
-
-.. tabs::
-
-  .. group-tab:: os: linux / arch: amd64
-
-
-    .. code-block:: bash
-
-       $ cat > \"$GO_ENV\" <<EOF
-       export GOPATH=${SERVICE_HOME}/go-apps
-       export PATH=\$PATH:${SERVICE_HOME}/local/go/bin:\$GOPATH/bin
-       EOF
-       $ sudo -i -u \"${SERVICE_USER}\"
-       (${SERVICE_USER}) $ echo 'source $GO_ENV' >> ~/.profile
-       (${SERVICE_USER}) $ mkdir ${SERVICE_HOME}/local
-       (${SERVICE_USER}) $ wget --progress=bar -O \"${GO_VERSION}.linux-amd64.tar.gz\" \\
-                   \"${GO_DL_URL}/${GO_VERSION}.linux-amd64.tar.gz\"
-       (${SERVICE_USER}) $ tar -C ${SERVICE_HOME}/local -xzf \"${GO_VERSION}.linux-amd64.tar.gz\"
-       (${SERVICE_USER}) $ which go
-       ${SERVICE_HOME}/local/go/bin/go
-
-.. END install go
-
-.. START install morty
-
-.. tabs::
-
-  .. group-tab:: bash
-
-    .. code-block:: bash
-
-       $ sudo -i -u \"${SERVICE_USER}\"
-       (${SERVICE_USER}) $ go get -v -u github.com/asciimoo/morty
-
-.. END install morty

docs/build-templates/searxng.rst

@@ -113,7 +113,7 @@ ${fedora_build}
 
        (${SERVICE_USER})$ command -v python && python --version
        $SEARXNG_PYENV/bin/python
-       Python 3.8.1
+       Python 3.11.10
 
        # update pip's boilerplate ..
        pip install -U pip
@@ -123,7 +123,7 @@ ${fedora_build}
 
        # jump to SearXNG's working tree and install SearXNG into virtualenv
        (${SERVICE_USER})$ cd \"$SEARXNG_SRC\"
-       (${SERVICE_USER})$ pip install -e .
+       (${SERVICE_USER})$ pip install --use-pep517 --no-build-isolation -e .
 
 
 .. END manage.sh update_packages

docs/conf.py

@@ -1,7 +1,7 @@
-# -*- coding: utf-8 -*-
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import  sys, os
+from pathlib import Path
 from pallets_sphinx_themes import ProjectLink
 
 from searx import get_setting
@@ -10,10 +10,9 @@ from searx.version import VERSION_STRING, GIT_URL, GIT_BRANCH
 # Project --------------------------------------------------------------
 
 project = 'SearXNG'
-copyright = '2021 SearXNG team, 2015-2021 Adam Tauber, Noémi Ványi'
-author = '2021 SearXNG team, 2015-2021 Adam Tauber'
+copyright = 'SearXNG team'
+author = 'SearXNG team'
 release, version = VERSION_STRING, VERSION_STRING
-
 SEARXNG_URL = get_setting('server.base_url') or 'https://example.org/searxng'
 ISSUE_URL = get_setting('brand.issue_url')
 DOCS_URL = get_setting('brand.docs_url')
@@ -22,6 +21,9 @@ PRIVACYPOLICY_URL = get_setting('general.privacypolicy_url')
 CONTACT_URL = get_setting('general.contact_url')
 WIKI_URL = get_setting('brand.wiki_url')
 
+SOURCEDIR = Path(__file__).parent.parent / "searx"
+os.environ['SOURCEDIR'] = str(SOURCEDIR)
+
 # hint: sphinx.ext.viewcode won't highlight when 'highlight_language' [1] is set
 #       to string 'none' [2]
 #
@@ -125,6 +127,7 @@ extensions = [
     "sphinx_tabs.tabs", # https://github.com/djungelorm/sphinx-tabs
     'myst_parser',  # https://www.sphinx-doc.org/en/master/usage/markdown.html
     'notfound.extension',  # https://github.com/readthedocs/sphinx-notfound-page
+    'sphinxcontrib.autodoc_pydantic',  # https://github.com/mansenfranzen/autodoc_pydantic
 ]
 
 autodoc_default_options = {
@@ -168,6 +171,7 @@ imgmath_image_format = 'svg'
 imgmath_font_size = 14
 # sphinx.ext.imgmath setup END
 
+html_show_sphinx = False
 html_theme_options = {"index_sidebar_logo": True}
 html_context = {"project_links": [] }
 html_context["project_links"].append(ProjectLink("Source", GIT_URL + '/tree/' + GIT_BRANCH))

docs/dev/contribution_guide.rst

@@ -4,7 +4,7 @@
 How to contribute
 =================
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry
@@ -148,7 +148,7 @@ live build
 Live build is like WYSIWYG.  If you want to edit the documentation, its
 recommended to use.  The Makefile target ``docs.live`` builds the docs, opens
 URL in your favorite browser and rebuilds every time a reST file has been
-changed.
+changed (:ref:`make docs.clean`).
 
 .. code:: sh
 
@@ -183,3 +183,8 @@ commit and push:
 .. code:: sh
 
    $ make docs.clean docs.gh-pages
+
+.. attention::
+
+   If you are working in your own brand, don't forget to adjust your
+   :ref:`settings brand`.

docs/dev/engines/demo/demo_offline.rst

@@ -4,6 +4,11 @@
 Demo Offline Engine
 ===================
 
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
 .. automodule:: searx.engines.demo_offline
   :members:
 

docs/dev/engines/demo/demo_online.rst

@@ -4,6 +4,11 @@
 Demo Online Engine
 ==================
 
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
 .. automodule:: searx.engines.demo_online
   :members:
 

docs/dev/engines/engine_overview.rst

@@ -4,6 +4,11 @@
 Engine Overview
 ===============
 
+.. contents::
+   :depth: 3
+   :local:
+   :backlinks: entry
+
 .. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
 
 .. sidebar:: Further reading ..
@@ -11,10 +16,6 @@ Engine Overview
    - :ref:`configured engines`
    - :ref:`settings engine`
 
-.. contents::
-   :depth: 3
-   :backlinks: entry
-
 SearXNG is a metasearch-engine_, so it uses different search engines to provide
 better results.
 
@@ -49,12 +50,12 @@ Engine File
    categories              list        categories, in which the engine is working
    paging                  boolean     support multiple pages
    time_range_support      boolean     support search time range
-   engine_type             str         - ``online`` :ref:`[ref] <demo online engine>` by
+   engine_type             str         - ``online`` :ref:`[ref] <online engines>` by
                                          default, other possibles values are:
                                        - ``offline`` :ref:`[ref] <offline engines>`
-                                       - ``online_dictionary``
-                                       - ``online_currency``
-                                       - ``online_url_search``
+                                       - ``online_dictionary`` :ref:`[ref] <online dictionary>`
+                                       - ``online_currency`` :ref:`[ref] <online currency>`
+                                       - ``online_url_search`` :ref:`[ref] <online url search>`
    ======================= =========== ========================================================
 
 .. _engine settings:
@@ -86,8 +87,8 @@ For a more  detailed description, see :ref:`settings engine` in the :ref:`settin
 Overrides
 ---------
 
-A few of the options have default values in the namespace of engine's python
-modul, but are often overwritten by the settings.  If ``None`` is assigned to an
+A few of the options have default values in the namespace of the engine's python
+module, but are often overwritten by the settings.  If ``None`` is assigned to an
 option in the engine file, it has to be redefined in the settings, otherwise
 SearXNG will not start with that engine (global names with a leading underline can
 be ``None``).
@@ -239,12 +240,18 @@ following parameters can be used to specify a search request:
 .. _engine results:
 .. _engine media types:
 
-Media Types
-===========
+Result Types (``template``)
+===========================
 
 Each result item of an engine can be of different media-types.  Currently the
-following media-types are supported.  To set another media-type as ``default``,
-the parameter ``template`` must be set to the desired type.
+following media-types are supported.  To set another media-type as
+:ref:`template default`, the parameter ``template`` must be set to the desired
+type.
+
+.. _template default:
+
+``default``
+-----------
 
 .. table::  Parameter of the **default** media type:
    :width: 100%
@@ -259,23 +266,65 @@ the parameter ``template`` must be set to the desired type.
    ========================= =====================================================
 
 
-.. table::  Parameter of the **images** media type:
+.. _template images:
+
+``images``
+----------
+
+.. list-table:: Parameter of the **images** media type
+   :header-rows: 2
    :width: 100%
 
-   ========================= =====================================================
-   result-parameter          information
-   ------------------------- -----------------------------------------------------
-   template                  is set to ``images.html``
-   ========================= =====================================================
-   url                       string, url to the result site
-   title                     string, title of the result *(partly implemented)*
-   content                   *(partly implemented)*
-   publishedDate             :py:class:`datetime.datetime`,
-                             time of publish *(partly implemented)*
-   img\_src                  string, url to the result image
-   thumbnail\_src            string, url to a small-preview image
-   ========================= =====================================================
+   * - result-parameter
+     - Python type
+     - information
 
+   * - template
+     - :py:class:`str`
+     - is set to ``images.html``
+
+   * - url
+     - :py:class:`str`
+     - url to the result site
+
+   * - title
+     - :py:class:`str`
+     - title of the result
+
+   * - content
+     - :py:class:`str`
+     - description of the image
+
+   * - publishedDate
+     - :py:class:`datetime <datetime.datetime>`
+     - time of publish
+
+   * - img_src
+     - :py:class:`str`
+     - url to the result image
+
+   * - thumbnail_src
+     - :py:class:`str`
+     - url to a small-preview image
+
+   * - resolution
+     - :py:class:`str`
+     - the resolution of the image (e.g. ``1920 x 1080`` pixel)
+
+   * - img_format
+     - :py:class:`str`
+     - the format of the image (e.g. ``png``)
+
+   * - filesize
+     - :py:class:`str`
+     - size of bytes in :py:obj:`human readable <searx.humanize_bytes>` notation
+       (e.g. ``MB`` for 1024 \* 1024 Bytes filesize).
+
+
+.. _template videos:
+
+``videos``
+----------
 
 .. table::  Parameter of the **videos** media type:
    :width: 100%
@@ -290,8 +339,16 @@ the parameter ``template`` must be set to the desired type.
    content                   *(not implemented yet)*
    publishedDate             :py:class:`datetime.datetime`, time of publish
    thumbnail                 string, url to a small-preview image
+   length                    :py:class:`datetime.timedelta`, duration of result
+   views                     string, view count in humanized number format
    ========================= =====================================================
 
+
+.. _template torrent:
+
+``torrent``
+-----------
+
 .. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
 
 .. table::  Parameter of the **torrent** media type:
@@ -315,6 +372,12 @@ the parameter ``template`` must be set to the desired type.
    torrentfile               string, torrentfile of the result
    ========================= =====================================================
 
+
+.. _template map:
+
+``map``
+-------
+
 .. table::  Parameter of the **map** media type:
    :width: 100%
 
@@ -342,6 +405,12 @@ the parameter ``template`` must be set to the desired type.
    address.country           country of object
    ========================= =====================================================
 
+
+.. _template paper:
+
+``paper``
+---------
+
 .. _BibTeX format: https://www.bibtex.com/g/bibtex-format/
 .. _BibTeX field types: https://en.wikipedia.org/wiki/BibTeX#Field_types
 
@@ -430,3 +499,73 @@ the parameter ``template`` must be set to the desired type.
    * - html_url
      - :py:class:`str`
      - URL to full article, HTML version
+
+
+.. _template packages:
+
+``packages``
+------------
+
+.. list-table:: Parameter of the **packages** media type
+   :header-rows: 2
+   :width: 100%
+
+   * - result-parameter
+     - Python type
+     - information
+
+   * - template
+     - :py:class:`str`
+     - is set to ``packages.html``
+
+   * - title
+     - :py:class:`str`
+     - title of the result
+
+   * - content
+     - :py:class:`str`
+     - abstract
+
+   * - package_name
+     - :py:class:`str`
+     - the name of the package
+
+   * - version
+     - :py:class:`str`
+     - the current version of the package
+
+   * - maintainer
+     - :py:class:`str`
+     - the maintainer or author of the project
+
+   * - publishedDate
+     - :py:class:`datetime <datetime.datetime>`
+     - date of latest update or release
+
+   * - tags
+     - :py:class:`List <list>`\ [\ :py:class:`str`\ ]
+     - free tag list
+
+   * - popularity
+     - :py:class:`str`
+     - the popularity of the package, e.g. rating or download count
+
+   * - license_name
+     - :py:class:`str`
+     - the name of the license
+
+   * - license_url
+     - :py:class:`str`
+     - the web location of a license copy
+
+   * - homepage
+     - :py:class:`str`
+     - the url of the project's homepage
+
+   * - source_code_url
+     - :py:class:`str`
+     - the location of the project's source code
+
+   * - links
+     - :py:class:`dict`
+     - additional links in the form of ``{'link_name': 'http://example.com'}``

docs/dev/engines/enginelib.rst

@@ -1,15 +1,20 @@
 .. _searx.enginelib:
 
-============
-Engine model
-============
+==============
+Engine Library
+==============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
 
 .. automodule:: searx.enginelib
   :members:
 
 .. _searx.enginelib.traits:
 
-=============
+
 Engine traits
 =============
 

docs/dev/engines/engines.rst

@@ -0,0 +1,9 @@
+.. _searx.engines loader:
+
+========================
+SearXNG's engines loader
+========================
+
+.. automodule:: searx.engines
+  :members:
+

docs/dev/engines/index.rst

@@ -0,0 +1,107 @@
+.. _engine implementations:
+
+======================
+Engine Implementations
+======================
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+
+.. toctree::
+   :caption: Framework Components
+   :maxdepth: 2
+
+   enginelib
+   engines
+   engine_overview
+
+
+Engine Types
+============
+
+The :py:obj:`engine_type <searx.enginelib.Engine.engine_type>` of an engine
+determines which :ref:`search processor <searx.search.processors>` is used by
+the engine.
+
+In this section a list of the engines that are documented is given, a complete
+list of the engines can be found in the source under: :origin:`searx/engines`.
+
+.. _online engines:
+
+Online Engines
+--------------
+
+.. sidebar:: info
+
+   - :py:obj:`processors.online <searx.search.processors.online>`
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   demo/demo_online
+   xpath
+   mediawiki
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   online/*
+
+.. _offline engines:
+
+Offline Engines
+---------------
+
+.. sidebar:: info
+
+   - :py:obj:`processors.offline <searx.search.processors.offline>`
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   offline_concept
+   demo/demo_offline
+   offline/*
+
+.. _online url search:
+
+Online URL Search
+-----------------
+
+.. sidebar:: info
+
+   - :py:obj:`processors.online_url_search <searx.search.processors.online_url_search>`
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   online_url_search/*
+
+.. _online currency:
+
+Online Currency
+---------------
+
+.. sidebar:: info
+
+   - :py:obj:`processors.online_currency <searx.search.processors.online_currency>`
+
+*no engine of this type is documented yet / comming soon*
+
+.. _online dictionary:
+
+Online Dictionary
+-----------------
+
+.. sidebar:: info
+
+   - :py:obj:`processors.online_dictionary <searx.search.processors.online_dictionary>`
+
+*no engine of this type is documented yet / comming soon*

docs/dev/engines/mediawiki.rst

@@ -0,0 +1,13 @@
+.. _mediawiki engine:
+
+================
+MediaWiki Engine
+================
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.mediawiki
+  :members:

docs/dev/engines/offline/command-line-engines.rst

@@ -0,0 +1,23 @@
+.. _engine command:
+
+====================
+Command Line Engines
+====================
+
+.. sidebar:: info
+
+   - :origin:`command.py <searx/engines/command.py>`
+   - :ref:`offline engines`
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. sidebar:: info
+
+   Initial sponsored by `Search and Discovery Fund
+   <https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
+
+.. automodule:: searx.engines.command
+  :members:

docs/dev/engines/offline/nosql-engines.rst

@@ -1,3 +1,5 @@
+.. _nosql engines:
+
 ===============
 NoSQL databases
 ===============
@@ -8,6 +10,16 @@ NoSQL databases
    - `redis.io <https://redis.io/>`_
    - `MongoDB <https://www.mongodb.com>`_
 
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. sidebar:: info
+
+   Initial sponsored by `Search and Discovery Fund
+   <https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
+
 The following `NoSQL databases`_ are supported:
 
 - :ref:`engine redis_server`
@@ -30,15 +42,8 @@ can still add them and limit the access by setting ``tokens`` as described in
 section :ref:`private engines`.
 
 
-Configure the engines
-=====================
-
-`NoSQL databases`_ are used for storing arbitrary data without first defining
-their structure.
-
-
 Extra Dependencies
-------------------
+==================
 
 For using :ref:`engine redis_server` or :ref:`engine mongodb` you need to
 install additional packages in Python's Virtual Environment of your SearXNG
@@ -49,6 +54,13 @@ instance.  To switch into the environment (:ref:`searxng-src`) you can use
   (searxng-pyenv)$ pip install ...
 
 
+Configure the engines
+=====================
+
+`NoSQL databases`_ are used for storing arbitrary data without first defining
+their structure.
+
+
 .. _engine redis_server:
 
 Redis Server
@@ -62,29 +74,9 @@ Redis Server
    - redis.io_
    - :origin:`redis_server.py <searx/engines/redis_server.py>`
 
+.. automodule:: searx.engines.redis_server
+  :members:
 
-Redis is an open source (BSD licensed), in-memory data structure (key value
-based) store.  Before configuring the ``redis_server`` engine, you must install
-the dependency redis_.
-
-Select a database to search in and set its index in the option ``db``.  You can
-either look for exact matches or use partial keywords to find what you are
-looking for by configuring ``exact_match_only``.  You find an example
-configuration below:
-
-.. code:: yaml
-
-  # Required dependency: redis
-
-  - name: myredis
-    shortcut : rds
-    engine: redis_server
-    exact_match_only: false
-    host: '127.0.0.1'
-    port: 6379
-    enable_http: true
-    password: ''
-    db: 0
 
 .. _engine mongodb:
 
@@ -99,37 +91,7 @@ MongoDB
    - MongoDB_
    - :origin:`mongodb.py <searx/engines/mongodb.py>`
 
-MongoDB_ is a document based database program that handles JSON like data.
-Before configuring the ``mongodb`` engine, you must install the dependency
-redis_.
 
-In order to query MongoDB_, you have to select a ``database`` and a
-``collection``.  Furthermore, you have to select a ``key`` that is going to be
-searched.  MongoDB_ also supports the option ``exact_match_only``, so configure
-it as you wish.  Below is an example configuration for using a MongoDB
-collection:
-
-.. code:: yaml
-
-  # MongoDB engine
-  # Required dependency: pymongo
-
-  - name: mymongo
-    engine: mongodb
-    shortcut: md
-    exact_match_only: false
-    host: '127.0.0.1'
-    port: 27017
-    enable_http: true
-    results_per_page: 20
-    database: 'business'
-    collection: 'reviews'  # name of the db collection
-    key: 'name'            # key in the collection to search for
-
-
-Acknowledgment
-==============
-
-This development was sponsored by `Search and Discovery Fund
-<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
+.. automodule:: searx.engines.mongodb
+  :members:
 

docs/dev/engines/offline/search-indexer-engines.rst

@@ -0,0 +1,62 @@
+=================
+Local Search APIs
+=================
+
+.. sidebar:: further read
+
+   - `Comparison to alternatives
+     <https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html>`_
+
+.. contents::
+   :depth: 1
+   :local:
+   :backlinks: entry
+
+.. sidebar:: info
+
+   Initial sponsored by `Search and Discovery Fund
+   <https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
+
+Administrators might find themselves wanting to integrate locally running search
+engines.  The following ones are supported for now:
+
+* `Elasticsearch`_
+* `Meilisearch`_
+* `Solr`_
+
+Each search engine is powerful, capable of full-text search.  All of the engines
+above are added to ``settings.yml`` just commented out, as you have to
+``base_url`` for all them.
+
+Please note that if you are not using HTTPS to access these engines, you have to
+enable HTTP requests by setting ``enable_http`` to ``True``.
+
+Furthermore, if you do not want to expose these engines on a public instance,
+you can still add them and limit the access by setting ``tokens`` as described
+in section :ref:`private engines`.
+
+.. _engine meilisearch:
+
+MeiliSearch
+===========
+
+.. automodule:: searx.engines.meilisearch
+  :members:
+
+
+.. _engine elasticsearch:
+
+Elasticsearch
+=============
+
+.. automodule:: searx.engines.elasticsearch
+  :members:
+
+.. _engine solr:
+
+Solr
+====
+
+.. automodule:: searx.engines.solr
+  :members:
+

docs/dev/engines/offline/sql-engines.rst

@@ -10,12 +10,22 @@ SQL Engines
    - `PostgreSQL <https://www.postgresql.org>`_
    - `MySQL <https://www.mysql.com>`_
 
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. sidebar:: info
+
+   Initial sponsored by `Search and Discovery Fund
+   <https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
+
 With the *SQL engines* you can bind SQL databases into SearXNG.  The following
 Relational Database Management System (RDBMS) are supported:
 
 - :ref:`engine sqlite`
 - :ref:`engine postgresql`
-- :ref:`engine mysql_server`
+- :ref:`engine mysql_server` & :ref:`engine mariadb_server`
 
 All of the engines above are just commented out in the :origin:`settings.yml
 <searx/settings.yml>`, as you have to set the required attributes for the
@@ -42,6 +52,18 @@ add them and limit the access by setting ``tokens`` as described in section
 :ref:`private engines`.
 
 
+Extra Dependencies
+==================
+
+For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to
+install additional packages in Python's Virtual Environment of your SearXNG
+instance.  To switch into the environment (:ref:`searxng-src`) you can use
+:ref:`searxng.sh`::
+
+  $ sudo utils/searxng.sh instance cmd bash
+  (searxng-pyenv)$ pip install ...
+
+
 Configure the engines
 =====================
 
@@ -64,45 +86,8 @@ SQLite
 
    - :origin:`sqlite.py <searx/engines/sqlite.py>`
 
-.. _MediathekView: https://mediathekview.de/
-
-SQLite is a small, fast and reliable SQL database engine.  It does not require
-any extra dependency.  To demonstrate the power of database engines, here is a
-more complex example which reads from a MediathekView_ (DE) movie database.  For
-this example of the SQlite engine download the database:
-
-- https://liste.mediathekview.de/filmliste-v2.db.bz2
-
-and unpack into ``searx/data/filmliste-v2.db``.  To search the database use e.g
-Query to test: ``!mediathekview concert``
-
-.. code:: yaml
-
-   - name: mediathekview
-     engine: sqlite
-     disabled: False
-     categories: general
-     result_template: default.html
-     database: searx/data/filmliste-v2.db
-     query_str:  >-
-       SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title,
-              COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url,
-              description AS content
-         FROM film
-        WHERE title LIKE :wildcard OR description LIKE :wildcard
-        ORDER BY duration DESC
-
-
-Extra Dependencies
-------------------
-
-For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to
-install additional packages in Python's Virtual Environment of your SearXNG
-instance.  To switch into the environment (:ref:`searxng-src`) you can use
-:ref:`searxng.sh`::
-
-  $ sudo utils/searxng.sh instance cmd bash
-  (searxng-pyenv)$ pip install ...
+.. automodule:: searx.engines.sqlite
+  :members:
 
 
 .. _engine postgresql:
@@ -115,20 +100,10 @@ PostgreSQL
 .. sidebar:: info
 
    - :origin:`postgresql.py <searx/engines/postgresql.py>`
-   - ``pip install`` psycopg2_
+   - ``pip install`` `psycopg2-binary <psycopg2>`_
 
-PostgreSQL is a powerful and robust open source database.  Before configuring
-the PostgreSQL engine, you must install the dependency ``psychopg2``.  You can
-find an example configuration below:
-
-.. code:: yaml
-
-   - name: my_database
-     engine: postgresql
-     database: my_database
-     username: searxng
-     password: password
-     query_str: 'SELECT * from my_table WHERE my_column = %(query)s'
+.. automodule:: searx.engines.postgresql
+  :members:
 
 .. _engine mysql_server:
 
@@ -140,27 +115,20 @@ MySQL
    - :origin:`mysql_server.py <searx/engines/mysql_server.py>`
    - ``pip install`` :pypi:`mysql-connector-python <mysql-connector-python>`
 
-MySQL is said to be the most popular open source database. Before enabling MySQL
-engine, you must install the package ``mysql-connector-python``.
 
-The authentication plugin is configurable by setting ``auth_plugin`` in the
-attributes.  By default it is set to ``caching_sha2_password``.  This is an
-example configuration for querying a MySQL server:
+.. automodule:: searx.engines.mysql_server
+  :members:
 
-.. code:: yaml
+.. _engine mariadb_server:
 
-   - name: my_database
-     engine: mysql_server
-     database: my_database
-     username: searxng
-     password: password
-     limit: 5
-     query_str: 'SELECT * from my_table WHERE my_column=%(query)s'
+MariaDB
+--------
+
+.. sidebar:: info
+
+   - :origin:`mariadb_server.py <searx/engines/mariadb_server.py>`
+   - ``pip install`` :pypi:`mariadb <mariadb>`
 
 
-Acknowledgment
-==============
-
-This development was sponsored by `Search and Discovery Fund
-<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
-
+.. automodule:: searx.engines.mariadb_server
+  :members:

docs/dev/engines/offline_concept.rst

@@ -1,15 +1,14 @@
-.. _offline engines:
-
 ===============
-Offline Engines
+Offline Concept
 ===============
 
 .. sidebar:: offline engines
 
    - :ref:`demo offline engine`
-   - :ref:`sql engines`
    - :ref:`engine command`
-   - :origin:`Redis <searx/engines/redis_server.py>`
+   - :ref:`sql engines`
+   - :ref:`nosql engines`
+   - :py:obj:`searx.search.processors.offline`
 
 To extend the functionality of SearXNG, offline engines are going to be
 introduced.  An offline engine is an engine which does not need Internet
@@ -31,7 +30,6 @@ Programming Interface
   in advance.
 
 :py:func:`search(query, params) <searx.engines.demo_offline.searc>`
-
   Each offline engine has a function named ``search``.  This function is
   responsible to perform a search and return the results in a presentable
   format. (Where *presentable* means presentable by the selected result
@@ -69,10 +67,3 @@ administrators can set token(s) for each of the :ref:`private engines`.  If a
 query contains a valid token, then SearXNG performs the requested private
 search.  If not, requests from an offline engines return errors.
 
-
-Acknowledgement
-===============
-
-This development was sponsored by `Search and Discovery Fund
-<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_ .
-

docs/dev/engines/online/alpinelinux.rst

@@ -0,0 +1,13 @@
+.. _alpinelinux engine:
+
+=====================
+Alpine Linux Packages
+=====================
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.alpinelinux
+   :members:

docs/dev/engines/online/annas_archive.rst

@@ -0,0 +1,13 @@
+.. _annas_archive engine:
+
+==============
+Anna's Archive
+==============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.annas_archive
+   :members:

docs/dev/engines/online/archlinux.rst

@@ -4,6 +4,11 @@
 Arch Linux
 ==========
 
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
 .. automodule:: searx.engines.archlinux
   :members:
 

docs/dev/engines/online/bing.rst

@@ -4,7 +4,7 @@
 Bing Engines
 ============
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry

docs/dev/engines/online/bpb.rst

@@ -0,0 +1,13 @@
+.. _bpb engine:
+
+===
+Bpb
+===
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.bpb
+  :members:

docs/dev/engines/online/brave.rst

@@ -0,0 +1,13 @@
+.. _brave engine:
+
+=============
+Brave Engines
+=============
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.brave
+  :members:

docs/dev/engines/online/bt4g.rst

@@ -0,0 +1,14 @@
+.. _bt4g engine:
+
+====
+BT4G
+====
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.bt4g
+  :members:
+

docs/dev/engines/online/dailymotion.rst

@@ -4,5 +4,10 @@
 Dailymotion
 ===========
 
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
 .. automodule:: searx.engines.dailymotion
   :members:

docs/dev/engines/online/discourse.rst

@@ -0,0 +1,8 @@
+.. _discourse engine:
+
+================
+Discourse Forums
+================
+
+.. automodule:: searx.engines.discourse
+   :members:

docs/dev/engines/online/duckduckgo.rst

@@ -1,10 +1,10 @@
 .. _duckduckgo engines:
 
-=================
-DukcDukGo engines
-=================
+==================
+DuckDuckGo Engines
+==================
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry
@@ -12,7 +12,7 @@ DukcDukGo engines
 .. automodule:: searx.engines.duckduckgo
    :members:
 
-.. automodule:: searx.engines.duckduckgo_images
+.. automodule:: searx.engines.duckduckgo_extra
    :members:
 
 .. automodule:: searx.engines.duckduckgo_definitions

docs/dev/engines/online/geizhals.rst

@@ -0,0 +1,8 @@
+.. _gitea geizhals:
+
+========
+Geizhals
+========
+
+.. automodule:: searx.engines.geizhals
+  :members:

docs/dev/engines/online/gitea.rst

@@ -0,0 +1,8 @@
+.. _gitea engine:
+
+=====
+Gitea
+=====
+
+.. automodule:: searx.engines.gitea
+  :members:

docs/dev/engines/online/gitlab.rst

@@ -0,0 +1,8 @@
+.. _gitlab engine:
+
+======
+GitLab
+======
+
+.. automodule:: searx.engines.gitlab
+   :members:

docs/dev/engines/online/google.rst

@@ -4,7 +4,7 @@
 Google Engines
 ==============
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry

docs/dev/engines/online/lemmy.rst

@@ -0,0 +1,13 @@
+.. _lemmy engine:
+
+=====
+Lemmy
+=====
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.lemmy
+   :members:

docs/dev/engines/online/loc.rst

@@ -0,0 +1,13 @@
+.. _loc engine:
+
+===================
+Library of Congress
+===================
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.loc
+  :members:

docs/dev/engines/online/mastodon.rst

@@ -0,0 +1,13 @@
+.. _mastodon engine:
+
+========
+Mastodon
+========
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.mastodon
+  :members:

docs/dev/engines/online/moviepilot.rst

@@ -0,0 +1,13 @@
+.. _moviepilot engine:
+
+==========
+Moviepilot
+==========
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.moviepilot
+   :members:

docs/dev/engines/online/mrs.rst

@@ -0,0 +1,13 @@
+.. _mrs engine:
+
+=========================
+Matrix Rooms Search (MRS)
+=========================
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.mrs
+   :members:

docs/dev/engines/online/mullvad_leta.rst

@@ -0,0 +1,13 @@
+.. _voidlinux mullvad_leta:
+
+============
+Mullvad-Leta
+============
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.mullvad_leta
+   :members:

docs/dev/engines/online/mwmbl.rst

@@ -0,0 +1,27 @@
+.. _Mwmbl engine:
+
+============
+Mwmbl Engine
+============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+
+.. _mwmbl web engine:
+
+Mwmbl WEB
+=========
+
+.. automodule:: searx.engines.mwmbl
+  :members:
+
+
+.. _mwmbl autocomplete:
+
+Mwmbl Autocomplete
+==================
+
+.. autofunction:: searx.autocomplete.mwmbl

docs/dev/engines/online/odysee.rst

@@ -0,0 +1,13 @@
+.. _odysee engine:
+
+======
+Odysee
+======
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.odysee
+  :members:

docs/dev/engines/online/peertube.rst

@@ -4,7 +4,7 @@
 Peertube Engines
 ================
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry

docs/dev/engines/online/piped.rst

@@ -0,0 +1,13 @@
+.. _piped engine:
+
+=====
+Piped
+=====
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.piped
+  :members:

docs/dev/engines/online/presearch.rst

@@ -0,0 +1,13 @@
+.. _engine presearch:
+
+================
+Presearch Engine
+================
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.presearch
+   :members:

docs/dev/engines/online/qwant.rst

@@ -0,0 +1,13 @@
+.. _qwant engine:
+
+=====
+Qwant
+=====
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.qwant
+   :members:

docs/dev/engines/online/radio_browser.rst

@@ -0,0 +1,13 @@
+.. _RadioBrowser engine:
+
+============
+RadioBrowser
+============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.radio_browser
+   :members:

docs/dev/engines/online/recoll.rst

@@ -0,0 +1,13 @@
+.. _engine recoll:
+
+=============
+Recoll Engine
+=============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.recoll
+   :members:

docs/dev/engines/online/seekr.rst

@@ -0,0 +1,13 @@
+.. _seekr engine:
+
+=============
+Seekr Engines
+=============
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.seekr
+  :members:

docs/dev/engines/online/startpage.rst

@@ -1,10 +1,10 @@
 .. _startpage engines:
 
 =================
-Startpage engines
+Startpage Engines
 =================
 
-.. contents:: Contents
+.. contents::
    :depth: 2
    :local:
    :backlinks: entry

docs/dev/engines/online/tagesschau.rst

@@ -0,0 +1,13 @@
+.. _tagesschau engine:
+
+==============
+Tagesschau API
+==============
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.tagesschau
+  :members:

docs/dev/engines/online/torznab.rst

@@ -0,0 +1,13 @@
+.. _torznab engine:
+
+==============
+Torznab WebAPI
+==============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.torznab
+   :members:

docs/dev/engines/online/void.rst

@@ -0,0 +1,13 @@
+.. _voidlinux engine:
+
+==========================
+Void Linux binary packages
+==========================
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.voidlinux
+  :members: