[doc] rearranges Settings & Engines docs for better readability

We have built up detailed documentation of the *settings* and the *engines* over
the past few years.  However, this documentation was still spread over various
chapters and was difficult to navigate in its entirety.

This patch rearranges the Settings & Engines documentation for better
readability.

To review new ordered docs::

   make docs.clean docs.live

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

docs/dev/engines/demo/demo_offline.rst

@@ -0,0 +1,14 @@
+.. _demo offline engine:
+
+===================
+Demo Offline Engine
+===================
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.demo_offline
+  :members:
+

docs/dev/engines/demo/demo_online.rst

@@ -0,0 +1,14 @@
+.. _demo online engine:
+
+==================
+Demo Online Engine
+==================
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.demo_online
+  :members:
+

docs/dev/engines/engine_overview.rst

@@ -0,0 +1,468 @@
+.. _engines-dev:
+
+===============
+Engine Overview
+===============
+
+.. contents::
+   :depth: 3
+   :local:
+   :backlinks: entry
+
+.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
+
+.. sidebar:: Further reading ..
+
+   - :ref:`configured engines`
+   - :ref:`settings engine`
+
+SearXNG is a metasearch-engine_, so it uses different search engines to provide
+better results.
+
+Because there is no general search API which could be used for every search
+engine, an adapter has to be built between SearXNG and the external search
+engines.  Adapters are stored under the folder :origin:`searx/engines`.
+
+.. _general engine configuration:
+
+General Engine Configuration
+============================
+
+It is required to tell SearXNG the type of results the engine provides. The
+arguments can be set in the engine file or in the settings file (normally
+``settings.yml``). The arguments in the settings file override the ones in the
+engine file.
+
+It does not matter if an option is stored in the engine file or in the settings.
+However, the standard way is the following:
+
+.. _engine file:
+
+Engine File
+-----------
+
+.. table:: Common options in the engine module
+   :width: 100%
+
+   ======================= =========== ========================================================
+   argument                type        information
+   ======================= =========== ========================================================
+   categories              list        categories, in which the engine is working
+   paging                  boolean     support multiple pages
+   time_range_support      boolean     support search time range
+   engine_type             str         - ``online`` :ref:`[ref] <online engines>` by
+                                         default, other possibles values are:
+                                       - ``offline`` :ref:`[ref] <offline engines>`
+                                       - ``online_dictionary`` :ref:`[ref] <online dictionary>`
+                                       - ``online_currency`` :ref:`[ref] <online currency>`
+                                       - ``online_url_search`` :ref:`[ref] <online url search>`
+   ======================= =========== ========================================================
+
+.. _engine settings:
+
+Engine ``settings.yml``
+-----------------------
+
+For a more  detailed description, see :ref:`settings engine` in the :ref:`settings.yml`.
+
+.. table:: Common options in the engine setup (``settings.yml``)
+   :width: 100%
+
+   ======================= =========== ==================================================
+   argument                type        information
+   ======================= =========== ==================================================
+   name                    string      name of search-engine
+   engine                  string      name of searxng-engine (file name without ``.py``)
+   enable_http             bool        enable HTTP (by default only HTTPS is enabled).
+   shortcut                string      shortcut of search-engine
+   timeout                 string      specific timeout for search-engine
+   display_error_messages  boolean     display error messages on the web UI
+   proxies                 dict        set proxies for a specific engine
+                                       (e.g. ``proxies : {http: socks5://proxy:port,
+                                       https: socks5://proxy:port}``)
+   ======================= =========== ==================================================
+
+.. _engine overrides:
+
+Overrides
+---------
+
+A few of the options have default values in the namespace of engine's python
+modul, but are often overwritten by the settings.  If ``None`` is assigned to an
+option in the engine file, it has to be redefined in the settings, otherwise
+SearXNG will not start with that engine (global names with a leading underline can
+be ``None``).
+
+Here is an very simple example of the global names in the namespace of engine's
+module:
+
+.. code:: python
+
+   # engine dependent config
+   categories = ['general']
+   paging = True
+   _non_overwritten_global = 'foo'
+
+
+.. table:: The naming of overrides is arbitrary / recommended overrides are:
+   :width: 100%
+
+   ======================= =========== ===========================================
+   argument                type        information
+   ======================= =========== ===========================================
+   base_url                string      base-url, can be overwritten to use same
+                                       engine on other URL
+   number_of_results       int         maximum number of results per request
+   language                string      ISO code of language and country like en_US
+   api_key                 string      api-key if required by engine
+   ======================= =========== ===========================================
+
+.. _engine request:
+
+Making a Request
+================
+
+To perform a search an URL have to be specified.  In addition to specifying an
+URL, arguments can be passed to the query.
+
+.. _engine request arguments:
+
+Passed Arguments (request)
+--------------------------
+
+These arguments can be used to construct the search query.  Furthermore,
+parameters with default value can be redefined for special purposes.
+
+.. _engine request online:
+
+.. table:: If the ``engine_type`` is :py:obj:`online
+           <searx.search.processors.online.OnlineProcessor.get_params>`
+   :width: 100%
+
+   ====================== ============== ========================================================================
+   argument               type           default-value, information
+   ====================== ============== ========================================================================
+   url                    str            ``''``
+   method                 str            ``'GET'``
+   headers                set            ``{}``
+   data                   set            ``{}``
+   cookies                set            ``{}``
+   verify                 bool           ``True``
+   headers.User-Agent     str            a random User-Agent
+   category               str            current category, like ``'general'``
+   safesearch             int            ``0``, between ``0`` and ``2`` (normal, moderate, strict)
+   time_range             Optional[str]  ``None``, can be ``day``, ``week``, ``month``, ``year``
+   pageno                 int            current pagenumber
+   searxng_locale         str            SearXNG's locale selected by user.  Specific language code like
+                                         ``'en'``, ``'en-US'``, or ``'all'`` if unspecified.
+   ====================== ============== ========================================================================
+
+
+.. _engine request online_dictionary:
+
+.. table:: If the ``engine_type`` is :py:obj:`online_dictionary
+           <searx.search.processors.online_dictionary.OnlineDictionaryProcessor.get_params>`,
+           in addition to the :ref:`online <engine request online>` arguments:
+   :width: 100%
+
+   ====================== ============== ========================================================================
+   argument               type           default-value, information
+   ====================== ============== ========================================================================
+   from_lang              str            specific language code like ``'en_US'``
+   to_lang                str            specific language code like ``'en_US'``
+   query                  str            the text query without the languages
+   ====================== ============== ========================================================================
+
+.. _engine request online_currency:
+
+.. table:: If the ``engine_type`` is :py:obj:`online_currency
+           <searx.search.processors.online_currency.OnlineCurrencyProcessor.get_params>`,
+           in addition to the :ref:`online <engine request online>` arguments:
+   :width: 100%
+
+   ====================== ============== ========================================================================
+   argument               type           default-value, information
+   ====================== ============== ========================================================================
+   amount                 float          the amount to convert
+   from                   str            ISO 4217 code
+   to                     str            ISO 4217 code
+   from_name              str            currency name
+   to_name                str            currency name
+   ====================== ============== ========================================================================
+
+.. _engine request online_url_search:
+
+.. table:: If the ``engine_type`` is :py:obj:`online_url_search
+           <searx.search.processors.online_url_search.OnlineUrlSearchProcessor.get_params>`,
+           in addition to the :ref:`online <engine request online>` arguments:
+   :width: 100%
+
+   ====================== ============== ========================================================================
+   argument               type           default-value, information
+   ====================== ============== ========================================================================
+   search_url             dict           URLs from the search query:
+
+                                         .. code:: python
+
+                                            {
+                                              'http': str,
+                                              'ftp': str,
+                                              'data:image': str
+                                            }
+   ====================== ============== ========================================================================
+
+Specify Request
+---------------
+
+The function :py:func:`def request(query, params):
+<searx.engines.demo_online.request>` always returns the ``params`` variable, the
+following parameters can be used to specify a search request:
+
+.. table::
+   :width: 100%
+
+   =================== =========== ==========================================================================
+   argument            type        information
+   =================== =========== ==========================================================================
+   url                 str         requested url
+   method              str         HTTP request method
+   headers             set         HTTP header information
+   data                set         HTTP data information
+   cookies             set         HTTP cookies
+   verify              bool        Performing SSL-Validity check
+   allow_redirects     bool        Follow redirects
+   max_redirects       int         maximum redirects, hard limit
+   soft_max_redirects  int         maximum redirects, soft limit. Record an error but don't stop the engine
+   raise_for_httperror bool        True by default: raise an exception if the HTTP code of response is >= 300
+   =================== =========== ==========================================================================
+
+
+.. _engine results:
+.. _engine media types:
+
+Result Types (``template``)
+===========================
+
+Each result item of an engine can be of different media-types.  Currently the
+following media-types are supported.  To set another media-type as
+:ref:`template default`, the parameter ``template`` must be set to the desired
+type.
+
+.. _template default:
+
+``default``
+-----------
+
+.. table::  Parameter of the **default** media type:
+   :width: 100%
+
+   ========================= =====================================================
+   result-parameter          information
+   ========================= =====================================================
+   url                       string, url of the result
+   title                     string, title of the result
+   content                   string, general result-text
+   publishedDate             :py:class:`datetime.datetime`, time of publish
+   ========================= =====================================================
+
+
+.. _template images:
+
+``images``
+----------
+
+.. table::  Parameter of the **images** media type:
+   :width: 100%
+
+   ========================= =====================================================
+   result-parameter          information
+   ------------------------- -----------------------------------------------------
+   template                  is set to ``images.html``
+   ========================= =====================================================
+   url                       string, url to the result site
+   title                     string, title of the result *(partly implemented)*
+   content                   *(partly implemented)*
+   publishedDate             :py:class:`datetime.datetime`,
+                             time of publish *(partly implemented)*
+   img\_src                  string, url to the result image
+   thumbnail\_src            string, url to a small-preview image
+   ========================= =====================================================
+
+
+.. _template videos:
+
+``videos``
+----------
+
+.. table::  Parameter of the **videos** media type:
+   :width: 100%
+
+   ========================= =====================================================
+   result-parameter          information
+   ------------------------- -----------------------------------------------------
+   template                  is set to ``videos.html``
+   ========================= =====================================================
+   url                       string, url of the result
+   title                     string, title of the result
+   content                   *(not implemented yet)*
+   publishedDate             :py:class:`datetime.datetime`, time of publish
+   thumbnail                 string, url to a small-preview image
+   ========================= =====================================================
+
+
+.. _template torrent:
+
+``torrent``
+-----------
+
+.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
+
+.. table::  Parameter of the **torrent** media type:
+   :width: 100%
+
+   ========================= =====================================================
+   result-parameter          information
+   ------------------------- -----------------------------------------------------
+   template                  is set to ``torrent.html``
+   ========================= =====================================================
+   url                       string, url of the result
+   title                     string, title of the result
+   content                   string, general result-text
+   publishedDate             :py:class:`datetime.datetime`,
+                             time of publish *(not implemented yet)*
+   seed                      int, number of seeder
+   leech                     int, number of leecher
+   filesize                  int, size of file in bytes
+   files                     int, number of files
+   magnetlink                string, magnetlink_ of the result
+   torrentfile               string, torrentfile of the result
+   ========================= =====================================================
+
+
+.. _template map:
+
+``map``
+-------
+
+.. table::  Parameter of the **map** media type:
+   :width: 100%
+
+   ========================= =====================================================
+   result-parameter          information
+   ------------------------- -----------------------------------------------------
+   template                  is set to ``map.html``
+   ========================= =====================================================
+   url                       string, url of the result
+   title                     string, title of the result
+   content                   string, general result-text
+   publishedDate             :py:class:`datetime.datetime`, time of publish
+   latitude                  latitude of result (in decimal format)
+   longitude                 longitude of result (in decimal format)
+   boundingbox               boundingbox of result (array of 4. values
+                             ``[lat-min, lat-max, lon-min, lon-max]``)
+   geojson                   geojson of result (https://geojson.org/)
+   osm.type                  type of osm-object (if OSM-Result)
+   osm.id                    id of osm-object (if OSM-Result)
+   address.name              name of object
+   address.road              street name of object
+   address.house_number      house number of object
+   address.locality          city, place of object
+   address.postcode          postcode of object
+   address.country           country of object
+   ========================= =====================================================
+
+
+.. _template paper:
+
+``paper``
+---------
+
+.. _BibTeX format: https://www.bibtex.com/g/bibtex-format/
+.. _BibTeX field types: https://en.wikipedia.org/wiki/BibTeX#Field_types
+
+.. list-table:: Parameter of the **paper** media type /
+                see `BibTeX field types`_ and `BibTeX format`_
+   :header-rows: 2
+   :width: 100%
+
+   * - result-parameter
+     - Python type
+     - information
+
+   * - template
+     - :py:class:`str`
+     - is set to ``paper.html``
+
+   * - title
+     - :py:class:`str`
+     - title of the result
+
+   * - content
+     - :py:class:`str`
+     - abstract
+
+   * - comments
+     - :py:class:`str`
+     - free text display in italic below the content
+
+   * - tags
+     - :py:class:`List <list>`\ [\ :py:class:`str`\ ]
+     - free tag list
+
+   * - publishedDate
+     - :py:class:`datetime <datetime.datetime>`
+     - last publication date
+
+   * - type
+     - :py:class:`str`
+     - short description of medium type, e.g. *book*, *pdf* or *html* ...
+
+   * - authors
+     - :py:class:`List <list>`\ [\ :py:class:`str`\ ]
+     - list of authors of the work (authors with a "s")
+
+   * - editor
+     - :py:class:`str`
+     - list of editors of a book
+
+   * - publisher
+     - :py:class:`str`
+     - name of the publisher
+
+   * - journal
+     - :py:class:`str`
+     - name of the journal or magazine the article was
+       published in
+
+   * - volume
+     - :py:class:`str`
+     - volume number
+
+   * - pages
+     - :py:class:`str`
+     - page range where the article is
+
+   * - number
+     - :py:class:`str`
+     - number of the report or the issue number for a journal article
+
+   * - doi
+     - :py:class:`str`
+     - DOI number (like ``10.1038/d41586-018-07848-2``)
+
+   * - issn
+     - :py:class:`List <list>`\ [\ :py:class:`str`\ ]
+     - ISSN number like ``1476-4687``
+
+   * - isbn
+     - :py:class:`List <list>`\ [\ :py:class:`str`\ ]
+     - ISBN number like ``9780201896831``
+
+   * - pdf_url
+     - :py:class:`str`
+     - URL to the full article, the PDF version
+
+   * - html_url
+     - :py:class:`str`
+     - URL to full article, HTML version
+

docs/dev/engines/enginelib.rst

@@ -0,0 +1,22 @@
+.. _searx.enginelib:
+
+==============
+Engine Library
+==============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.enginelib
+  :members:
+
+.. _searx.enginelib.traits:
+
+
+Engine traits
+=============
+
+.. automodule:: searx.enginelib.traits
+  :members:

docs/dev/engines/engines.rst

@@ -0,0 +1,9 @@
+.. _searx.engines loader:
+
+========================
+SearXNG's engines loader
+========================
+
+.. automodule:: searx.engines
+  :members:
+

docs/dev/engines/index.rst

@@ -0,0 +1,97 @@
+.. _engine implementations:
+
+======================
+Engine Implementations
+======================
+
+Framework Components
+====================
+
+.. toctree::
+   :maxdepth: 2
+
+   enginelib
+   engines
+   engine_overview
+
+
+Engine Types
+============
+
+The :py:obj:`engine_type <searx.enginelib.Engine.engine_type>` of an engine
+determines which :ref:`search processor <searx.search.processors>` is used by
+the engine.
+
+In this section a list of the enignes that are documented is given, a complete
+list of the engines can be found in the source under: :origin:`searx/engines`.
+
+.. _online engines:
+
+Online Engines
+--------------
+
+.. sidebar:: info
+
+   - :py:obj:`processors.online <searx.search.processors.online>`
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   demo/demo_online
+   xpath
+   online/*
+
+.. _offline engines:
+
+Offline Engines
+---------------
+
+.. sidebar:: info
+
+   - :py:obj:`processors.offline <searx.search.processors.offline>`
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   offline_concept
+   demo/demo_offline
+   offline/*
+
+.. _online url search:
+
+Online URL Search
+-----------------
+
+.. sidebar:: info
+
+   - :py:obj:`processors.online_url_search <searx.search.processors.online_url_search>`
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   online_url_search/*
+
+.. _online currency:
+
+Online Currency
+---------------
+
+.. sidebar:: info
+
+   - :py:obj:`processors.online_currency <searx.search.processors.online_currency>`
+
+*no engine of this type is documented yet / comming soon*
+
+.. _online dictionary:
+
+Online Dictionary
+-----------------
+
+.. sidebar:: info
+
+   - :py:obj:`processors.online_dictionary <searx.search.processors.online_dictionary>`
+
+*no engine of this type is documented yet / comming soon*

docs/dev/engines/offline/command-line-engines.rst

@@ -0,0 +1,23 @@
+.. _engine command:
+
+====================
+Command Line Engines
+====================
+
+.. sidebar:: info
+
+   - :origin:`command.py <searx/engines/command.py>`
+   - :ref:`offline engines`
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. sidebar:: info
+
+   Initial sponsored by `Search and Discovery Fund
+   <https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
+
+.. automodule:: searx.engines.command
+  :members:

docs/dev/engines/offline/nosql-engines.rst

@@ -0,0 +1,97 @@
+.. _nosql engines:
+
+===============
+NoSQL databases
+===============
+
+.. sidebar:: further read
+
+   - `NoSQL databases <https://en.wikipedia.org/wiki/NoSQL>`_
+   - `redis.io <https://redis.io/>`_
+   - `MongoDB <https://www.mongodb.com>`_
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. sidebar:: info
+
+   Initial sponsored by `Search and Discovery Fund
+   <https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
+
+The following `NoSQL databases`_ are supported:
+
+- :ref:`engine redis_server`
+- :ref:`engine mongodb`
+
+All of the engines above are just commented out in the :origin:`settings.yml
+<searx/settings.yml>`, as you have to set various options and install
+dependencies before using them.
+
+By default, the engines use the ``key-value`` template for displaying results /
+see :origin:`simple <searx/templates/simple/result_templates/key-value.html>`
+theme.  If you are not satisfied with the original result layout, you can use
+your own template, set ``result_template`` attribute to ``{template_name}`` and
+place the templates at::
+
+  searx/templates/{theme_name}/result_templates/{template_name}
+
+Furthermore, if you do not wish to expose these engines on a public instance, you
+can still add them and limit the access by setting ``tokens`` as described in
+section :ref:`private engines`.
+
+
+Extra Dependencies
+==================
+
+For using :ref:`engine redis_server` or :ref:`engine mongodb` you need to
+install additional packages in Python's Virtual Environment of your SearXNG
+instance.  To switch into the environment (:ref:`searxng-src`) you can use
+:ref:`searxng.sh`::
+
+  $ sudo utils/searxng.sh instance cmd bash
+  (searxng-pyenv)$ pip install ...
+
+
+Configure the engines
+=====================
+
+`NoSQL databases`_ are used for storing arbitrary data without first defining
+their structure.
+
+
+.. _engine redis_server:
+
+Redis Server
+------------
+
+.. _redis: https://github.com/andymccurdy/redis-py#installation
+
+.. sidebar:: info
+
+   - ``pip install`` redis_
+   - redis.io_
+   - :origin:`redis_server.py <searx/engines/redis_server.py>`
+
+.. automodule:: searx.engines.redis_server
+  :members:
+
+
+.. _engine mongodb:
+
+MongoDB
+-------
+
+.. _pymongo: https://github.com/mongodb/mongo-python-driver#installation
+
+.. sidebar:: info
+
+   - ``pip install`` pymongo_
+   - MongoDB_
+   - :origin:`mongodb.py <searx/engines/mongodb.py>`
+
+
+.. automodule:: searx.engines.mongodb
+  :members:
+

docs/dev/engines/offline/search-indexer-engines.rst

@@ -0,0 +1,62 @@
+=================
+Local Search APIs
+=================
+
+.. sidebar:: further read
+
+   - `Comparison to alternatives
+     <https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html>`_
+
+.. contents::
+   :depth: 1
+   :local:
+   :backlinks: entry
+
+.. sidebar:: info
+
+   Initial sponsored by `Search and Discovery Fund
+   <https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
+
+Administrators might find themselves wanting to integrate locally running search
+engines.  The following ones are supported for now:
+
+* `Elasticsearch`_
+* `Meilisearch`_
+* `Solr`_
+
+Each search engine is powerful, capable of full-text search.  All of the engines
+above are added to ``settings.yml`` just commented out, as you have to
+``base_url`` for all them.
+
+Please note that if you are not using HTTPS to access these engines, you have to
+enable HTTP requests by setting ``enable_http`` to ``True``.
+
+Furthermore, if you do not want to expose these engines on a public instance,
+you can still add them and limit the access by setting ``tokens`` as described
+in section :ref:`private engines`.
+
+.. _engine meilisearch:
+
+MeiliSearch
+===========
+
+.. automodule:: searx.engines.meilisearch
+  :members:
+
+
+.. _engine elasticsearch:
+
+Elasticsearch
+=============
+
+.. automodule:: searx.engines.elasticsearch
+  :members:
+
+.. _engine solr:
+
+Solr
+====
+
+.. automodule:: searx.engines.solr
+  :members:
+

docs/dev/engines/offline/sql-engines.rst

@@ -0,0 +1,121 @@
+.. _sql engines:
+
+===========
+SQL Engines
+===========
+
+.. sidebar:: further read
+
+   - `SQLite <https://www.sqlite.org/index.html>`_
+   - `PostgreSQL <https://www.postgresql.org>`_
+   - `MySQL <https://www.mysql.com>`_
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. sidebar:: info
+
+   Initial sponsored by `Search and Discovery Fund
+   <https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
+
+With the *SQL engines* you can bind SQL databases into SearXNG.  The following
+Relational Database Management System (RDBMS) are supported:
+
+- :ref:`engine sqlite`
+- :ref:`engine postgresql`
+- :ref:`engine mysql_server`
+
+All of the engines above are just commented out in the :origin:`settings.yml
+<searx/settings.yml>`, as you have to set the required attributes for the
+engines, e.g. ``database:`` ...
+
+.. code:: yaml
+
+   - name: ...
+     engine: {sqlite|postgresql|mysql_server}
+     database: ...
+     result_template: {template_name}
+     query_str: ...
+
+By default, the engines use the ``key-value`` template for displaying results /
+see :origin:`simple <searx/templates/simple/result_templates/key-value.html>`
+theme.  If you are not satisfied with the original result layout, you can use
+your own template, set ``result_template`` attribute to ``{template_name}`` and
+place the templates at::
+
+  searx/templates/{theme_name}/result_templates/{template_name}
+
+If you do not wish to expose these engines on a public instance, you can still
+add them and limit the access by setting ``tokens`` as described in section
+:ref:`private engines`.
+
+
+Extra Dependencies
+==================
+
+For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to
+install additional packages in Python's Virtual Environment of your SearXNG
+instance.  To switch into the environment (:ref:`searxng-src`) you can use
+:ref:`searxng.sh`::
+
+  $ sudo utils/searxng.sh instance cmd bash
+  (searxng-pyenv)$ pip install ...
+
+
+Configure the engines
+=====================
+
+The configuration of the new database engines are similar.  You must put a valid
+SQL-SELECT query in ``query_str``.  At the moment you can only bind at most one
+parameter in your query.  By setting the attribute ``limit`` you can define how
+many results you want from the SQL server.  Basically, it is the same as the
+``LIMIT`` keyword in SQL.
+
+Please, do not include ``LIMIT`` or ``OFFSET`` in your SQL query as the engines
+rely on these keywords during paging.  If you want to configure the number of
+returned results use the option ``limit``.
+
+.. _engine sqlite:
+
+SQLite
+------
+
+.. sidebar:: info
+
+   - :origin:`sqlite.py <searx/engines/sqlite.py>`
+
+.. automodule:: searx.engines.sqlite
+  :members:
+
+
+.. _engine postgresql:
+
+PostgreSQL
+----------
+
+.. _psycopg2: https://www.psycopg.org/install
+
+.. sidebar:: info
+
+   - :origin:`postgresql.py <searx/engines/postgresql.py>`
+   - ``pip install`` `psycopg2-binary <psycopg2>`_
+
+.. automodule:: searx.engines.postgresql
+  :members:
+
+.. _engine mysql_server:
+
+MySQL
+-----
+
+.. sidebar:: info
+
+   - :origin:`mysql_server.py <searx/engines/mysql_server.py>`
+   - ``pip install`` :pypi:`mysql-connector-python <mysql-connector-python>`
+
+
+.. automodule:: searx.engines.mysql_server
+  :members:
+

docs/dev/engines/offline_concept.rst

@@ -0,0 +1,69 @@
+===============
+Offline Concept
+===============
+
+.. sidebar:: offline engines
+
+   - :ref:`demo offline engine`
+   - :ref:`engine command`
+   - :ref:`sql engines`
+   - :ref:`nosql engines`
+   - :py:obj:`searx.search.processors.offline`
+
+To extend the functionality of SearXNG, offline engines are going to be
+introduced.  An offline engine is an engine which does not need Internet
+connection to perform a search and does not use HTTP to communicate.
+
+Offline engines can be configured, by adding those to the `engines` list of
+:origin:`settings.yml <searx/settings.yml>`.  An example skeleton for offline
+engines can be found in :ref:`demo offline engine` (:origin:`demo_offline.py
+<searx/engines/demo_offline.py>`).
+
+
+Programming Interface
+=====================
+
+:py:func:`init(engine_settings=None) <searx.engines.demo_offline.init>`
+  All offline engines can have their own init function to setup the engine before
+  accepting requests. The function gets the settings from settings.yml as a
+  parameter. This function can be omitted, if there is no need to setup anything
+  in advance.
+
+:py:func:`search(query, params) <searx.engines.demo_offline.searc>`
+  Each offline engine has a function named ``search``.  This function is
+  responsible to perform a search and return the results in a presentable
+  format. (Where *presentable* means presentable by the selected result
+  template.)
+
+  The return value is a list of results retrieved by the engine.
+
+Engine representation in ``/config``
+  If an engine is offline, the attribute ``offline`` is set to ``True``.
+
+.. _offline requirements:
+
+Extra Dependencies
+==================
+
+If an offline engine depends on an external tool, SearXNG does not install it by
+default.  When an administrator configures such engine and starts the instance,
+the process returns an error with the list of missing dependencies.  Also,
+required dependencies will be added to the comment/description of the engine, so
+admins can install packages in advance.
+
+If there is a need to install additional packages in *Python's Virtual
+Environment* of your SearXNG instance you need to switch into the environment
+(:ref:`searxng-src`) first, for this you can use :ref:`searxng.sh`::
+
+  $ sudo utils/searxng.sh instance cmd bash
+  (searxng-pyenv)$ pip install ...
+
+
+Private engines (Security)
+==========================
+
+To limit the access to offline engines, if an instance is available publicly,
+administrators can set token(s) for each of the :ref:`private engines`.  If a
+query contains a valid token, then SearXNG performs the requested private
+search.  If not, requests from an offline engines return errors.
+

docs/dev/engines/online/annas_archive.rst

@@ -0,0 +1,13 @@
+.. _annas_archive engine:
+
+==============
+Anna's Archive
+==============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.annas_archive
+   :members:

docs/dev/engines/online/archlinux.rst

@@ -0,0 +1,14 @@
+.. _archlinux engine:
+
+==========
+Arch Linux
+==========
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.archlinux
+  :members:
+

docs/dev/engines/online/bing.rst

@@ -0,0 +1,43 @@
+.. _bing engines:
+
+============
+Bing Engines
+============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+
+.. _bing web engine:
+
+Bing WEB
+========
+
+.. automodule:: searx.engines.bing
+  :members:
+
+.. _bing images engine:
+
+Bing Images
+===========
+
+.. automodule:: searx.engines.bing_images
+  :members:
+
+.. _bing videos engine:
+
+Bing Videos
+===========
+
+.. automodule:: searx.engines.bing_videos
+  :members:
+
+.. _bing news engine:
+
+Bing News
+=========
+
+.. automodule:: searx.engines.bing_news
+  :members:

docs/dev/engines/online/dailymotion.rst

@@ -0,0 +1,13 @@
+.. _dailymotion engine:
+
+===========
+Dailymotion
+===========
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.dailymotion
+  :members:

docs/dev/engines/online/duckduckgo.rst

@@ -0,0 +1,22 @@
+.. _duckduckgo engines:
+
+=================
+DukcDukGo Engines
+=================
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.duckduckgo
+   :members:
+
+.. automodule:: searx.engines.duckduckgo_images
+   :members:
+
+.. automodule:: searx.engines.duckduckgo_definitions
+   :members:
+
+.. automodule:: searx.engines.duckduckgo_weather
+   :members:

docs/dev/engines/online/google.rst

@@ -0,0 +1,76 @@
+.. _google engines:
+
+==============
+Google Engines
+==============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+
+.. _google API:
+
+Google API
+==========
+
+.. _Query Parameter Definitions:
+   https://developers.google.com/custom-search/docs/xml_results#WebSearch_Query_Parameter_Definitions
+
+SearXNG's implementation of the Google API is mainly done in
+:py:obj:`get_google_info <searx.engines.google.get_google_info>`.
+
+For detailed description of the *REST-full* API see: `Query Parameter
+Definitions`_.  The linked API documentation can sometimes be helpful during
+reverse engineering.  However, we cannot use it in the freely accessible WEB
+services; not all parameters can be applied and some engines are more *special*
+than other (e.g. :ref:`google news engine`).
+
+
+.. _google web engine:
+
+Google WEB
+==========
+
+.. automodule:: searx.engines.google
+  :members:
+
+.. _google autocomplete:
+
+Google Autocomplete
+====================
+
+.. autofunction:: searx.autocomplete.google_complete
+
+.. _google images engine:
+
+Google Images
+=============
+
+.. automodule:: searx.engines.google_images
+  :members:
+
+.. _google videos engine:
+
+Google Videos
+=============
+
+.. automodule:: searx.engines.google_videos
+  :members:
+
+.. _google news engine:
+
+Google News
+===========
+
+.. automodule:: searx.engines.google_news
+  :members:
+
+.. _google scholar engine:
+
+Google Scholar
+==============
+
+.. automodule:: searx.engines.google_scholar
+  :members:

docs/dev/engines/online/peertube.rst

@@ -0,0 +1,27 @@
+.. _peertube engines:
+
+================
+Peertube Engines
+================
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+
+.. _peertube video engine:
+
+Peertube Video
+==============
+
+.. automodule:: searx.engines.peertube
+  :members:
+
+.. _sepiasearch engine:
+
+SepiaSearch
+===========
+
+.. automodule:: searx.engines.sepiasearch
+  :members:

docs/dev/engines/online/recoll.rst

@@ -0,0 +1,13 @@
+.. _engine recoll:
+
+=============
+Recoll Engine
+=============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.recoll
+   :members:

docs/dev/engines/online/startpage.rst

@@ -0,0 +1,13 @@
+.. _startpage engines:
+
+=================
+Startpage Engines
+=================
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.startpage
+   :members:

docs/dev/engines/online/torznab.rst

@@ -0,0 +1,13 @@
+.. _torznab engine:
+
+==============
+Torznab WebAPI
+==============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.torznab
+   :members:

docs/dev/engines/online/wikipedia.rst

@@ -0,0 +1,27 @@
+.. _wikimedia engines:
+
+=========
+Wikimedia
+=========
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+
+.. _wikipedia engine:
+
+Wikipedia
+=========
+
+.. automodule:: searx.engines.wikipedia
+  :members:
+
+.. _wikidata engine:
+
+Wikidata
+=========
+
+.. automodule:: searx.engines.wikidata
+  :members:

docs/dev/engines/online/yahoo.rst

@@ -0,0 +1,13 @@
+.. _yahoo engine:
+
+============
+Yahoo Engine
+============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.yahoo
+  :members:

docs/dev/engines/online_url_search/tineye.rst

@@ -0,0 +1,14 @@
+.. _tineye engine:
+
+======
+Tineye
+======
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.tineye
+  :members:
+

docs/dev/engines/xpath.rst

@@ -0,0 +1,14 @@
+.. _xpath engine:
+
+============
+XPath Engine
+============
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.engines.xpath
+  :members:
+