All favicons implementations have been documented and moved to the Python
package:
searx.favicons
There is a configuration (based on Pydantic) for the favicons and all its
components:
searx.favicons.config
A solution for caching favicons has been implemented:
searx.favicon.cache
If the favicon is already in the cache, the returned URL is a data URL [1]
(something like `data:image/png;base64,...`). By generating a data url from
the FaviconCache, additional HTTP roundtripps via the favicon_proxy are saved:
favicons.proxy.favicon_url
The favicon proxy service now sets a HTTP header "Cache-Control: max-age=...":
favicons.proxy.favicon_proxy
The resolvers now also provide the mime type (data, mime):
searx.favicon.resolvers
[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URLs
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The intention of this PR is to modernize the settings_loader implementations.
The concept is old (remember, this is partly from 2014), back then we only had
one config file, meanwhile we have had a folder with config files for a very
long time. Callers can now load a YAML configuration from this folder as
follows ::
settings_loader.get_yaml_cfg('my-config.yml')
- BTW this is a fix of #3557.
- Further the `existing_filename_or_none` construct dates back to times when
there was not yet a `pathlib.Path` in all Python versions we supported in the
past.
- Typehints have been added wherever appropriate
At the same time, this patch should also be downward compatible and not
introduce a new environment variable. The localization of the folder with the
configurations is further based on:
SEARXNG_SETTINGS_PATH (wich defaults to /etc/searxng/settings.yml)
Which means, the default config folder is `/etc/searxng/`.
ATTENTION: intended functional changes!
If SEARXNG_SETTINGS_PATH was set and pointed to a not existing file, the
previous implementation silently loaded the default configuration. This
behavior has been changed: if the file or folder does not exist, an
EnvironmentError exception will be thrown in future.
Closes: https://github.com/searxng/searxng/issues/3557
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
HTTP GET vs POST
================
Already diskussed in "Method POST harms UX without providing a tangible privacy
benefit" [1]. At that time we agreed to POST. However, as we are having more
and more drawbacks with POST, I suggest that we reconsider our previous
decision. The latest cause was [2], but we also have other problems that
negatively affect the UI [1]:
* Open Link in New Tab: does not work for our search result tabs because they
aren't actual links.
* Bookmarking: a search becomes more difficult with POST. You cannot just press
a browser keyboard shortcut `Ctrl+D` ... no you have to copy the Search URL from
the sidebar. This can pose a real struggle for less technically-minded users.
* Sharing: a search with somebody else becomes more difficult with POST (for
the same reason).
Lock HTTP method in the preferences
===================================
If the user changes the HTTP method in his settings, e.g. from GET to POST, but
has not removed the SearXNG instance from the WEB browser and added it again,
the WEB browser will continue to work with the old setting (GET), while entries
in the HTML form use the newly set method (POST). Not realted to this commit,
but this complication is also known from autocomplete[3].
Only very few maintainers are aware of this fact and probably none of the users
know about it. We should provide a setup in our defaults that is manageable in
its entirety and comprehensible for the user. For this reason, the option to
select the HTTP method in the preferences is also disabled in this commit.
[1] https://github.com/searxng/searxng/issues/711
[2] https://github.com/searxng/searxng/issues/3590
[3] https://github.com/searxng/searxng/pull/2333#issuecomment-1565392120
Closes: https://github.com/searxng/searxng/issues/3590
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
All the environments defined in ./utils/brand.env are generated on the fly, so
there is no longer a need to define the brand environment in this file and all
the workflows to handle this file.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- the option server:public_instance lacks some documentation
- the processing of this option belongs in the limiter and not
in botdetection module
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch was inspired by the discussion around PR-2882 [2]. The goals of this
patch are:
1. Convert plugin searx.plugin.limiter to normal code [1]
2. isolation of botdetection from the limiter [2]
3. searx/{tools => botdetection}/config.py and drop searx.tools
4. in URL /config, 'limiter.enabled' is true only if the limiter is really
enabled (Redis is available).
This patch moves all the code that belongs to botdetection into namespace
searx.botdetection and code that belongs to limiter is placed in namespace
searx.limiter.
Tthe limiter used to be a plugin at some point botdetection was added, it was
not a plugin. The modularization of these two components was long overdue.
With the clear modularization, the documentation could then also be organized
according to the architecture.
[1] https://github.com/searxng/searxng/pull/2882
[2] https://github.com/searxng/searxng/pull/2882#issuecomment-1741716891
To test:
- check the app works without the limiter, check `/config`
- check the app works with the limiter and with the token, check `/config`
- make docs.live .. and read
- http://0.0.0.0:8000/admin/searx.limiter.html
- http://0.0.0.0:8000/src/searx.botdetection.html#botdetection
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The change in the hotkey mechanism introduced in 317db5b04 does not allow
configuration via `settings.yml`. This commit adds that functionality.
Closes: #2898
BTW force modularization of the ./mange script into sub modules:
- utils/lib_sxng_data.sh
- utils/lib_sxng_node.sh
- utils/lib_sxng_static.sh
- utils/lib_sxng_test.sh
- utils/lib_sxng_themes.sh
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
We have built up detailed documentation of the *settings* and the *engines* over
the past few years. However, this documentation was still spread over various
chapters and was difficult to navigate in its entirety.
This patch rearranges the Settings & Engines documentation for better
readability.
To review new ordered docs::
make docs.clean docs.live
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Markus Heiser
Brock Vojkovic
Bnyro
holysoles
Ember
Alex Balgavy
ToxyFlog1627