There are several reasons why we should prefer markdown-it-py over mistletoe:
- Get identical rendering results in SearXNG's `/info` pages and the SearXNG's
project documentation which is build by Sphinx-doc.
In the Sphinx-doc we use the MyST parser to render Markdown and the MyST
parser itself is built on top of the markdown-it-py package.
- markdown-it-py has a typographer that supports *replacements*
and *smartquotes* (e.g. em-dash, copyright, ellipsis, ...) [1]
- markdown-it-py is much more flexible compared to mistletoe [2]
- markdown-it-py is the fastest CommonMark compliant parser in python [3]
[1] https://markdown-it-py.readthedocs.io/en/latest/using.html#typographic-components
[2] https://markdown-it-py.readthedocs.io/en/latest/plugins.html
[3] https://markdown-it-py.readthedocs.io/en/latest/other.html#performance
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
With this patch ``searxng.msg`` files can be added to SearXNG. In
``searxng.msg`` files messages can be defined which are not captured by babel's
gettext, like the generic names of the categories or messages that are stored in
constants.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch implements a bolierplate to share content from info-pages of the
SearXNG instance (URL /info) with the project documentation (path /docs/user).
The info pages are using Markdown (CommonMark), to include them in the project
documentation (reST) the myst-parser [1] is used in the Sphinx-doc build chain.
If base_url is known (defined in settings.yml) links to the instance are also
inserted into the project documentation::
searxng_extra/docs_prebuild
[1] https://www.sphinx-doc.org/en/master/usage/markdown.html
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
can replace filtron:
* rate limite the number of request per IP and per (IP, User-Agent)
* block some bots
use Redis
data stored in Redis never contains the IP addresses, only HMAC using the secret_key
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
The PluginStore is already initalized when the application is initalized
searx.plugins.initialize(application)
BTW: remove unneeded Flask import
Closes: https://github.com/searxng/searxng/issues/828
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
With Sphinx-doc update 4.4.0 we get some warnings about links that can be
replaced by already defined 'sphinx.ext.extlinks':
admin/engines/sql-engines.rst:144: WARNING: hardcoded link 'https://pypi.org/project/mysql-connector-python' could be replaced by an extlink (try using ':pypi:`mysql-connector-python`' instead)
docs/admin/installation-switch2ng.rst:10: WARNING: hardcoded link 'https://github.com/searxng/searxng/pull/446#issuecomment-954730358' could be replaced by an extlink (try using ':pull:`446#issuecomment-954730358`' instead)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The ? search operator has been broken for some time and
currently only raises the question why it's still there.
## Context ##
The query "Paris !images" searches for "Paris" in the "images" category.
Once upon a time Searx supported "Paris ?images" to search for "Paris"
in the currently enabled categories and the "images" category.
The feature makes sense ... the ? syntax does not.
We will hopefully introduce a +!images syntax in the future.
Fixes#702.
* allow not to record metrics (response time, etc...)
* this commit doesn't change the UI. If the metrics are disabled
/stats and /stats/errors will return empty response.
in /preferences, the columns response time and reliability will be empty.
The tab icon names are currently hard coded in the templates.
This commit lets us introduce an icon property in the future, e.g:
categories_as_tabs:
general:
icon: search-outline
Add a redis connector, the default DB connector is a socket at::
unix:///usr/local/searxng-redis/run/redis.sock?db=0
To set up a redis instance simply use::
$ ./manage redis.build
$ sudo -H ./manage redis.install
A hint for developers:
To get access rights to this instance, your developer account needs to be added
to the *searxng-redis* group::
$ sudo -H ./manage redis.addgrp "${USER}"
# don't forget to logout & login to get member of group
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Stuff in folder searxng_extra/ is not suitable for normal users and should only
be used by developers.
The script searxng_extra/standalone_searx.py must not give the impression that
it improves privacy. [1]
[1] https://github.com/searxng/searxng/pull/651#issuecomment-1001389726
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Previously all categories were displayed as search engine tabs.
This commit changes that so that only the categories listed under
categories_as_tabs in settings.yml are displayed.
This lets us introduce more categories without cluttering up the UI.
Categories not displayed as tabs can still be searched with !bangs.
Previously the documentation grouped the engines by their first
category so e.g. YouTube and Invidious were only shown in the
in the videos section but not in the music section.
This commit fixes this by iterating over searx.engines.categories,
which also has the added benefit that the sections are now in the
same order as the tabs in the user interface.
Environment variable to get live builds while modifying CSS & JS of a theme::
LIVE_THEME=simple make run
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* move `searx/static/themes/simple/img/searxng.svg` to `src/brand/searxng.svg`
* README.rst can use it without a reference to a theme.
* the simple theme can create `searx/static/themes/simple/img/searxng.png` using
the svg2png task
Suggested-by: @dalf https://github.com/searxng/searxng/pull/561#issuecomment-981747902
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Languages are supported by mapping the language to a domain. If domain is not
found in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.
BTW: fix issue reported at https://github.com/searx/searx/issues/3020
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Functions implemented to run *Weblate* workflows should use prefix 'weblate.':
- babel.setup.translations.worktree --> weblate.translations.worktree
Create git worktree ${TRANSLATIONS_WORKTREE} and checkout branch
'translations' from Weblate's counterpart (weblate) of the SearXNG
(origin)::
remote weblate https://weblate.bubu1.eu/git/searxng/searxng/
- babel.weblate.to.translations --> weblate.to.translations
Update 'translations' branch of SearXNG (origin) with last additions from
Weblate.
- babel.translations.to.master --> weblate.translations.commit
Update 'translations' branch of SearXNG (origin) with last additions from
Weblate. Copy the changes to the master branch, compile translations and
create a commit in the local branch (master)
- babel.master.to.translations --> weblate.push.translations
Push *translation changes* from SearXNG (origin) to Weblate's
counterpart (weblate).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Markus Heiser
Alexandre Flament
Martin Fischer
Fabio Berchtold