Commit Graph

76 Commits

Author SHA1 Message Date
Markus Heiser 6e5f22e558 [mod] replace engines_languages.json by engines_traits.json
Implementations of the *traits* of the engines.

Engine's traits are fetched from the origin engine and stored in a JSON file in
the *data folder*.  Most often traits are languages and region codes and their
mapping from SearXNG's representation to the representation in the origin search
engine.

To load traits from the persistence::

    searx.enginelib.traits.EngineTraitsMap.from_data()

For new traits new properties can be added to the class::

    searx.enginelib.traits.EngineTraits

.. hint::

   Implementation is downward compatible to the deprecated *supported_languages
   method* from the vintage implementation.

   The vintage code is tagged as *deprecated* an can be removed when all engines
   has been ported to the *traits method*.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-03-24 10:37:42 +01:00
Markus Heiser ed8a169029 [doc] update documentation of the installation procedures
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-30 13:39:35 +02:00
Markus Heiser d3226b3df5 [fix] Sphinx 5.x: will warn about missleading extlink definitions
This patch fixes the WARNING messages that pops up since Sphinx 5.x:

    WARNING: extlinks: Sphinx-6.0 will require a caption string to contain
             exactly one '%s' and all other '%' need to be escaped as '%%'.

[1] https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-05 17:10:19 +02:00
Markus Heiser b224761a1b [doc] intersphinx: fix https://python-babel.github.io/flask-babel
The URL https://flask-babel.tkte.ch/ is no longer valid [1].

[1] https://github.com/python-babel/flask-babel/commit/0847cc6284

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-06-22 09:22:21 +02:00
samsaptidev 31005595c9
Add privacypolicy_url option 2022-06-16 11:56:24 +02:00
Markus Heiser ef4239c68a [doc] fix some leftovers from ad964562c
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-06-14 16:31:41 +02:00
Markus Heiser 08876df7c2 [mod] collection of redis functions and lua scripts (initial)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-05-30 11:10:30 +02:00
Markus Heiser 9215281232 [mod] replace Markdown parser mistletoe by markdown-it-py
There are several reasons why we should prefer markdown-it-py over mistletoe:

- Get identical rendering results in SearXNG's `/info` pages and the SearXNG's
  project documentation which is build by Sphinx-doc.

  In the Sphinx-doc we use the MyST parser to render Markdown and the MyST
  parser itself is built on top of the markdown-it-py package.

- markdown-it-py has a typographer that supports *replacements*
  and *smartquotes* (e.g. em-dash, copyright, ellipsis, ...) [1]

- markdown-it-py is much more flexible compared to mistletoe [2]

- markdown-it-py is the fastest CommonMark compliant parser in python [3]

[1] https://markdown-it-py.readthedocs.io/en/latest/using.html#typographic-components
[2] https://markdown-it-py.readthedocs.io/en/latest/plugins.html
[3] https://markdown-it-py.readthedocs.io/en/latest/other.html#performance

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-04-06 15:49:50 +02:00
Markus Heiser 24f53276c1 [doc] add sphinx-notfound-page
Closes: https://github.com/searxng/searxng/issues/703
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-03-22 21:18:17 +01:00
Markus Heiser 8751940169 [interim fix] of sphinx-tabs and sphinx-jinja
The myst-parser requires >= docutils v.0.17 what ends in a dependency hell where
plugins sphinx-tabs and sphinx-jinja we use are involved.

This patch can be reverted when [2], [3], [4]  are solved and new release is
available / see [1].

[1] https://github.com/searxng/searxng/pull/954
[2] https://github.com/executablebooks/sphinx-tabs/issues/152
[3] https://github.com/executablebooks/sphinx-tabs/pull/153
[4] https://github.com/executablebooks/sphinx-tabs/pull/154

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-03-12 14:25:34 +01:00
Markus Heiser b1912607ae [mod] replace /help by /info pages and include pages in project docs
This patch implements a bolierplate to share content from info-pages of the
SearXNG instance (URL /info) with the project documentation (path /docs/user).

The info pages are using Markdown (CommonMark), to include them in the project
documentation (reST) the myst-parser [1] is used in the Sphinx-doc build chain.

If base_url is known (defined in settings.yml) links to the instance are also
inserted into the project documentation::

    searxng_extra/docs_prebuild

[1] https://www.sphinx-doc.org/en/master/usage/markdown.html

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-03-12 11:36:31 +01:00
Markus Heiser bdc28e3353 [fix] docs - don't add plugins twice
The PluginStore is already initalized when the application is initalized

    searx.plugins.initialize(application)

BTW: remove unneeded Flask import

Closes: https://github.com/searxng/searxng/issues/828
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-27 08:27:41 +01:00
Markus Heiser 613fb15599 [fix] Documentation of the builtin plugins
Closes: https://github.com/searxng/searxng/issues/637
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 19:59:42 +01:00
Martin Fischer 1e195f5b95 [mod] move group_engines_in_tab to searx.webutils 2022-01-05 11:03:44 +01:00
Martin Fischer 8e9ad1ccc2 [enh] introduce categories_as_tabs
Previously all categories were displayed as search engine tabs.
This commit changes that so that only the categories listed under
categories_as_tabs in settings.yml are displayed.

This lets us introduce more categories without cluttering up the UI.
Categories not displayed as tabs  can still be searched with !bangs.
2022-01-03 07:01:49 +01:00
Martin Fischer 02e9bdf755 [doc] engine tables: show engines in all categories
Previously the documentation grouped the engines by their first
category so e.g. YouTube and Invidious were only shown in the
in the videos section but not in the music section.

This commit fixes this by iterating over searx.engines.categories,
which also has the added benefit that the sections are now in the
same order as the tabs in the user interface.
2022-01-03 07:01:49 +01:00
Martin Fischer d8af94b721 [doc] engine-table: stop sorting by Disabled
It's only the default value for disabled it's not that important,
since users can easily toggle it anytime in the engine preferences.
2022-01-03 07:01:49 +01:00
Markus Heiser 188efe53e8 [doc] add global TOC to sidebar
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-12-24 07:05:33 +01:00
Martin Fischer ca7f1a2da6 [doc] engine-table: link module documentation if it exists 2021-12-21 21:06:17 +01:00
Martin Fischer 33983809e4 [doc] say how many engines are supported / enabled 2021-12-21 13:00:31 +01:00
Martin Fischer e28c6bda35 [doc] introduce about.language and sort engines by it 2021-12-21 09:58:51 +01:00
Martin Fischer df0d0ecaab [doc] engine-table: sort by Disabled and Name 2021-12-20 23:57:06 +01:00
Markus Heiser 9349c71c54 [mod] themes/simple/img/searxng.svg -> src/brand/searxng.svg
* move `searx/static/themes/simple/img/searxng.svg` to `src/brand/searxng.svg`

* README.rst can use it without a reference to a theme.

* the simple theme can create `searx/static/themes/simple/img/searxng.png` using
  the svg2png task

Suggested-by: @dalf https://github.com/searxng/searxng/pull/561#issuecomment-981747902
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-29 19:53:28 +01:00
Markus Heiser 27bb55ab07 [brand] SearXNG - docs html_theme = "searxng"
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-18 17:33:48 +01:00
Markus Heiser 3e5057405a [mod] doc - html_logo = themes/simple/src/svg/searxng-wordmark.svg
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-18 17:29:25 +01:00
Markus Heiser 2cc2406859 [mod] doc - in sidebar, add links to the reST sources
Offer links to the reST sources (aka) html_show_sourcelink [1].  Links to the
reST source are sometimes very helpful, especially in our resT-Primer [2] :)

[1] https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_show_sourcelink
[2] https://searxng.github.io/searxng/dev/reST.html

Related-to: https://github.com/pallets/pallets-sphinx-themes/issues/32
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-18 17:27:49 +01:00
Markus Heiser dc1442a2d1 [mod] Tools to install and maintain NVM versions manager for Node.js
[1] https://github.com/nvm-sh/nvm

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-16 23:31:11 +01:00
Markus Heiser a9fc4885f2 [brand] SearXNG - bash env SEARXNG_URL
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser ecb7e73e03 [brand] docs - normalize project name to SearXNG
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-13 14:36:24 +02:00
Alexandre Flament 4833d85e2e [doc] update copyright notice and HTML title 2021-09-10 11:49:57 +02:00
Alexandre Flament 4b43775c91 version based on the git repository
This commit remove the need to update the brand for GIT_URL and GIT_BRANCH:
there are read from the git repository.

It is possible to call python -m searx.version freeze to freeze the current version.
Useful when the code is installed outside git (distro package, docker, etc...)
2021-07-30 14:40:09 +02:00
Markus Heiser 3e50e8de3e [mod] drop usage of the searx.brand namespace (python procs)
Added function searx.get_setting(name, default=_unset):
  Returns the value to which ``name`` point.  If there is no such name in the
  settings and the ``default`` is unset, a KeyError exception is raised.

In all the python processes ..

- make docs
- make buildenv
- make install (setup.py)

the usage of the 'brand.*' name space is replaced by 'searx.get_setting'
function.

- brand.SEARX_URL        --> get_setting('server.base_url')
- brand.GIT_URL          --> get_setting('brand.git_url')
- brand.GIT_BRANCH'      --> get_setting('server.base_url')
- brand.ISSUE_URL        --> get_setting('brand.issue_url')
- brand.DOCS_URL         --> get_setting('brand.docs_url')
- brand.PUBLIC_INSTANCES --> get_setting('brand.public_instances')
- brand.CONTACT_URL      --> get_setting('general.contact_url', '')
- brand.WIKI_URL         --> get_setting('brand.wiki_url')
- brand.TWITTER_URL      --> get_setting('brand.twitter_url', '')

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-18 15:55:42 +02:00
Markus Heiser 97d801e8a0 [fix] docs/conf.py - docutils docs moved to docutils.sourceforge.io
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00
Alexandre Flament dddf3e2be2 [mod] "make docs" doesn't initialize the engines
avoid external HTTP requests with "make docs"
2021-06-04 11:26:37 +02:00
Markus Heiser 15de8708b7 [docs] highlight source code linked from *automodule* content
sphinx.ext.viewcode won't highlight when 'highlight_language' [1] is set to
string 'none' [2]

[1] https://www.sphinx-doc.org/en/master/usage/extensions/viewcode.html
[2] https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-highlight_language

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-01 16:37:20 +02:00
Markus Heiser 28b25185c5 [brand] searxng -- fix links to issue tracker & WEB-GUI
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-25 14:25:08 +02:00
Markus Heiser d0e371f474 [mod] replace makefile boilerplate by 'manage' script
Replaces the make targets with the bash scripts

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-23 11:26:22 +02:00
Markus Heiser 9485179064 [mod] move brand options from Makefile to settings.yml
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-11 22:12:38 +01:00
Alexandre Flament eb1b5289d5 [mod] documentation: change the jinja context doesn't depend on searx.webapp
Before this commit, in the documentation, the jinja context is 'webapp' and contains
the global variable in the searx.webapp module.

This commit changes this to include only the mandatory variables to build the
documentation.
2020-12-27 10:00:35 +01:00
Alexandre Flament 38d32337e7 [mod] documentations & comments: update http://* URL to https://*.
About http://lesscss.org see https://github.com/less/less-docs/issues/520
2020-12-04 16:52:25 +01:00
rachmadani haryono c03e4c86bc
Feature/standalone searx update (#1591)
* chg: dev: update standalone_searx

parent d8a5df721b33dd8a7cc9e21dba4060f21d629f69
author rachmadaniHaryono <foreturiga@gmail.com> 1603896594 +0800
committer rachmadaniHaryono <foreturiga@gmail.com> 1603896619 +0800

chg: dev: debug engine_shortcuts
chg: dev: only initilize if engine is given
chg: dev: split main
chg: dev: standalone_searx
chg: dev: update standalone_searx
chg: doc: remove unnecessary log
chg: test: differentiate travis
chg: test: disable shortcut
chg: test: use default engine settings
fix: dev: category choices
fix: dev: duplicate engine shortcut
fix: dev: travis python3
fix: test:  use empty string as shortcut
fix: test: apkm
fix: test: engine shortcut
fix: test: mypy
fix: test: parameter
fix: test: pep8
fix: test: py2 compatibilities
fix: test: searx settings
fix: test: travis engines
new: dev: deduplicate engine
new: dev: main receive engines parameter
new: dev: parse_argument accept engines parameter
new: dev: split search query from get_result func
new: test: basic result case
Suggestions: use RawTextQuery to make the suggestions URLs. Update all themes accordingly.

* new: doc: searx import and init

* chg: dev: parse_argument

- doc
- run on __main__
- simple parse_args

* chg: doc: module

* chg: dev: import section

- remove unused python path modification
- new required package

* chg: dev: script run

- parse_argument func return directly parsed results
- main func return dict instead json text
- dump directly on sys.stdout.write

* chg: dev: get_search_query and get_search_query func

* chg: dev: main func

- move inner function outside
- return dict instead of json text

* new: dev: add utils to doc sys path

* new: doc: standalone_searx

* fix: doc: run script

* chg: dev: mypy type hint

* chg: dev: SearchQuery don't have attr engines

* chg: dev: reset engines __init__

* chg: test: unit test update

* chg: dev: pylint and flake8

* new: test: standalone_searx

* chg: dev: main func and doc

* chg: dev: import and type hint

* new: dev: main func

- remove get_result func
- single func which just translate dict

* chg: test: put mypy on dev requirement

* chg: doc: update

* new: doc: add standalone_searx module member

* chg: doc: shell command line

* chg: dev: remove mypy

* chg: doc: module docstring
2020-11-04 12:38:54 +00:00
Alexandre Flament 2cafc5462d [fix] revert PR #2232 and #2230 2020-10-01 16:38:54 +02:00
Alexandre Flament 9a856083df [fix] Sphinx 3.2.1 build: call searx.engines.initialize_engines
Fix admin/engines.html
Fix #2230
2020-09-30 10:20:12 +02:00
Alexandre Flament 6c6985aaf8
Merge pull request #2230 from dalf/fix-sphinx-build
[fix] Sphinx 3.2.1 build: add missing "engines" variable in the jinja context
2020-09-30 09:29:09 +02:00
Alexandre Flament 6d56c754a7 [fix] Sphinx 3.2.1 build: add missing "engines" variable in the jinja context
Fix the admin/engines.html page
2020-09-29 12:30:10 +02:00
Alexandre Flament f204e4903d [fix] migration from github.com/asciimoo/searx to github.com/searx/searx : fix URLs 2020-09-28 16:44:14 +02:00
Markus Heiser aa5f947157 [enh] add 'kernel-include' reST-directive
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-06-18 18:51:31 +02:00
Markus Heiser 0742c2504d build environment: add GIT_BRANCH variable
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-29 16:28:17 +02:00
Markus Heiser 7b4cf2eb48 tooling box: simplify build enviroments
- no more need for a .config.mk
- docs: use searx.brands environment
- searx.sh, filtron.sh & morty.sh are sourcing utils/brand.env

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-29 15:09:34 +02:00
Markus Heiser 2d956696e9 docs: building (PDF) books / build user book
BTW: cleaned up Makefile target help

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-21 18:45:38 +01:00