Commit Graph

909 Commits

Author SHA1 Message Date
Markus Heiser
21d0428cf2 [mod] brand - partial migration of settings to msgspec.Struct (#5280)
The settings are currently an untyped key/value structure, whose types are
dynamically built at runtime.  The construction process of this structure
is *hand-crafted*.

In the long term, we want a static typing of this structure, based on a standard
tool.  The ``msgspec.Struct`` structures are suitable as a standard tool.

This patch makes a first step towards static typing and implements the "brand"
section using ``msgspec.Struct`` structures.

BTW: searx/settings_defaults.py - ``git_url`` and ``git_branch`` had been
removed in aee613d256, this is a leftover.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-10 16:14:29 +02:00
Markus Heiser
c6f1ea12b1 [fix] engine - cppreference has no longer a search function (#5273)
cppreference has replaced its search (``mwiki/index.php?title=``) with a DDG
search.

The engine was first introduced in SearXNG with PR-3274 [1], and even back then
the mediawiki proved to be incompatible, which is why the API could not be used
at the time. Now there isn't even a dedicated search function anymore.. I think
the cppreference project suffers from a lack of maintenance.

[1] https://github.com/searxng/searxng/pull/3247

Closes: https://github.com/searxng/searxng/issues/5271

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-03 08:40:24 +02:00
Markus Heiser
1520a8d545 [mod] ADS engine: revision of the engine (Paper result)
Revision of the Astrophysics Data System (ADS) engine / use of the result type
Paper as well as other typifications.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20 10:56:46 +02:00
Markus Heiser
f8f7adce6b [mod] Z-Library engine: revision of the engine (Paper result)
Revision of the engine / use of the result type Paper as well as other
typifications.

The engine has been placed on inactive because no service is currently
available, or at least not known in the SearXNG community [1]

[1] https://github.com/searxng/searxng/issues/3610

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20 10:56:46 +02:00
Markus Heiser
4c42704c80 [mod] Springer Nature engine: revision of the engine (Paper result)
Revision of the engine / use of the result type Paper as well as other
typifications.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20 10:56:46 +02:00
Markus Heiser
4b4bf0ecaf [mod] Semantic Scholar engine: revision of the engine (Paper result)
Revision of the engine / use of the result type Paper as well as other
typifications.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20 10:56:46 +02:00
Markus Heiser
bb22bb1831 [mod] PubMed engine: revision of the engine (Paper result)
Revision of the engine / use of the result type Paper as well as other
typifications.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20 10:56:46 +02:00
Markus Heiser
96e63df8ca [mod] Open Library engine: revision of the engine (Paper result)
Revision of the engine / use of the result type Paper as well as other
typifications.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20 10:56:46 +02:00
Markus Heiser
3ec6d65f9b [mod] CORE engine: revision of the engine (Paper result)
Revision of the engine / use of the result type Paper as well as other
typifications.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20 10:56:46 +02:00
Markus Heiser
22e73727c0 [mod] Anna's Archive engine: revision of the engine (Paper result)
Revision of the engine / use of the result type Paper as well as other
typifications.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20 10:56:46 +02:00
Markus Heiser
6c3fb9e42b [mod] arXiv engine: revision of the engine (Paper result)
Revision of the engine / use of the result type Paper as well as other
typifications.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-20 10:56:46 +02:00
Markus Heiser
8f8343dc0d [mod] addition of various type hints / engine processors
Continuation of #5147 .. typification of the engine processors.

BTW:

- removed obsolete engine property https_support
- fixed & improved currency_convert
- engine instances can now implement a engine.setup method

[#5147] https://github.com/searxng/searxng/pull/5147

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-18 19:40:03 +02:00
Markus Heiser
a9b088d832 [feat] engines yacy & piped: enable individual configuration of URLs (#5195)
With this change it is possible with individual engines (yacy & piped)
to configure individual URLs.

Related:

- https://github.com/searxng/searxng/issues/4869#issuecomment-327335928
- https://github.com/searxng/searxng/pull/3472/files#r1595586019
- https://github.com/searxng/searxng/issues/3428#issuecomment-2102142530

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-10 12:57:36 +02:00
Brock Vojkovic
314924bc0e [upd] disable public domain image archive engine by default 2025-09-09 16:58:27 +08:00
Bnyro
b93cc2f9f8 [feat] engines: add repology.org engine for linux packages (#5103)
Repology_ monitors a huge number of package repositories and other sources
comparing packages versions across them and gathering other information.

Repology_ shows you in which repositories a given project is packaged, which
version is the latest and which needs updating, who maintains the package, and
other related information.

.. _Repology: https://repology.org/docs/about

Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Format: reST
2025-09-01 16:33:31 +02:00
Markus Heiser
501327ddf1 [fix] engine: searchcode.com is offline (inactive) (#5170)
Searchcode.com is offline, and its future is still uncertain [1], so the engine
will be deactivated for the time being. If Searchcode.com doesn't come back
online soon, we can remove the engine entirely.

[1] https://boyter.org/posts/searchcode-is-being-rebooted/
[2] https://github.com/searxng/searxng/pull/5131#issuecomment-3239156555


Reported-by: Bnyro <bnyro@tutanota.com> [2]

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-09-01 15:35:55 +02:00
Bnyro
5ca08c1813 [feat] plugins: add new time/timezone search plugin
The plugin uses the ``GeoLocation`` class, which is already implemented in the
context of weather forecasts, to determine the time zone. The ``DateTime`` class
is used for the localized display of date and time.

Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2025-08-28 10:49:50 +02:00
Markus Heiser
92463ce6a7 [doc] adds the missing documentation of the server.method settings.
TL;DR; For all the issues that comes with HTTP POST I recommend instance
maintainers to switch to GET and lock the property in the preferences:

```yaml
server:
  method: GET

preferences:
  lock:
    - method
```

We don't want this in the defaults of the SearXNG distributions for the pros vs
cons listed in this discussion:

- https://github.com/searxng/searxng/pull/3619
2025-08-26 08:20:56 +02:00
muthukumaran R
a0ff173799 [feat] engines: add OpenAlex Works engine (#5102)
- Adds a new engine `searx/engines/openalex.py` that integrates the OpenAlex
  Works API to return scientific paper results using the `paper.html` template.
- Uses the official API (no auth required); supports OpenAlex polite pool via `mailto`.
- Adds developer docs at `docs/dev/engines/online/openalex.rst`.

OpenAlex API reference: https://docs.openalex.org/how-to-use-the-api/api-overview
2025-08-24 14:17:30 +02:00
Bnyro
0369682690 [fix] selfhst icons: icon list url invalid, set to active
- the previous CDN icon list url no longer works
- a list of all icons is mirrored to the JSDelivr CDN however
- there's no reason to set the engine to inactive now that we use public CDNs
2025-08-20 14:27:17 +02:00
Filip Mikina
6b57705e50 [feat] engines: add GitHub Code Search engine (#5074)
This patch adds GitHub Code Search [1] engine to allow querying the codebases.

Template code.html is changed to allow passthrough of strip and highlighting
options.

Engine Searchcode is adjusted to pass filename and not rely on hardcoded
extensions.

GitHub search code API does not return the exact code line indices, this
implementation assigns the code arbitrary numbers starting from 1
(effectively relabeling the code).

The API allows for unauth calls, and the default engine settings default to
that, although the calls are heavily rate limited.

The 'text' lexer is the default pygments lexer when parsing fails.

[1] https://docs.github.com/en/rest/search/search?apiVersion=2022-11-28#search-code

Co-authored-by: Markus Heiser <markus.heiser@darmarIT.de>
2025-08-20 07:35:31 +02:00
Austin-Olacsi
cf5061dc70 [feat] engines: add Marginalia (#5087)
To get an API key follow instructions at [1].

[1] https://about.marginalia-search.com/article/api/

Related (historical ordered):
- https://github.com/searxng/searxng/issues/1620
- https://github.com/searxng/searxng/issues/1673
- https://github.com/searxng/searxng/pull/1627
- https://github.com/searxng/searxng/pull/2489

Closes:
- https://github.com/searxng/searxng/issues/3034

Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2025-08-09 08:38:11 +02:00
Markus Heiser
2e62eb5d68 [fix] engine yummly: website were taken offline in December 2024 (#5080)
The app and website were taken offline in December 2024, with the latter
pointing to KitchenAid's US website. [1]

[1] https://en.wikipedia.org/wiki/Yummly

Closes: https://github.com/searxng/searxng/issues/5079
2025-08-03 10:49:14 +02:00
Fjara
f04c273732 [fix] correct comment in settings.yml for value to disable scheduling (#5052)
settings.yml correct value to disable scheduling
2025-07-27 17:36:39 +02:00
Markus Heiser
649a8dd577 [fix] cleanup: rename searx leftovers to SearXNG (#5049)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-07-26 06:22:46 +02:00
Bnyro
6b16a04e7e [mod] wordnik: convert to answerer (#4980)
Wordnik is now an answerer and not in the infobox anymore: it uses the
translations answerer, because it provides all the features needed. By default,
only its first results is shown

Additionally a new "define" category is added - I know, it's the same as the
"dictionaries" category, but I don't think we can alias categories.  This allows
to search e.g. for `!define tree`, the idea is to allow easy searches for
definitions of words.

Related:

- https://github.com/searxng/searxng/issues/4111
2025-07-24 07:42:31 +02:00
Markus Heiser
574b285efa [mod] remove option ui.static_use_hash (cache busting) (#5004)
Cache busting has caused serious problems for users in the past, here are two
examples:

- https://github.com/searxng/searxng/issues/4419
- https://github.com/searxng/searxng/issues/4481

And it makes development and deployment significantly more complex because it
binds the client side to the server side:

- https://github.com/searxng/searxng/pull/4466

In the light of a decoupled development of the WEB clients from the server side:

- https://github.com/searxng/searxng/pull/4988

is it appropriate to abandon this feature. In fact,  it has been ineffective
since #4436 anyway.

However, the benefit has always been questionable, since at best only a few kB
of data are saved (at least in the context of an image_proxy, the effect is below
the detection limit). Ultimately, the client is responsible for caching.

Related: https://github.com/searxng/searxng/issues?q=label%3A%22clear%20browser%20cache%22

Closes: https://github.com/searxng/searxng/pull/4466
Closes: https://github.com/searxng/searxng/issues/1326
Closes: https://github.com/searxng/searxng/issues/964

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-07-11 16:53:36 +02:00
Gaspard d'Hautefeuille
f798ddd492 [mod] migrate from Redis to Valkey (#4795)
This patch migrates from `redis==5.2.1` [1] to `valkey==6.1.0` [2].

The migration to valkey is necessary because the company behind Redis has decided
to abandon the open source license. After experiencing a drop in user numbers,
they now want to run it under a dual license again. But this move demonstrates
once again how unreliable the company is and how it treats open source
developers.

To review first, read the docs::

    $ make docs.live

Follow the instructions to remove redis:

- http://0.0.0.0:8000/admin/settings/settings_redis.html

Config and install a local valkey DB:

- http://0.0.0.0:8000/admin/settings/settings_valkey.html

[1] https://pypi.org/project/redis/
[2] https://pypi.org/project/valkey/

Co-authored-by: HLFH <gaspard@dhautefeuille.eu>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2025-07-09 07:55:37 +02:00
Bnyro
6ff4035635 [feat] engines: add pixabay for royalty free images/videos
What's changed?
- this PR adds Pixabay, a collection of royalty free images
- additionaly it seems to have some videos, so there's an engine for it too

Author Notes
- when using SearXNG's transport, all our requests will get blocked, probably due to fingerprinting
- we should find an alternative solution because this is just a hacky change to make things work for now, but idk how ...
2025-07-05 13:45:19 +02:00
useralias
8757655644 [fix] restore startpage autocompleter
Changes:
- Undo deletions of the autocompleter in settings and logic
- Add fixed autocomplete function in autocomplete.py

Related:
- https://github.com/searxng/searxng/issues/4334
2025-06-25 17:38:31 +02:00
Markus Heiser
58c10f758b [mod] plugins: activate tracker url plugin by default
With the new implementation for removing tracker arguments from URLs, we use the
tracker definitions of the ClearURLs project [1].  These definitions are
regularly updated and significantly improve data protection, which is why we
should now also enable this plugin by default.

[1] https://github.com/ClearURLs

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-06-23 22:12:18 +02:00
Robert M. Clabough
ec892d3836 [feat] engines: add Tube Archivist engine (#4889)
Tube Archivist [1] is a self-hosted project which archives youtube videos on
your own local server.  This engine connects with Tube Archivist's search API to
allow searching from SearXNG into your own hosted videos.

[1] https://www.tubearchivist.com/

Signed-off-by:  Robert M. Clabough <robert@claobugh.tech>
Co-authored-by: Bnyro <bnyro@tutanota.com>
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2025-06-22 09:40:42 +02:00
Nanashi
64f00bf5fd [fix] remove dissem.in due to its closure (#4930)
remove dissem.in due to its closure [1]

[1] https://association.dissem.in/dissemin-closure.html
2025-06-22 08:52:46 +02:00
Brock Vojkovic
8888d71ab9 [mod] disable wallhaven by default 2025-06-10 15:57:39 +00:00
benpiano800
49a1f08a17 [feat] engines: add Minecraft Wiki (#4896) 2025-06-06 17:05:30 +02:00
Bnyro
20b40351b9 [chore] il post: disable by default 2025-05-26 17:39:09 +02:00
Zhijie He
156d1eb8c8 [feat] engines: add Naver engine (#4573)
Refactor Naver engine (Web, News, Images, Videos, Autocomplete)

- ref: https://search.naver.com/
- lang: `ko`
- Wikidata: https://www.wikidata.org/wiki/Q485639

Co-authored-by: Bnyro <bnyro@tutanota.com>
2025-05-21 18:25:02 +02:00
Markus Heiser
365b9426f1 [fix] engines: disable those with known issues (#4813)
- z-library https://github.com/searxng/searxng/issues/3610
- library of congress: https://github.com/searxng/searxng/issues/4810
- qwant: https://github.com/searxng/searxng/issues/3929

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-05-21 15:50:29 +02:00
Bnyro
88973f5431 [feat] engines: add uxwing engine for icons (#4819)
- uxwing provides attribution-free icons to use for design projects
- svgrepo was my go-to before, but it's ratelimiting a lot recently
2025-05-21 15:10:29 +02:00
Bnyro
8bff73c9b6 [refactor] icon engines: add new icon category (#4817)
Icons category makes sense because it allows to quickly search for free SVG
icons to use for websites / other designs with a quick `!icons` query

Icons don't seem to fit into the normal images category that well because icons
are quite a special type of images
2025-05-21 14:52:16 +02:00
Markus Heiser
b8b857d24c [mod] engine invidious: commented out / no public API available nowadays (#4800)
Reported-by: @unifox https://github.com/searxng/searxng/issues/2722#issuecomment-2884993248

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-05-18 13:20:27 +02:00
Ivan Gabaldon
2cfd3fc44b [enh] tidy: clean old morty, filtron, searx references
Everyone should have already switched from legacy methods
2025-05-13 10:37:02 +02:00
BrandonStudio
d47cf9db24 [feat] engine ChinaSo: support source filter for ChinaSo-News
* filtering ChinaSo-News results by source, option ``chinaso_news_source``
* add ChinaSo engine to the online docs https://docs.searxng.org/dev/engines/online/chinaso.html
* fix SearXNG categories in the settings.yml
* deactivate ChinaSo engines ``inactive: true`` until [1] is fixed
* configure network of the ChinaSo engines

[1] https://github.com/searxng/searxng/issues/4694

Signed-off-by: @BrandonStudio
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2025-05-02 14:22:51 +02:00
Markus Heiser
d19eb3903e [clean] drop unusable engine: server/project on curlie.org is broken
The websites of https://curlie.org are no longer usable, long runtimes and
recurring "Bad Gateway" messages .. the project is no longer maintained and is
therefore no longer useful in SearXNG.

Related: https://github.com/searxng/searxng/issues/1190
Closes: https://github.com/searxng/searxng/issues/3482

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-04-24 09:57:50 +02:00
Grant Lanham
851c0e5cc0 [fix] engine: re-implement mullvad leta integration
Re-writes the Mullvad Leta integration to work with the new breaking changes.

Mullvad Leta is a search engine proxy.  Currently Leta only offers text search
results not image, news or any other types of search result.  Leta acts as a
proxy to Google and Brave search results.

- Remove docstring comments regarding requiring the use of Mullvad VPN, which is
  no longer a hard requirement.

- configured two engines: ``mullvadleta`` (uses google) and
  ``mullvadleta brave`` (uses brave)

- since leta may not provide up-to-date search results, both search engines are
  disabled by default.

.. hint::

   Leta caches each search for up to 30 days.  For example, if you use search
   terms like ``news``, contrary to your intention you'll get very old results!

Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
Signed-off-by: Grant Lanham <contact@grantlanham.com>
2025-04-23 14:06:32 +02:00
Aadniz
07a94d4d2e [mod] include SEARXNG_METHOD environment variable 2025-04-23 07:29:26 +02:00
Zhijie He
808dcaf1e2 [feat] engine: add Steam engine 2025-04-18 09:30:17 +02:00
Markus Heiser
777ba6ddf8 [fix] disable engine ansa by default
Reported-by: https://github.com/searxng/searxng/pull/4575#issuecomment-2813007107
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-04-17 16:50:57 +02:00
Zhijie He
f94802f2d2 [feat] engines: add Hugging Face engine 2025-04-17 16:43:32 +02:00
Tommaso Colella
d1c584b961 [feat] engine: add engine for italian press agency ansa 2025-04-17 15:33:57 +02:00