Commit Graph

6279 Commits

Author SHA1 Message Date
Markus Heiser
ce48993997 [build] /static 2026-09-20 16:57:45 +02:00
Markus Heiser
e40de2d992 [mod] typification of SearXNG: add new result type Video (#6743)
- Python class:   searx/result_types/video.py
- Jinja template: searx/templates/simple/result_templates/videos.html
- CSS (less)      client/simple/src/less/result_types/video.less

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-09-20 16:57:45 +02:00
vojkovic
fdd8525b1a [build] /static 2026-09-20 18:58:14 +08:00
vojkovic
bcd2b709fe [mod] theme: improve thumbnails display on results page
Deletes thumbnails instead of showing an error image and also moves them to the right side. Also gives them a background and makes them into rounded 4:3 rectangles for better ui.
2026-09-20 18:58:14 +08:00
Brock Vojkovic
e75fd22d09 [mod] defaults: disable image engines with poor results (#6690)
The image search defaults are a bit of a mix of results atm with random stock photos, some broken thumbnails (artic) and svg icons.
2026-09-20 11:10:16 +08:00
Bnyro
e831fc2a1c [fix] categories: rename stock_images category to stock images (#6760)
Category ``stock_images`` is referenced to as ``stock images`` everywhere else,
so SXNG didn't find any translation for ``stock_images``.
2026-09-19 15:01:12 +02:00
WookieeOnCrack
c0014f015e [mod] brave - modernize response handling (#6742) 2026-09-19 13:04:37 +02:00
Bnyro
fe5bdd1bf2 [feat] magnific: solve captcha to bypass botblocking
Magnific now employs a very simple to solve CAPTCHA before
all API requests (essentially summing up some numbers).

Python script for testing:
```python
import re
from lxml import html
import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:156.0) Gecko/20100101 Firefox/156.0',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Accept-Language': 'en-US,en;q=0.9',
    'Sec-GPC': '1',
    'Connection': 'keep-alive',
    'Upgrade-Insecure-Requests': '1',
    'Sec-Fetch-Dest': 'document',
    'Sec-Fetch-Mode': 'navigate',
    'Sec-Fetch-Site': 'same-origin',
    'Priority': 'u=0, i',
    'Pragma': 'no-cache',
    'Cache-Control': 'no-cache',
}

resp = requests.get(
    "https://www.magnific.com/api/regular/search?term=tree&filters%5Bai-generated%5D%5Bexcluded%5D=1&page=1&locale=en&filters%5Blicense%5D=free",
    headers=headers,
)
cookies = resp.cookies
print(resp.text)

_NUMBER_RE = re.compile("\d+")
_BM_VERIFY = re.compile(r"\"bm-verify\":\s*\"(.*?)\"")

doc = html.fromstring(resp.text)
script = doc.xpath("//script")[0].text
numbers = [int(m) for m in _NUMBER_RE.findall(script)]
bm_verify = _BM_VERIFY.search(resp.text)
if not bm_verify:
    exit(1)
bm_verify = bm_verify.group(1)
print("bm-verify: " + bm_verify)

solution = sum(numbers)
print(f"sum({numbers}) = {solution}")

resp = requests.post(
    "https://www.magnific.com/_sec/verify?provider=interstitial",
    json={"bm-verify": bm_verify, "pow": solution},
    headers=headers,
    cookies=cookies,
)

resp = requests.get(
    "https://www.magnific.com/api/regular/search?term=bird&filters%5Bai-generated%5D%5Bexcluded%5D=1&page=1&locale=en&filters%5Blicense%5D=free",
    headers=headers,
    cookies=resp.cookies,
)
print(resp.text)
```
2026-09-19 11:31:00 +02:00
Brock Vojkovic
367fb6537c [fix] engines: tusksearch cors request headers (#6758)
Added the ``Origin`` and ``Sec-Fetch-Site`` http headers to bypass tusksearch's latest bot detection.
2026-09-19 07:30:06 +02:00
dependabot[bot]
c0042add30 [upd] web-client (simple): Bump the minor group in /client/simple with 3 updates (#6750)
* [upd] web-client (simple): Bump the minor group

Bumps the minor group in /client/simple with 3 updates: [@biomejs/biome](https://github.com/biomejs/biome/tree/HEAD/packages/@biomejs/biome), [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) and [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).

Updates `@biomejs/biome` from 2.5.12 to 2.5.13
- [Release notes](https://github.com/biomejs/biome/releases)
- [Changelog](https://github.com/biomejs/biome/blob/main/packages/@biomejs/biome/CHANGELOG.md)
- [Commits](https://github.com/biomejs/biome/commits/@biomejs/biome@2.5.13/packages/@biomejs/biome)

Updates `@types/node` from 26.5.0 to 26.5.1
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Updates `vite` from 8.2.2 to 8.3.0
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/create-vite@8.3.0/packages/vite)

---
updated-dependencies:
- dependency-name: "@biomejs/biome"
  dependency-version: 2.5.13
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
- dependency-name: "@types/node"
  dependency-version: 26.5.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
- dependency-name: vite
  dependency-version: 8.3.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [upd] sync lockfile

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ivan Gabaldon <igabaldon@inetol.net>
2026-09-18 17:23:06 +02:00
Bnyro
c0ec29fdc3 [fix] pixabay: crashes when there are no results 2026-09-18 17:18:59 +02:00
Bnyro
74ffd01152 [feat] engines: add littlelayer (general / indie) engine 2026-09-18 16:45:36 +02:00
Markus Heiser
280aceb1ae [mod] add new category 'stock_images' - a specialization of 'images' (#6703)
Engines like Pexels provide images that one would more likely expect in a
category named ``stock_images``.

By classifying engines like Pexels more precisely, we enable a more specific
handling of them, which also helps to avoid mixing them with the more general
image search on the internet. [1]

We leave these engines *enabled* by default, take them out of ``images`` and
move them into the specialization ``stock_images``.

This makes it possible, on the one hand, for the admin to configure a tab in the
UI, and on the other hand, the user can always select the group directly using
the search syntax ``!stock_images ...``.

What we need to keep in mind:

1. Technically speaking, there are no subcategories in the strict
   sense.  Organizationally, the *subcategories* are derived from the
   `categories_as_tabs`.

2. From an admin’s perspective, `categories_as_tabs` is the tool for
   structuring their content.

3. In the context of “enabled/disabled by default”: We should avoid having
   the admin have to activate many individual engines when they want to
   structure their content (UI tabs).

So if we leave an engine enabled by default and **move** it from `images` to a
new category, the admin only needs to add the new category to
`categories_as_tabs` to restructure their content.  If the new category is also
part of the translations (see `searx/searxng.msg` in this PR), this UI structure
would also be available internationally.

[1] https://github.com/searxng/searxng/pull/6690#issuecomment-5646291679

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-09-18 16:13:42 +02:00
Bnyro
1f7711ef1f [mod] categories: move app search engines to it category instead of files 2026-09-18 13:19:28 +02:00
Bnyro
12df7aed60 [del] library of congress: now behind cloudflare (#6756) 2026-09-18 12:27:30 +02:00
Bnyro
96c35f1ed3 [fix] findfiles images: crashes due to changed HTML layout 2026-09-18 11:47:45 +02:00
Bnyro
087ec6fbda [fix] 500px: crashes when there's no description (#6745) 2026-09-18 07:42:08 +02:00
Bnyro
fc028154a4 [fix] giphy: API key extraction fails (#6744) 2026-09-18 07:41:40 +02:00
Bnyro
274b63b677 [refactor] engines: migrate to SXNG_Response.html() (#6718) 2026-09-17 12:22:38 +02:00
Bnyro
3169aafd28 [feat] SXNG_Response: add method to parse html body (#6718) 2026-09-17 12:22:38 +02:00
Bnyro
c497719921 [feat] yandex web: add time range support 2026-09-17 09:46:21 +02:00
Bnyro
e57ebb0aa5 [refactor] engine processors: add type for time range and safesearch in OnlineParams 2026-09-17 09:46:21 +02:00
Bnyro
77d07c8743 [mod] yandex: simplify request method and drop unused params/cookies
We previously set the 'cookie' cookie value with a timestamp
from 2024 to `yq=...`. The cookie is not actually used in Yandex,
instead we probably wanted to set the `yq` cookie instead. But
since the engine works without it, it can be dropped.

Also simplifies the request method a bit to be easier to read.
2026-09-17 09:46:21 +02:00
Brock Vojkovic
9aab16aef7 [fix] network: stop http redirects (#6740)
Stop curl from following from a HTTPS origin to a HTTP page

See: https://curl.se/libcurl/c/CURLOPT_REDIR_PROTOCOLS_STR.html

Reported-by: Rohit Dixit <00.00.xit@gmail.com>
2026-09-17 15:27:01 +08:00
Bnyro
3accbaca79 [feat] mojeek: automatically solve altcha captchas 2026-09-17 09:00:33 +02:00
Bnyro
9d82be9d89 [feat] utils.py: add altcha captcha solver 2026-09-17 09:00:33 +02:00
Markus Heiser
461f174b09 [mod] engine: pexels - added type hints and migrate to Image-Results (#6739)
Code style somewhat unified, type annotations added, ``LegacyResult`` replaced
by ``Image``, but no functional modifications.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-09-16 16:46:16 +02:00
Markus Heiser
1354f3952b [feat] mediathekviewweb: add description to queried fields (#6717)
Queries like `!mvw hercule poirot` usually yield no results if the `description`
is not also included in the search.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-09-16 16:35:47 +02:00
Bnyro
f9c9c35546 [fix] mediathekviewweb: crashes when result has no hd video url (#6717) 2026-09-16 16:35:47 +02:00
Bnyro
71a2424c24 [feat] mediathekviewweb: modernize engine to use EngineResults (#6717) 2026-09-16 16:35:47 +02:00
Brock Vojkovic
ce19e9fae3 [fix] image_proxy compressed images (#6730)
Prevents the image proxy from decompressing brotli/gzip in transit.

See: https://curl.se/libcurl/c/CURLOPT_ACCEPT_ENCODING.html

Setting accept_encoding to none prevents curl from automatically decompressing the received contents.

Signed-off-by: vojkovic <git@vojk.au>
2026-09-16 22:23:11 +08:00
WookieeOnCrack
f725cc7936 [fix] yandex: image result xpath + modernization (#6728) 2026-09-16 10:38:32 +02:00
vojkovic
ca49650407 [fix] engines: yahoo modernisation and ybv cookie 2026-09-15 17:37:19 +08:00
vojkovic
94218a3ac6 [build] /static 2026-09-15 09:56:24 +08:00
vojkovic
beb324c0f5 [mod] simple: apply nojs via stylesheet 2026-09-15 09:56:24 +08:00
vojkovic
604ee1698c [fix] engines: flaticon encode non ascii words 2026-09-15 08:50:46 +08:00
Ivan Gabaldon
d4ce87c234 [mod] py: format (#6560) 2026-09-13 23:48:01 +02:00
Bnyro
42829cecd6 [feat] results: limit titles and content to fixed length 2026-09-13 15:05:47 +02:00
Bnyro
3b80090654 [del] adobe stock: remove because it's behind a datadome captcha 2026-09-13 14:49:03 +02:00
Bnyro
e61d097562 [fix] 500px: migrate to new search endpoint 2026-09-13 14:48:44 +02:00
Bnyro
32f2da4ef0 [fix] public domain image archive: fails to extract API url 2026-09-13 13:43:01 +02:00
Markus Heiser
a667e8c905 [mod] openverse - use of internal APIs and addition result fields (#6701)
Additional fields:
  add resolution, thumbnail and author to the image result

Types:
  use EngineResults and add type hints from internall APIs

Disabled engine:
  as of 09/2026: The availability of https://openverse.org/ is poor, and queries
  and image requests often time out. However, these issues also occur when using
  a WEB browser to search on openverse.org or view images.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-09-12 18:18:32 +02:00
vojkovic
0a94da4368 [fix] engines: naver images updated matcher 2026-09-12 18:17:41 +02:00
Markus Heiser
56b1f64541 [mod] engine: bing images - use internal API & type hints (#6699)
Switching to internal APIs such as ``EngineResults`` and
adding more type hints.  No functional changes.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-09-12 11:22:01 +02:00
Markus Heiser
923a307454 [fix] engine: bing images - remove market place argument mkt (#6699)
The ``mkt`` argument does no longer exists and the ``async`` argument has been
renamed to ``mmasync``.

For still unknown reasons (IP based?), some users had to observe that the title
was missing [1].  I myself was not able to reproduce this error, however the
evaluation of the title was additionally expanded by the attribute value of the
``<a title=".."`` element.

Related:

- [1] https://github.com/searxng/searxng/pull/6690#issuecomment-5636811572

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-09-12 11:22:01 +02:00
vojkovic
87bf8c86ed [fix] engines: dogpile web needs api token 2026-09-12 16:32:42 +08:00
Markus Heiser
6a27c21008 [fix] engine: pinterest - fix empty titles and complete modernization (#6694)
* [fix] engine: pinterest - fix empty titles and complete modernization

The titles from the fields ``title`` and ``grid_title`` are mostly empty or have
short strings without meaningful content.  Various fields for the title are now
being queried, which have more informative value.

As part of the bug fix, the engine was completely revised and modernized.

Related:

- https://github.com/searxng/searxng/pull/6690#issuecomment-5631113121

Co-authored-by: @vojkovic
2026-09-11 16:24:58 +02:00
Brock Vojkovic
ffe96f8a6f [fix] engines: bing first word results (#6671)
Fixes the bing web engine, it was just using the first word of the query for the search and return random junk other times. see: vojkovic#10

Swapped to use bing's setlang and cc params. I found us, cn, ru return complete garbage 100% of the time. I reckon that if you don't have an ip address from there it will just return garbage, so those three are skipped. Also removed accept language override because it didn't change anything anymore.


- Closes: https://github.com/searxng/searxng/issues/4964
- Related: https://github.com/vojkovic/searxng/issues/10
2026-09-11 08:41:13 +02:00
Markus Heiser
931fd9787b [fix] engine: core.ac.uk - don't split a string into a tag list (#6689) 2026-09-10 14:45:43 +02:00
vojkovic
42e1d61296 [fix] engines: startpage anubis solver 2026-09-10 19:12:21 +08:00