jandieman/searxng
2
0
Fork 0
mirror of https://github.com/searxng/searxng.git synced 2026-09-23 06:36:12 +00:00
Code Issues Projects Releases Wiki Activity
9,775 Commits 5 Branches 0 Tags
e831fc2a1cad50c9979b5f6f680376410218188c
Commit Graph

9775 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Bnyro
e831fc2a1c [fix] categories: rename stock_images category to stock images (#6760) 
Category ``stock_images`` is referenced to as ``stock images`` everywhere else,
so SXNG didn't find any translation for ``stock_images``.
 2026-09-19 15:01:12 +02:00
WookieeOnCrack
c0014f015e [mod] brave - modernize response handling (#6742) 2026-09-19 13:04:37 +02:00
Bnyro
fe5bdd1bf2 [feat] magnific: solve captcha to bypass botblocking 
Magnific now employs a very simple to solve CAPTCHA before
all API requests (essentially summing up some numbers).

Python script for testing:
```python
import re
from lxml import html
import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:156.0) Gecko/20100101 Firefox/156.0',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
    'Accept-Language': 'en-US,en;q=0.9',
    'Sec-GPC': '1',
    'Connection': 'keep-alive',
    'Upgrade-Insecure-Requests': '1',
    'Sec-Fetch-Dest': 'document',
    'Sec-Fetch-Mode': 'navigate',
    'Sec-Fetch-Site': 'same-origin',
    'Priority': 'u=0, i',
    'Pragma': 'no-cache',
    'Cache-Control': 'no-cache',
}

resp = requests.get(
    "https://www.magnific.com/api/regular/search?term=tree&filters%5Bai-generated%5D%5Bexcluded%5D=1&page=1&locale=en&filters%5Blicense%5D=free",
    headers=headers,
)
cookies = resp.cookies
print(resp.text)

_NUMBER_RE = re.compile("\d+")
_BM_VERIFY = re.compile(r"\"bm-verify\":\s*\"(.*?)\"")

doc = html.fromstring(resp.text)
script = doc.xpath("//script")[0].text
numbers = [int(m) for m in _NUMBER_RE.findall(script)]
bm_verify = _BM_VERIFY.search(resp.text)
if not bm_verify:
    exit(1)
bm_verify = bm_verify.group(1)
print("bm-verify: " + bm_verify)

solution = sum(numbers)
print(f"sum({numbers}) = {solution}")

resp = requests.post(
    "https://www.magnific.com/_sec/verify?provider=interstitial",
    json={"bm-verify": bm_verify, "pow": solution},
    headers=headers,
    cookies=cookies,
)

resp = requests.get(
    "https://www.magnific.com/api/regular/search?term=bird&filters%5Bai-generated%5D%5Bexcluded%5D=1&page=1&locale=en&filters%5Blicense%5D=free",
    headers=headers,
    cookies=resp.cookies,
)
print(resp.text)
```
 2026-09-19 11:31:00 +02:00
Brock Vojkovic
367fb6537c [fix] engines: tusksearch cors request headers (#6758) 
Added the ``Origin`` and ``Sec-Fetch-Site`` http headers to bypass tusksearch's latest bot detection.
 2026-09-19 07:30:06 +02:00
dependabot[bot]
c0042add30 [upd] web-client (simple): Bump the minor group in /client/simple with 3 updates (#6750) 
* [upd] web-client (simple): Bump the minor group

Bumps the minor group in /client/simple with 3 updates: [@biomejs/biome](https://github.com/biomejs/biome/tree/HEAD/packages/@biomejs/biome), [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) and [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).

Updates `@biomejs/biome` from 2.5.12 to 2.5.13
- [Release notes](https://github.com/biomejs/biome/releases)
- [Changelog](https://github.com/biomejs/biome/blob/main/packages/@biomejs/biome/CHANGELOG.md)
- [Commits](https://github.com/biomejs/biome/commits/@biomejs/biome@2.5.13/packages/@biomejs/biome)

Updates `@types/node` from 26.5.0 to 26.5.1
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Updates `vite` from 8.2.2 to 8.3.0
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/create-vite@8.3.0/packages/vite)

---
updated-dependencies:
- dependency-name: "@biomejs/biome"
  dependency-version: 2.5.13
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
- dependency-name: "@types/node"
  dependency-version: 26.5.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
- dependency-name: vite
  dependency-version: 8.3.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [upd] sync lockfile

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ivan Gabaldon <igabaldon@inetol.net>
 2026-09-18 17:23:06 +02:00
Bnyro
c0ec29fdc3 [fix] pixabay: crashes when there are no results 2026-09-18 17:18:59 +02:00
dependabot[bot]
579809a053 [upd] github-actions: Bump docker/setup-qemu-action from 4.3.0 to 4.4.0 (#6747) 
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 4.3.0 to 4.4.0.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](1f40c72289...9901266195)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: 4.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-18 17:10:17 +02:00
dependabot[bot]
9cde918dd2 [upd] github-actions: Bump actions/download-artifact from 4.3.0 to 8.0.1 (#6748) 
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.3.0 to 8.0.1.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](d3f86a106a...3e5f45b2cf)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: 8.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-18 17:08:07 +02:00
dependabot[bot]
63b827edc7 [upd] github-actions: Bump actions/upload-artifact from 4.6.2 to 7.0.1 (#6749) 
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.6.2 to 7.0.1.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](ea165f8d65...043fb46d1a)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-18 17:07:54 +02:00
Bnyro
74ffd01152 [feat] engines: add littlelayer (general / indie) engine 2026-09-18 16:45:36 +02:00
Markus Heiser
280aceb1ae [mod] add new category 'stock_images' - a specialization of 'images' (#6703) 
Engines like Pexels provide images that one would more likely expect in a
category named ``stock_images``.

By classifying engines like Pexels more precisely, we enable a more specific
handling of them, which also helps to avoid mixing them with the more general
image search on the internet. [1]

We leave these engines *enabled* by default, take them out of ``images`` and
move them into the specialization ``stock_images``.

This makes it possible, on the one hand, for the admin to configure a tab in the
UI, and on the other hand, the user can always select the group directly using
the search syntax ``!stock_images ...``.

What we need to keep in mind:

1. Technically speaking, there are no subcategories in the strict
   sense.  Organizationally, the *subcategories* are derived from the
   `categories_as_tabs`.

2. From an admin’s perspective, `categories_as_tabs` is the tool for
   structuring their content.

3. In the context of “enabled/disabled by default”: We should avoid having
   the admin have to activate many individual engines when they want to
   structure their content (UI tabs).

So if we leave an engine enabled by default and **move** it from `images` to a
new category, the admin only needs to add the new category to
`categories_as_tabs` to restructure their content.  If the new category is also
part of the translations (see `searx/searxng.msg` in this PR), this UI structure
would also be available internationally.

[1] https://github.com/searxng/searxng/pull/6690#issuecomment-5646291679

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-09-18 16:13:42 +02:00
Bnyro
1f7711ef1f [mod] categories: move app search engines to it category instead of files 2026-09-18 13:19:28 +02:00
Bnyro
12df7aed60 [del] library of congress: now behind cloudflare (#6756) 2026-09-18 12:27:30 +02:00
Bnyro
96c35f1ed3 [fix] findfiles images: crashes due to changed HTML layout 2026-09-18 11:47:45 +02:00
dependabot[bot]
0d6910ae5a [upd] pypi: Bump the minor group with 3 updates (#6751) 
Bumps the minor group with 3 updates: [granian](https://github.com/emmett-framework/granian), [selenium](https://github.com/SeleniumHQ/Selenium) and [basedpyright](https://github.com/detachhead/basedpyright).


Updates `granian` from 2.8.2 to 2.8.3
- [Release notes](https://github.com/emmett-framework/granian/releases)
- [Commits](https://github.com/emmett-framework/granian/compare/v2.8.2...v2.8.3)

Updates `selenium` from 4.48.0 to 4.49.0
- [Release notes](https://github.com/SeleniumHQ/Selenium/releases)
- [Commits](https://github.com/SeleniumHQ/Selenium/compare/selenium-4.48.0...selenium-4.49.0)

Updates `basedpyright` from 1.40.0 to 1.40.1
- [Release notes](https://github.com/detachhead/basedpyright/releases)
- [Commits](https://github.com/detachhead/basedpyright/compare/v1.40.0...v1.40.1)
 2026-09-18 09:32:40 +02:00
Bnyro
087ec6fbda [fix] 500px: crashes when there's no description (#6745) 2026-09-18 07:42:08 +02:00
Bnyro
fc028154a4 [fix] giphy: API key extraction fails (#6744) 2026-09-18 07:41:40 +02:00
Bnyro
274b63b677 [refactor] engines: migrate to SXNG_Response.html() (#6718) 2026-09-17 12:22:38 +02:00
Bnyro
3169aafd28 [feat] SXNG_Response: add method to parse html body (#6718) 2026-09-17 12:22:38 +02:00
Bnyro
c497719921 [feat] yandex web: add time range support 2026-09-17 09:46:21 +02:00
Bnyro
e57ebb0aa5 [refactor] engine processors: add type for time range and safesearch in OnlineParams 2026-09-17 09:46:21 +02:00
Bnyro
77d07c8743 [mod] yandex: simplify request method and drop unused params/cookies 
We previously set the 'cookie' cookie value with a timestamp
from 2024 to `yq=...`. The cookie is not actually used in Yandex,
instead we probably wanted to set the `yq` cookie instead. But
since the engine works without it, it can be dropped.

Also simplifies the request method a bit to be easier to read.
 2026-09-17 09:46:21 +02:00
Brock Vojkovic
9aab16aef7 [fix] network: stop http redirects (#6740) 
Stop curl from following from a HTTPS origin to a HTTP page

See: https://curl.se/libcurl/c/CURLOPT_REDIR_PROTOCOLS_STR.html

Reported-by: Rohit Dixit <00.00.xit@gmail.com>
 2026-09-17 15:27:01 +08:00
Bnyro
3accbaca79 [feat] mojeek: automatically solve altcha captchas 2026-09-17 09:00:33 +02:00
Bnyro
9d82be9d89 [feat] utils.py: add altcha captcha solver 2026-09-17 09:00:33 +02:00
Markus Heiser
461f174b09 [mod] engine: pexels - added type hints and migrate to Image-Results (#6739) 
Code style somewhat unified, type annotations added, ``LegacyResult`` replaced
by ``Image``, but no functional modifications.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-09-16 16:46:16 +02:00
Markus Heiser
1354f3952b [feat] mediathekviewweb: add description to queried fields (#6717) 
Queries like `!mvw hercule poirot` usually yield no results if the `description`
is not also included in the search.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-09-16 16:35:47 +02:00
Bnyro
f9c9c35546 [fix] mediathekviewweb: crashes when result has no hd video url (#6717) 2026-09-16 16:35:47 +02:00
Bnyro
71a2424c24 [feat] mediathekviewweb: modernize engine to use EngineResults (#6717) 2026-09-16 16:35:47 +02:00
Brock Vojkovic
ce19e9fae3 [fix] image_proxy compressed images (#6730) 
Prevents the image proxy from decompressing brotli/gzip in transit.

See: https://curl.se/libcurl/c/CURLOPT_ACCEPT_ENCODING.html

Setting accept_encoding to none prevents curl from automatically decompressing the received contents.

Signed-off-by: vojkovic <git@vojk.au>
 2026-09-16 22:23:11 +08:00
WookieeOnCrack
f725cc7936 [fix] yandex: image result xpath + modernization (#6728) 2026-09-16 10:38:32 +02:00
vojkovic
ca49650407 [fix] engines: yahoo modernisation and ybv cookie 2026-09-15 17:37:19 +08:00
vojkovic
94218a3ac6 [build] /static 2026-09-15 09:56:24 +08:00
vojkovic
beb324c0f5 [mod] simple: apply nojs via stylesheet 2026-09-15 09:56:24 +08:00
vojkovic
604ee1698c [fix] engines: flaticon encode non ascii words 2026-09-15 08:50:46 +08:00
Ivan Gabaldon
ef05645f09 [mod] container: rework builds (#6720) 
Free from ghcr.io/searxng/cache! Now we save and load images without polluting
this shared (and useless) layer cache, which also makes it easy to download
artifacts for local debugging.

I also removed this nonsense about virtualenv normalization, this will be
properly fixed by using lockfiles on py dependencies.
 2026-09-14 16:06:00 +02:00
Ivan Gabaldon
d4ce87c234 [mod] py: format (#6560) 2026-09-13 23:48:01 +02:00
dependabot[bot]
ab5dd7599e [upd] pypi: Bump black from 25.9.0 to 26.5.1 (#6560) 
Bumps [black](https://github.com/psf/black) from 25.9.0 to 26.5.1.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/25.9.0...26.5.1)

---
updated-dependencies:
- dependency-name: black
  dependency-version: 26.5.1
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-09-13 23:48:01 +02:00
vojkovic
230c3632d9 [mod] github: convert issue templates to yaml forms 2026-09-13 22:37:48 +08:00
Bnyro
42829cecd6 [feat] results: limit titles and content to fixed length 2026-09-13 15:05:47 +02:00
Bnyro
3b80090654 [del] adobe stock: remove because it's behind a datadome captcha 2026-09-13 14:49:03 +02:00
Bnyro
e61d097562 [fix] 500px: migrate to new search endpoint 2026-09-13 14:48:44 +02:00
Bnyro
32f2da4ef0 [fix] public domain image archive: fails to extract API url 2026-09-13 13:43:01 +02:00
Brock Vojkovic
96479fff65 [fix] ci: run armv7 test on the amd64 runner (#6711) 2026-09-13 09:25:21 +02:00
dependabot[bot]
d4f00d15d4 [upd] pypi: Bump the minor group with 4 updates (#6693) 
Bumps the minor group with 4 updates: [lxml](https://github.com/lxml/lxml), [curl-cffi](https://github.com/lexiforest/curl_cffi), [wlc](https://github.com/WeblateOrg/wlc) and [basedpyright](https://github.com/detachhead/basedpyright).


Updates `lxml` from 6.1.2 to 6.1.3
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](https://github.com/lxml/lxml/compare/lxml-6.1.2...lxml-6.1.3)

Updates `curl-cffi` from 0.16.1 to 0.16.3
- [Release notes](https://github.com/lexiforest/curl_cffi/releases)
- [Changelog](https://github.com/lexiforest/curl_cffi/blob/main/docs/changelog.rst)
- [Commits](https://github.com/lexiforest/curl_cffi/compare/v0.16.1...v0.16.3)

Updates `wlc` from 2.1.1 to 2.2.0
- [Release notes](https://github.com/WeblateOrg/wlc/releases)
- [Changelog](https://github.com/WeblateOrg/wlc/blob/main/CHANGES.md)
- [Commits](https://github.com/WeblateOrg/wlc/compare/2.1.1...2.2.0)

Updates `basedpyright` from 1.39.10 to 1.40.0
- [Release notes](https://github.com/detachhead/basedpyright/releases)
- [Commits](https://github.com/detachhead/basedpyright/compare/v1.39.10...v1.40.0)
 2026-09-12 18:20:51 +02:00
Markus Heiser
a667e8c905 [mod] openverse - use of internal APIs and addition result fields (#6701) 
Additional fields:
  add resolution, thumbnail and author to the image result

Types:
  use EngineResults and add type hints from internall APIs

Disabled engine:
  as of 09/2026: The availability of https://openverse.org/ is poor, and queries
  and image requests often time out. However, these issues also occur when using
  a WEB browser to search on openverse.org or view images.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-09-12 18:18:32 +02:00
vojkovic
0a94da4368 [fix] engines: naver images updated matcher 2026-09-12 18:17:41 +02:00
Markus Heiser
56b1f64541 [mod] engine: bing images - use internal API & type hints (#6699) 
Switching to internal APIs such as ``EngineResults`` and
adding more type hints.  No functional changes.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-09-12 11:22:01 +02:00
Markus Heiser
923a307454 [fix] engine: bing images - remove market place argument mkt (#6699) 
The ``mkt`` argument does no longer exists and the ``async`` argument has been
renamed to ``mmasync``.

For still unknown reasons (IP based?), some users had to observe that the title
was missing [1].  I myself was not able to reproduce this error, however the
evaluation of the title was additionally expanded by the attribute value of the
``<a title=".."`` element.

Related:

- [1] https://github.com/searxng/searxng/pull/6690#issuecomment-5636811572

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2026-09-12 11:22:01 +02:00
vojkovic
87bf8c86ed [fix] engines: dogpile web needs api token 2026-09-12 16:32:42 +08:00