12 Commits

Author SHA1 Message Date
Bnyro
a1144dda3e [mod] engines: migrate video engines away from get_embedded_stream_url 2026-09-03 17:28:29 +02:00
Bnyro
86008c9dd6 [feat] results: automatically set embedded stream url for video results
Currently, many engines that provide video results call the same
`get_embedded_stream_url` method for setting the `iframe_src`. There
is no value in that because this logic is engine-specific and probably
many video engines forgot to implement this.

With these changes, it's done automatically, so engine implementors
don't have to worry about setting an `iframe_src` (unless the engine
explicitly has a field for it).
2026-09-03 17:28:29 +02:00
Bnyro
8f452ee892 [fix] chatnoir: fails to extract API session parameters 2026-09-03 10:03:05 +02:00
Markus Heiser
745d5b6fc5 [fix] engine: braveapi - braveapi pagination sends an invalid offset (#6627)
Closes: https://github.com/searxng/searxng/issues/6545
2026-09-03 07:52:31 +02:00
Bnyro
05cd77f71b [mod] s1search: update mirrors to the ones that are not cloudflared 2026-09-02 16:59:05 +02:00
Om Alve
b4f616fed7 [mod] engines: migrate to setup() from init() for simple tasks 2026-09-02 16:30:00 +02:00
Markus Heiser
fd29de6c55 [fix] engine: meilisearch - AttributeError: no attribute 'about' (#6626)
- Closes: https://github.com/searxng/searxng/issues/6622

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-09-02 14:26:22 +02:00
Bnyro
eca1af4be1 [del] heexy: remove engine due to cloudflare captcha (#6625) 2026-09-02 13:43:03 +02:00
Markus Heiser
7a01f5830c [fix] hardening engine: ddg weather - get GEO location from response (#6623)
The previous implementation ran into an error if the search term contained words
other than just the location (ValueError was raised).

To test engine use search terms like:

    !ddw weather berlin germany

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-09-02 13:40:53 +02:00
Ivan Gabaldon
18af21159b [mod] py: clearurls update location (#6618)
We should opt to retrieve the lists through jsDelivr service as our first choice
and avoid hitting the GH (GitHub) and GL (GitLab) servers.
2026-09-01 21:43:50 +02:00
dependabot[bot]
248e37991c [upd] web-client (simple): Bump the minor group (#6588)
Bumps the minor group in /client/simple with 4 updates: [@biomejs/biome](https://github.com/biomejs/biome/tree/HEAD/packages/@biomejs/biome), [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node), [svgo](https://github.com/svg/svgo) and [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite).


Updates `@biomejs/biome` from 2.5.9 to 2.5.10
- [Release notes](https://github.com/biomejs/biome/releases)
- [Changelog](https://github.com/biomejs/biome/blob/main/packages/@biomejs/biome/CHANGELOG.md)
- [Commits](https://github.com/biomejs/biome/commits/@biomejs/biome@2.5.10/packages/@biomejs/biome)

Updates `@types/node` from 26.2.0 to 26.3.0
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Updates `svgo` from 4.0.2 to 4.1.0
- [Release notes](https://github.com/svg/svgo/releases)
- [Commits](https://github.com/svg/svgo/compare/v4.0.2...v4.1.0)

Updates `vite` from 8.2.1 to 8.2.2
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.2.2/packages/vite)

---
updated-dependencies:
- dependency-name: "@biomejs/biome"
  dependency-version: 2.5.10
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
- dependency-name: "@types/node"
  dependency-version: 26.3.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: svgo
  dependency-version: 4.1.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor
- dependency-name: vite
  dependency-version: 8.2.2
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-01 21:42:44 +02:00
github-actions[bot]
ff424745fd [mod] data: update searx.data - update_external_bangs.py (#6593)
Co-authored-by: searxng-bot <searxng-bot@users.noreply.github.com>
2026-09-01 21:38:36 +02:00
60 changed files with 4109 additions and 4234 deletions

View File

@@ -15,8 +15,8 @@
"swiped-events": "1.2.0"
},
"devDependencies": {
"@biomejs/biome": "2.5.9",
"@types/node": "^26.2.0",
"@biomejs/biome": "2.5.10",
"@types/node": "^26.3.0",
"browserslist": "^4.28.8",
"browserslist-to-esbuild": "^2.1.1",
"edge.js": "^6.5.1",
@@ -27,9 +27,9 @@
"stylelint": "^17.14.1",
"stylelint-config-standard-less": "^4.1.0",
"stylelint-prettier": "^5.0.3",
"svgo": "^4.0.2",
"svgo": "^4.1.0",
"typescript": "~7.0.2",
"vite": "^8.2.1",
"vite": "^8.2.2",
"vite-bundle-analyzer": "^1.3.9"
}
},
@@ -69,9 +69,9 @@
}
},
"node_modules/@biomejs/biome": {
"version": "2.5.9",
"resolved": "https://registry.npmjs.org/@biomejs/biome/-/biome-2.5.9.tgz",
"integrity": "sha512-KkgCvdHB4IhtpHpF564plA9jo6fDOwWGQ/3jvreLzgOtRLEDoPqr7QO9qejNA8jKwDsSkAKr77hqBHnyUbIw4g==",
"version": "2.5.10",
"resolved": "https://registry.npmjs.org/@biomejs/biome/-/biome-2.5.10.tgz",
"integrity": "sha512-WRKXARA3kTuiV5sxqTpobJ/I0MVd4vk3pOL6wnp5az4LntFIhWTj1RWZq3DI9PCEN3lXcqy7p5aqUHzvq8AXyQ==",
"dev": true,
"license": "MIT OR Apache-2.0",
"bin": {
@@ -85,20 +85,20 @@
"url": "https://opencollective.com/biome"
},
"optionalDependencies": {
"@biomejs/cli-darwin-arm64": "2.5.9",
"@biomejs/cli-darwin-x64": "2.5.9",
"@biomejs/cli-linux-arm64": "2.5.9",
"@biomejs/cli-linux-arm64-musl": "2.5.9",
"@biomejs/cli-linux-x64": "2.5.9",
"@biomejs/cli-linux-x64-musl": "2.5.9",
"@biomejs/cli-win32-arm64": "2.5.9",
"@biomejs/cli-win32-x64": "2.5.9"
"@biomejs/cli-darwin-arm64": "2.5.10",
"@biomejs/cli-darwin-x64": "2.5.10",
"@biomejs/cli-linux-arm64": "2.5.10",
"@biomejs/cli-linux-arm64-musl": "2.5.10",
"@biomejs/cli-linux-x64": "2.5.10",
"@biomejs/cli-linux-x64-musl": "2.5.10",
"@biomejs/cli-win32-arm64": "2.5.10",
"@biomejs/cli-win32-x64": "2.5.10"
}
},
"node_modules/@biomejs/cli-darwin-arm64": {
"version": "2.5.9",
"resolved": "https://registry.npmjs.org/@biomejs/cli-darwin-arm64/-/cli-darwin-arm64-2.5.9.tgz",
"integrity": "sha512-am22pX2aBqznqq1eMyIj/bZ++riF3Lk6ct7cbv+gQK0csFhr+d8O0RkOi2FF2qSgFgANbqNkIZ0/PxlnW2pLFg==",
"version": "2.5.10",
"resolved": "https://registry.npmjs.org/@biomejs/cli-darwin-arm64/-/cli-darwin-arm64-2.5.10.tgz",
"integrity": "sha512-ItCrxKK6SXVT6flYs0qIuBd4AA3TTTl4d66Re6YI2FuGZnN85NmuYNzkiTJUyYw8qBLv69L5zTUB6uyWd++h3Q==",
"cpu": [
"arm64"
],
@@ -113,9 +113,9 @@
}
},
"node_modules/@biomejs/cli-darwin-x64": {
"version": "2.5.9",
"resolved": "https://registry.npmjs.org/@biomejs/cli-darwin-x64/-/cli-darwin-x64-2.5.9.tgz",
"integrity": "sha512-l44KWDHLDvEnD0N/XcrVs7VXb3A18xL7QS3WB0eL93wbmk529ffIG55vleGCqaunpRUjLrdnjK05Qki1dsjylg==",
"version": "2.5.10",
"resolved": "https://registry.npmjs.org/@biomejs/cli-darwin-x64/-/cli-darwin-x64-2.5.10.tgz",
"integrity": "sha512-yLsPU9pAmtChXDu8vhKAzErqe+LeeYuwuUB2FZMkRitsmdodxsYRa9KHrFispsUHzzOu+9HB3nP/TQxyia+Sjw==",
"cpu": [
"x64"
],
@@ -130,9 +130,9 @@
}
},
"node_modules/@biomejs/cli-linux-arm64": {
"version": "2.5.9",
"resolved": "https://registry.npmjs.org/@biomejs/cli-linux-arm64/-/cli-linux-arm64-2.5.9.tgz",
"integrity": "sha512-ICaK+IYaVZvKbBxX2rwrPT0DdUDMnE9Vm3nQGe+mltQPmUg19pONzkPWGdY4FCsoreDETWDynvdt4ysCbF5gNQ==",
"version": "2.5.10",
"resolved": "https://registry.npmjs.org/@biomejs/cli-linux-arm64/-/cli-linux-arm64-2.5.10.tgz",
"integrity": "sha512-VG8uQW/86a1roLaIFvtIbEigxIdzdJ190oGyg1tV7VYeQtOS+x10sflk7WbuXgw91EtZX5DlIIIej1YqkNLlcg==",
"cpu": [
"arm64"
],
@@ -150,9 +150,9 @@
}
},
"node_modules/@biomejs/cli-linux-arm64-musl": {
"version": "2.5.9",
"resolved": "https://registry.npmjs.org/@biomejs/cli-linux-arm64-musl/-/cli-linux-arm64-musl-2.5.9.tgz",
"integrity": "sha512-7ImVPwBLCtkmpR5esd8RHhTqW94f0JLJQum6AneYcy94jRm18TaPPm7slaigGzFhfgt3QiD1Vj52LKmBAnKizA==",
"version": "2.5.10",
"resolved": "https://registry.npmjs.org/@biomejs/cli-linux-arm64-musl/-/cli-linux-arm64-musl-2.5.10.tgz",
"integrity": "sha512-t1QAKZwQJRB4dvgJSgFiQ4BNfNPChg69BNonz854qLVxnjT3UvDzQg9mbkTJRu35ZqU0Rw10A73J8Urgbg2RPw==",
"cpu": [
"arm64"
],
@@ -170,9 +170,9 @@
}
},
"node_modules/@biomejs/cli-linux-x64": {
"version": "2.5.9",
"resolved": "https://registry.npmjs.org/@biomejs/cli-linux-x64/-/cli-linux-x64-2.5.9.tgz",
"integrity": "sha512-z22Q/zFYSvbIJfW1CbfZPu4X8PddS6Qd2ORbc6h+aT6EcwAxUF3m6fA4HjNvA3TU4X0dTJRwNPB165ES3PJXzg==",
"version": "2.5.10",
"resolved": "https://registry.npmjs.org/@biomejs/cli-linux-x64/-/cli-linux-x64-2.5.10.tgz",
"integrity": "sha512-4O6T0eq2heoHZN0a9UX+rWQoxXEBaKf+lRi2hbsGlHneUz9BWXM76nEWMK7Eeq8gzMxR1khQB6BFpAASpeXqGg==",
"cpu": [
"x64"
],
@@ -190,9 +190,9 @@
}
},
"node_modules/@biomejs/cli-linux-x64-musl": {
"version": "2.5.9",
"resolved": "https://registry.npmjs.org/@biomejs/cli-linux-x64-musl/-/cli-linux-x64-musl-2.5.9.tgz",
"integrity": "sha512-RXGaD0o1/pTTguYw1aeDJh9ad6Lfrui0fI7mBderTyGr7WuUJkBIttgLkR3XJyoxOkkgfBDspaUT8wXArTqLZw==",
"version": "2.5.10",
"resolved": "https://registry.npmjs.org/@biomejs/cli-linux-x64-musl/-/cli-linux-x64-musl-2.5.10.tgz",
"integrity": "sha512-pgDDqp9JybHm2I0KRgzN6i4+lt8xu4iqxUwLzglUMmOmyRTU1AYBGKzh9sNMOtIjah7xoWvKHlLVetvyifzoiQ==",
"cpu": [
"x64"
],
@@ -210,9 +210,9 @@
}
},
"node_modules/@biomejs/cli-win32-arm64": {
"version": "2.5.9",
"resolved": "https://registry.npmjs.org/@biomejs/cli-win32-arm64/-/cli-win32-arm64-2.5.9.tgz",
"integrity": "sha512-nHK+/HHC+D0ogAHUxomgoSTdjImb6fmNNVTKmf0tyu4eDL1DqPKIHc+i+UL8+b0RnAu8224qo8F2tCVnaT0A3w==",
"version": "2.5.10",
"resolved": "https://registry.npmjs.org/@biomejs/cli-win32-arm64/-/cli-win32-arm64-2.5.10.tgz",
"integrity": "sha512-pxAbxduPO4xq/Cvgaa2lOrs9BB0hEXmmDqfMNP4ZOffGOkUrD1/QGw9UAMpFQpX2P8MqTIIRuQKcmetum4Oa6A==",
"cpu": [
"arm64"
],
@@ -227,9 +227,9 @@
}
},
"node_modules/@biomejs/cli-win32-x64": {
"version": "2.5.9",
"resolved": "https://registry.npmjs.org/@biomejs/cli-win32-x64/-/cli-win32-x64-2.5.9.tgz",
"integrity": "sha512-Yiq0H56LjXSSw/hd9YkXgSLQfzyDJzbzU2TezozxyNw+uKWAqOtqGVvBfzKRRDiaFF5avGAhHdWKx7LtDOShUw==",
"version": "2.5.10",
"resolved": "https://registry.npmjs.org/@biomejs/cli-win32-x64/-/cli-win32-x64-2.5.10.tgz",
"integrity": "sha512-M+2dgBsl3lXRiTfgPVc2p3anS4Tocojke4rzFLScZ2Y/wmF+36dRb1iHCLiyGqOzQGyTplZH1HnEYviiAqi3nA==",
"cpu": [
"x64"
],
@@ -1577,9 +1577,9 @@
}
},
"node_modules/@types/node": {
"version": "26.2.0",
"resolved": "https://registry.npmjs.org/@types/node/-/node-26.2.0.tgz",
"integrity": "sha512-5IviulTZeRNp2vAJ514cc/HUlY5nZ9fCbq9DMyC52BrhFZACo3nI0R7qBxhQmo/d27NFe96ur/b7Wwxklda+kg==",
"version": "26.3.0",
"resolved": "https://registry.npmjs.org/@types/node/-/node-26.3.0.tgz",
"integrity": "sha512-L3fgrnchriRC2ExBflb8j4uZZURHZfQsmQeyVzhjcHW4kkwVyo8/0h1B2MVzMTrYUJYu6G7EWs14hW/L9putqw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -2284,17 +2284,17 @@
}
},
"node_modules/css-select": {
"version": "5.2.2",
"resolved": "https://registry.npmjs.org/css-select/-/css-select-5.2.2.tgz",
"integrity": "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw==",
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/css-select/-/css-select-6.0.0.tgz",
"integrity": "sha512-rZZVSLle8v0+EY8QAkDWrKhpgt6SA5OtHsgBnsj6ZaLb5dmDVOWUDtQitd9ydxxvEjhewNudS6eTVU7uOyzvXw==",
"dev": true,
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0",
"css-what": "^6.1.0",
"domhandler": "^5.0.2",
"domutils": "^3.0.1",
"nth-check": "^2.0.1"
"css-what": "^7.0.0",
"domhandler": "^5.0.3",
"domutils": "^3.2.2",
"nth-check": "^2.1.1"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
@@ -2315,9 +2315,9 @@
}
},
"node_modules/css-what": {
"version": "6.2.2",
"resolved": "https://registry.npmjs.org/css-what/-/css-what-6.2.2.tgz",
"integrity": "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==",
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/css-what/-/css-what-7.0.0.tgz",
"integrity": "sha512-wD5oz5xibMOPHzy13CyGmogB3phdvcDaB5t0W/Nr5Z2O/agcB8YwOz6e2Lsp10pNDzBoDO9nVa3RGs/2BttpHQ==",
"dev": true,
"license": "BSD-2-Clause",
"engines": {
@@ -4275,9 +4275,9 @@
"optional": true
},
"node_modules/sax": {
"version": "1.6.0",
"resolved": "https://registry.npmjs.org/sax/-/sax-1.6.0.tgz",
"integrity": "sha512-6R3J5M4AcbtLUdZmRv2SygeVaM7IhrLXu9BmnOGmmACak8fiUtOsYNWUS4uK7upbmHIBbLBeFeI//477BKLBzA==",
"version": "1.6.1",
"resolved": "https://registry.npmjs.org/sax/-/sax-1.6.1.tgz",
"integrity": "sha512-42tBVwLWnaQvW5zc4HbZrTuWccECCZfBi92FDuwtqxasH+JbPB3/FOKb1m222K42R4WxuxzzMsTswfzgtSu64Q==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
@@ -4769,19 +4769,19 @@
"dev": true
},
"node_modules/svgo": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/svgo/-/svgo-4.0.2.tgz",
"integrity": "sha512-ekx94z1rRc5LDi6oSUaeRnYhd0UOJxdtQCL2rF8xpWxD3TPAsISWOrxezqGovqS38GRZOdpDfvQe3ts6F7nsng==",
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/svgo/-/svgo-4.1.0.tgz",
"integrity": "sha512-bkxnTg1kSU0guhIBmibA6UUhrQmPVA1XsQLN+ylCd+UWzbnLkySOcXpyk1mrl05f+pcaCx2eHb+sp6BgMZWX+Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"commander": "^11.1.0",
"css-select": "^5.1.0",
"css-select": "^6.0.0",
"css-tree": "^3.0.1",
"css-what": "^6.1.0",
"css-what": "^7.0.0",
"csso": "^5.0.5",
"picocolors": "^1.1.1",
"sax": "^1.5.0"
"sax": "1.6.1"
},
"bin": {
"svgo": "bin/svgo.js"
@@ -5044,16 +5044,16 @@
"license": "MIT"
},
"node_modules/vite": {
"version": "8.2.1",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.2.1.tgz",
"integrity": "sha512-EU/eS7BH3XROHh2YnBefjM6DBKA6ZeMZEYQbj7NLWg5wHYlhB8B/Mayd5XsgWq+NFYccDOTemRpdETWR6Ka/lw==",
"version": "8.2.2",
"resolved": "https://registry.npmjs.org/vite/-/vite-8.2.2.tgz",
"integrity": "sha512-cFKLV/PRgAUlIRm5WjMjJ86jrftzpqcgH+Us+DS8mI3CDNiH30Whrz8uHL3+MOLPAgqbMBAqWdAHAphOAM+z/Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"lightningcss": "^1.33.0",
"picomatch": "^4.0.5",
"postcss": "^8.5.25",
"rolldown": "~1.2.1",
"postcss": "^8.5.26",
"rolldown": "~1.2.4",
"tinyglobby": "^0.2.17"
},
"bin": {
@@ -5070,7 +5070,7 @@
},
"peerDependencies": {
"@types/node": "^20.19.0 || >=22.12.0",
"@vitejs/devtools": "^0.4.0",
"@vitejs/devtools": "^0.4.0 || ^0.5.0",
"esbuild": "^0.27.0 || ^0.28.0",
"jiti": ">=1.21.0",
"less": "^4.0.0",

View File

@@ -29,8 +29,8 @@
"swiped-events": "1.2.0"
},
"devDependencies": {
"@biomejs/biome": "2.5.9",
"@types/node": "^26.2.0",
"@biomejs/biome": "2.5.10",
"@types/node": "^26.3.0",
"browserslist": "^4.28.8",
"browserslist-to-esbuild": "^2.1.1",
"edge.js": "^6.5.1",
@@ -41,9 +41,9 @@
"stylelint": "^17.14.1",
"stylelint-config-standard-less": "^4.1.0",
"stylelint-prettier": "^5.0.3",
"svgo": "^4.0.2",
"svgo": "^4.1.0",
"typescript": "~7.0.2",
"vite": "^8.2.1",
"vite": "^8.2.2",
"vite-bundle-analyzer": "^1.3.9"
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -28,11 +28,11 @@ class TrackerPatternsDB:
ctx_name = "data_tracker_patterns"
# ClearURL rule lists, the first one that responds HTTP 200 is used
CLEAR_LIST_URL = [
# ClearURL rule lists, the first one that responds HTTP 200 is used
"https://rules1.clearurls.xyz/data.minify.json",
"https://cdn.jsdelivr.net/gh/clearurls/rules@refs/heads/gh-pages/data.minify.json",
"https://rules2.clearurls.xyz/data.minify.json",
"https://raw.githubusercontent.com/ClearURLs/Rules/refs/heads/master/data.min.json",
"https://rules1.clearurls.xyz/data.minify.json",
]
class Fields:

View File

@@ -6,7 +6,8 @@ from urllib.parse import urlencode
from datetime import datetime
from searx.exceptions import SearxEngineAPIException
from searx.utils import html_to_text, get_embeded_stream_url
from searx.result_types import EngineResults
from searx.utils import html_to_text
about = {
"website": "https://tv.360kan.com/",
@@ -29,12 +30,12 @@ def request(query, params):
return params
def response(resp):
def response(resp) -> EngineResults:
try:
data = resp.json()
except Exception as e:
raise SearxEngineAPIException(f"Invalid response: {e}") from e
results = []
res = EngineResults()
if "data" not in data or "result" not in data["data"]:
raise SearxEngineAPIException("Invalid response")
@@ -50,16 +51,15 @@ def response(resp):
except (ValueError, TypeError):
published_date = None
results.append(
{
'url': entry["play_url"],
'title': html_to_text(entry["title"]),
'content': html_to_text(entry["description"]),
'template': 'videos.html',
'publishedDate': published_date,
'thumbnail': entry["cover_img"],
"iframe_src": get_embeded_stream_url(entry["play_url"]),
}
res.add(
res.types.LegacyResult(
url=entry["play_url"],
title=html_to_text(entry["title"]),
content=html_to_text(entry["description"]),
template='videos.html',
publishedDate=published_date,
thumbnail=entry["cover_img"],
)
)
return results
return res

View File

@@ -187,8 +187,9 @@ def set_loggers(engine: "Engine|types.ModuleType", engine_name: str):
def update_engine_attributes(engine: "Engine | types.ModuleType", engine_data: dict[str, t.Any]):
# pylint: disable=too-many-branches
# set engine attributes from engine_data
# set / update engine attributes from engine_data
kvargs: dict[str, t.Any]
engine.about = getattr(engine, "about", EngineAbout())
if isinstance(engine.about, EngineAbout):
kvargs = {**msgspec.to_builtins(engine.about), **engine_data.get("about", {})}
else:

View File

@@ -35,7 +35,7 @@ Implementation
==============
"""
import typing as t
from datetime import datetime, timedelta
from urllib.parse import urlencode
@@ -85,7 +85,7 @@ Additional subcategories:
# Do we need support for "free_collection" and "include_stock_enterprise"?
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if not categories:
raise ValueError("adobe_stock engine: categories is unset")
@@ -100,9 +100,9 @@ def init(_):
raise ValueError("adobe_stock engine: adobe_content_types is unset")
if isinstance(adobe_content_types, list):
for t in adobe_content_types:
if t not in ADOBE_VALID_TYPES:
raise ValueError("adobe_stock engine: adobe_content_types: '%s' is invalid" % t)
for content_type in adobe_content_types:
if content_type not in ADOBE_VALID_TYPES:
raise ValueError("adobe_stock engine: adobe_content_types: '%s' is invalid" % content_type)
else:
raise ValueError(
"adobe_stock engine: adobe_content_types must be a list of strings not %s" % type(adobe_content_types)

View File

@@ -49,6 +49,9 @@ CACHE: EngineCache
def setup(engine_settings: dict[str, t.Any]) -> bool:
if baidu_category not in ('general', 'images', 'it'):
raise SearxEngineAPIException(f"Unsupported category: {baidu_category}")
global CACHE # pylint: disable=global-statement
CACHE = EngineCache(engine_settings["name"])
return True
@@ -65,11 +68,6 @@ def get_image_cookies(headers: dict[str, str]) -> dict[str, str]:
return cookies
def init(_):
if baidu_category not in ('general', 'images', 'it'):
raise SearxEngineAPIException(f"Unsupported category: {baidu_category}")
def request(query, params):
page_num = params["pageno"]

View File

@@ -45,7 +45,7 @@ CACHE_SESSION_ID_KEY = "session_id_key"
KEYWORD_RE = re.compile(r"\[\/?Keyword\]")
def init(engine_settings: dict[str, t.Any]) -> bool:
def setup(engine_settings: dict[str, t.Any]) -> bool:
global CACHE # pylint: disable=global-statement
CACHE = EngineCache(engine_name=engine_settings["name"])
return True

View File

@@ -135,7 +135,7 @@ from searx.utils import (
eval_xpath_getindex,
eval_xpath_list,
extract_text,
get_embeded_stream_url,
get_embedded_stream_url,
js_obj_str_to_json_str,
js_obj_str_to_python,
)
@@ -338,7 +338,7 @@ def _parse_search(resp: SXNG_Response) -> EngineResults:
if len(video_tag):
# In my tests a video tag in the WEB search was most often not a
# video, except the ones from youtube ..
iframe_src = get_embeded_stream_url(url)
iframe_src = get_embedded_stream_url(url)
if iframe_src:
item["iframe_src"] = iframe_src
item["template"] = "videos.html"
@@ -406,9 +406,6 @@ def _parse_videos(json_resp: dict[str, t.Any]) -> EngineResults:
)
if result["thumbnail"] is not None:
item["thumbnail"] = result["thumbnail"]["src"]
iframe_src = get_embeded_stream_url(result["url"])
if iframe_src:
item["iframe_src"] = iframe_src
res.add(item)

View File

@@ -40,7 +40,7 @@ if t.TYPE_CHECKING:
about = {
"website": "https://api.search.brave.com/",
"wikidata_id": None,
"official_api_documentation": "https://api-dashboard.search.brave.com/documentation",
"official_api_documentation": "https://api-dashboard.search.brave.com/api-reference/web/search/get",
"use_official_api": True,
"require_api_key": True,
"results": "JSON",
@@ -63,8 +63,10 @@ base_url = "https://api.search.brave.com/res/v1/web/search"
time_range_map = {"day": "past_day", "week": "past_week", "month": "past_month", "year": "past_year"}
"""Mapping of SearXNG time ranges to Brave API time ranges."""
max_page = 10
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
"""Initialize the engine."""
if not api_key:
raise SearxEngineAPIException("No API key provided")
@@ -75,7 +77,7 @@ def request(query: str, params: "OnlineParams") -> None:
search_args: dict[str, str | int | None] = {
"q": query,
"count": results_per_page,
"offset": (params["pageno"] - 1) * results_per_page,
"offset": params["pageno"] - 1,
"text_decorations": False,
}

View File

@@ -41,7 +41,7 @@ search_index = "cw22"
<https://www.chatnoir.eu/docs/api-general>`_ for a full list."""
def _obtain_api_key() -> tuple[str, str, str]:
def _obtain_api_key() -> tuple[str, str]:
home_resp = get(base_url)
if not home_resp.ok:
raise SearxEngineAPIException("failed to obtain api key")
@@ -58,10 +58,9 @@ def _obtain_api_key() -> tuple[str, str, str]:
)
if not token_resp.ok:
raise SearxEngineAPIException("failed to obtain api key")
session_id = token_resp.cookies["sessionid"]
scraped_api_key = token_resp.json()["token"]["token"]
return csrf_token, session_id, scraped_api_key
return csrf_token, scraped_api_key
def request(query: str, params: "OnlineParams"):
@@ -73,7 +72,7 @@ def request(query: str, params: "OnlineParams"):
params["headers"].update(headers)
else:
csrf_token, session_id, scraped_api_key = _obtain_api_key()
csrf_token, scraped_api_key = _obtain_api_key()
headers = {
"Authorization": f"Bearer {scraped_api_key}",
@@ -81,7 +80,7 @@ def request(query: str, params: "OnlineParams"):
}
params["headers"].update(headers)
params["cookies"] = {"csrftoken": session_id, "sessionid": session_id}
params["cookies"] = {"csrftoken": csrf_token}
params["url"] = f"{base_url}/api/v1/_search"
params["method"] = "POST"

View File

@@ -78,7 +78,7 @@ time_range_dict = {'day': '24h', 'week': '1w', 'month': '1m', 'year': '1y'}
base_url = "https://www.chinaso.com"
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if chinaso_news_source not in t.get_args(ChinasoNewsSourceType):
raise ValueError(f"Unsupported news source: {chinaso_news_source}")

View File

@@ -74,6 +74,7 @@ Implementations
===============
"""
import typing as t
import re
from os.path import expanduser, isabs, realpath, commonprefix
from shlex import split as shlex_split
@@ -100,7 +101,7 @@ _command_logger = logger.getChild('command')
_compiled_parse_regex = {}
def init(engine_settings):
def setup(engine_settings: dict[str, t.Any]) -> bool | None:
check_parsing_options(engine_settings)
if 'command' not in engine_settings:

View File

@@ -36,7 +36,7 @@ base_url = "https://www.dogpile.com"
safe_search_map = {0: "none", 1: "moderate", 2: "heavy"}
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if dogpile_categ not in ("search", "images", "videos", "news"):
raise ValueError("invalid search type: %s" % dogpile_categ)

View File

@@ -10,7 +10,8 @@ from datetime import datetime
from urllib.parse import urlencode
from urllib.parse import quote_plus
from searx.utils import get_embeded_stream_url, html_to_text, gen_useragent, extr
from searx.result_types import EngineResults, MainResult, LegacyResult, Image
from searx.utils import html_to_text, gen_useragent, extr
from searx.network import get # see https://github.com/searxng/searxng/issues/762
from searx.engines.duckduckgo import fetch_traits # pylint: disable=unused-import
@@ -47,7 +48,7 @@ _HTTP_User_Agent: str = gen_useragent()
send_accept_language_header = False
def init(engine_settings: dict[str, t.Any]):
def setup(engine_settings: dict[str, t.Any]) -> bool | None:
if engine_settings["ddg_category"] not in ["images", "videos", "news"]:
raise ValueError(f"Unsupported DuckDuckGo category: {engine_settings['ddg_category']}")
@@ -148,54 +149,51 @@ def request(query: str, params: "OnlineParams") -> None:
def _image_result(result):
return {
'template': 'images.html',
'url': result['url'],
'title': result['title'],
'content': '',
'thumbnail_src': result['thumbnail'],
'img_src': result['image'],
'resolution': '%s x %s' % (result['width'], result['height']),
'source': result['source'],
}
return Image(
url=result['url'],
title=result['title'],
content='',
thumbnail_src=result['thumbnail'],
img_src=result['image'],
resolution='%s x %s' % (result['width'], result['height']),
source=result['source'],
)
def _video_result(result):
return {
'template': 'videos.html',
'url': result['content'],
'title': result['title'],
'content': result['description'],
'thumbnail': result['images'].get('small') or result['images'].get('medium'),
'iframe_src': get_embeded_stream_url(result['content']),
'source': result['provider'],
'length': result['duration'],
'metadata': result.get('uploader'),
}
return LegacyResult(
template='videos.html',
url=result['content'],
title=result['title'],
content=result['description'],
thumbnail=result['images'].get('small') or result['images'].get('medium'),
source=result['provider'],
length=result['duration'],
metadata=result.get('uploader'),
)
def _news_result(result):
return {
'url': result['url'],
'title': result['title'],
'content': html_to_text(result['excerpt']),
'source': result['source'],
'publishedDate': datetime.fromtimestamp(result['date']),
}
return MainResult(
url=result['url'],
title=result['title'],
content=html_to_text(result['excerpt']),
publishedDate=datetime.fromtimestamp(result['date']),
)
def response(resp):
results = []
def response(resp: "SXNG_Response") -> EngineResults:
res = EngineResults()
res_json = resp.json()
for result in res_json['results']:
if ddg_category == 'images':
results.append(_image_result(result))
res.add(_image_result(result))
elif ddg_category == 'videos':
results.append(_video_result(result))
res.add(_video_result(result))
elif ddg_category == 'news':
results.append(_news_result(result))
res.add(_news_result(result))
else:
raise ValueError(f"Invalid duckduckgo category: {ddg_category}")
return results
return res

View File

@@ -109,7 +109,19 @@ def response(resp: SXNG_Response):
json_data = loads(resp.text[resp.text.find('\n') + 1 : resp.text.rfind('\n') - 2])
geoloc = weather.GeoLocation.by_query(resp.search_params["query"])
location = json_data.get("location")
if not location:
return res
metadata = json_data.get("weatherAlerts", {}).get("metadata", {})
geoloc = weather.GeoLocation(
name=location,
latitude=metadata.get("latitude"),
longitude=metadata.get("longitude"),
elevation=0,
country_code=metadata.get("language").split("-")[-1],
timezone=json_data.get("location"),
)
weather_answer = EngineResults.types.WeatherAnswer(
current=_weather_data(geoloc, json_data["currentWeather"]),

View File

@@ -41,6 +41,7 @@ authentication configured to read from ``my-index`` index.
"""
import typing as t
from json import loads, dumps
from searx.exceptions import SearxEngineAPIException
from searx.result_types import EngineResults
@@ -68,7 +69,7 @@ show_metadata = False
page_size = 10
def init(engine_settings):
def setup(engine_settings: dict[str, t.Any]) -> bool | None:
if 'query_type' in engine_settings and engine_settings['query_type'] not in _available_query_types:
raise ValueError('unsupported query type', engine_settings['query_type'])

View File

@@ -78,7 +78,7 @@ content_max_characters: int = 500
"""Maximum characters for the requested content."""
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if not api_key:
raise SearxEngineAPIException("No API key provided")
if not 1 <= results_per_page <= 100:

View File

@@ -47,7 +47,7 @@ CACHE_VALID_DURATION = 30 * 24 * 3600 # one month, same as website
"""Duration how long settings cookies are valid."""
def init(engine_settings: dict[str, t.Any]):
def setup(engine_settings: dict[str, t.Any]) -> bool | None:
global CACHE # pylint: disable=global-statement
CACHE = EngineCache(engine_settings["name"])

View File

@@ -38,6 +38,7 @@ Implementation
"""
import typing as t
from urllib.parse import urlencode
from dateutil import parser
@@ -77,7 +78,7 @@ page_size: int = 10
"""Maximum number of results per page (default 10)."""
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if not base_url:
raise ValueError('gitea engine: base_url is unset')

View File

@@ -10,7 +10,6 @@ from searx.utils import (
eval_xpath_getindex,
eval_xpath_list,
extract_text,
get_embeded_stream_url,
parse_duration_string,
)
@@ -79,7 +78,6 @@ def response(resp: "SXNG_Response") -> EngineResults:
title=title,
thumbnail=thumbnail,
length=length,
iframe_src=get_embeded_stream_url(url) or "",
template="videos.html",
)
)

View File

@@ -1,127 +0,0 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Heexy_ is a minimalist search engine that focuses on privacy.
Although it also supports news and videos, these are not implemented here
because they usually return no result to very few irrelevant ones.
It seems to use Bing internally, as the image thumbnails are loaded from Bing.
.. _Heexy: https://docs.heexy.org/introduction
"""
from urllib.parse import urlencode
import typing as t
from searx.enginelib import EngineCache
from searx.network import get
from searx.exceptions import SearxEngineAPIException, SearxEngineAccessDeniedException
from searx.result_types import EngineResults
from searx.utils import gen_useragent
if t.TYPE_CHECKING:
from searx.extended_types import SXNG_Response
from searx.search.processors import OnlineParams
about = {
"website": "https://heexy.org",
"wikidata_id": None,
"official_api_documentation": None,
"use_official_api": False,
"require_api_key": False,
"results": "JSON",
}
paging = True
safesearch = True
categories = ["general"]
heexy_categ = "web"
"""Category to search in. Can be either "web" or "image"."""
base_url = "https://heexy.org"
api_url = "https://seapi.heexy.org"
safe_search_map = {0: "off", 1: "on", 2: "on"}
CACHE: EngineCache
"""Cache for storing the ``X-Data-Cacheft`` token (acts like an API key)."""
def setup(engine_settings: dict[str, t.Any]) -> bool:
global CACHE # pylint: disable=global-statement
if heexy_categ not in ("web", "image"):
raise ValueError("invalid search category: %s" % heexy_categ)
CACHE = EngineCache(engine_settings["name"])
return True
def _get_api_token(query: str) -> str:
"""The API token is independent of the search query. We just need any query
to obtain it initially, and don't hardcode it here to decrease chances of
getting blocked. The token must be passed as ``X-Data-Cacheft`` header."""
cached_token: str = CACHE.get("token")
if cached_token:
return cached_token
resp = get(
f"{base_url}/search?q={query}", headers={"User-Agent": gen_useragent(), "Accept-Language": "en-US,en:q=0.9"}
)
if not resp.ok:
raise SearxEngineAPIException("failed to obtain request token: invalid response code")
token = resp.cookies["cacheft"]
if not token:
raise SearxEngineAPIException("failed to obtain request token: no token found")
CACHE.set("token", token, expire=3 * 60)
return token
def request(query: str, params: "OnlineParams") -> None:
args = {
"q": query,
"page": params["pageno"],
"safe": safe_search_map[params["safesearch"]],
}
if params["searxng_locale"] != "all":
args["lang"] = params["searxng_locale"].split("-")[0]
params["url"] = f"{api_url}/search/{heexy_categ}?{urlencode(args)}"
params["headers"]["Origin"] = api_url
params["cookies"]["cacheft"] = _get_api_token(query)
def response(resp: "SXNG_Response"):
res = EngineResults()
json_resp = resp.json()
if not json_resp["success"]:
raise SearxEngineAccessDeniedException()
result: dict[str, str]
for result in json_resp["results"]:
if heexy_categ == "web":
res.add(
res.types.MainResult(
url=result["url"],
title=result["title"],
content=result["description"],
)
)
elif heexy_categ == "image":
res.add(
res.types.Image(
title=result["description"],
url=result["url"],
thumbnail_src=result["image"],
img_src=result["rawImage"],
)
)
return res

View File

@@ -33,6 +33,7 @@ Implementations
"""
import typing as t
from urllib.parse import urlencode
from datetime import datetime
@@ -62,7 +63,7 @@ huggingface_endpoint = 'models'
"""
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if huggingface_endpoint not in ('datasets', 'models', 'spaces'):
raise SearxEngineAPIException(f"Unsupported Hugging Face endpoint: {huggingface_endpoint}")

View File

@@ -8,6 +8,7 @@ No public instance offer a public API now
"""
import typing as t
import time
import random
from urllib.parse import quote_plus, urlparse
@@ -34,7 +35,7 @@ time_range_support = True
base_url: list[str] | str = []
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if not base_url:
raise ValueError("missing invidious base_url")

View File

@@ -87,7 +87,7 @@ api_key = ""
"""Kagi API key. Required for using this engine."""
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if not api_key:
raise ValueError("api_key is required for using kagi")

View File

@@ -51,7 +51,7 @@ enable_http2 = False
safe_search_map = {0: "Off", 1: "Moderate", 2: "Strict"}
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if luxxle_categ not in ("search", "images", "videos", "news"):
raise ValueError("invalid luxxle category: %s" % luxxle_categ)

View File

@@ -164,7 +164,7 @@ def response(resp: SXNG_Response):
return res
def init(_: dict[str, t.Any]):
def setup(_: dict[str, t.Any]) -> bool:
if not api_key:
logger.error("missing api_key: see https://about.marginalia-search.com/article/api")

View File

@@ -26,6 +26,8 @@ Implementations
"""
import typing as t
try:
import mariadb # pyright: ignore [reportMissingImports]
except ImportError:
@@ -60,15 +62,17 @@ paging = True
_connection = None
def init(engine_settings):
global _connection # pylint: disable=global-statement
def setup(engine_settings: dict[str, t.Any]) -> bool | None:
if 'query_str' not in engine_settings:
raise ValueError('query_str cannot be empty')
if not engine_settings['query_str'].lower().startswith('select '):
raise ValueError('only SELECT query is supported')
def init(_):
global _connection # pylint: disable=global-statement
_connection = mariadb.connect(database=database, user=username, password=password, host=host, port=port)

View File

@@ -35,7 +35,7 @@ Here is a simple example to query a Meilisearch instance:
"""
# pylint: disable=global-statement
import typing as t
from json import dumps
from searx.result_types import EngineResults
from searx.extended_types import SXNG_Response
@@ -49,7 +49,7 @@ categories = ['general']
paging = True
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if index == '':
raise ValueError('index cannot be empty')

View File

@@ -1,6 +1,7 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Mojeek (general, images, news)"""
import typing as t
from datetime import datetime
from urllib.parse import urlencode
@@ -51,7 +52,7 @@ region_param = "arc"
_delta_kwargs = {"day": "days", "week": "weeks", "month": "months", "year": "years"}
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if search_type not in ("", "images", "news"):
raise ValueError(f"Invalid search type {search_type}")

View File

@@ -19,6 +19,7 @@ Implementation
==============
"""
import typing as t
from urllib.parse import quote_plus
about = {
@@ -37,7 +38,7 @@ matrix_url = "https://matrix.to"
page_size = 20
def init(engine_settings): # pylint: disable=unused-argument
def setup(_: dict[str, t.Any]) -> bool | None:
"""The ``base_url`` must be set in the configuration, if ``base_url`` is not
set, a :py:obj:`ValueError` is raised during initialization.

View File

@@ -25,6 +25,8 @@ Implementations
"""
import typing as t
from searx.result_types import EngineResults
try:
@@ -60,15 +62,17 @@ paging = True
_connection = None
def init(engine_settings):
global _connection # pylint: disable=global-statement
def setup(engine_settings: dict[str, t.Any]) -> bool | None:
if 'query_str' not in engine_settings:
raise ValueError('query_str cannot be empty')
if not engine_settings['query_str'].lower().startswith('select '):
raise ValueError('only SELECT query is supported')
def init(_):
global _connection # pylint: disable=global-statement
_connection = mysql.connector.connect(
database=database,
user=username,

View File

@@ -2,6 +2,8 @@
# pylint: disable=line-too-long
"""Naver for SearXNG"""
import typing as t
from urllib.parse import urlencode
from lxml import html
@@ -16,7 +18,6 @@ from searx.utils import (
html_to_text,
parse_duration_string,
js_obj_str_to_python,
get_embeded_stream_url,
)
# engine metadata
@@ -67,7 +68,7 @@ naver_category_dict = {
}
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if naver_category not in ('general', 'images', 'news', 'videos'):
raise SearxEngineAPIException(f"Unsupported category: {naver_category}")
@@ -193,7 +194,7 @@ def parse_news(data):
def parse_videos(data):
results = []
res = EngineResults()
dom = html.fromstring(data)
@@ -212,15 +213,14 @@ def parse_videos(data):
except (ValueError, TypeError):
pass
results.append(
{
"template": "videos.html",
"title": extract_text(eval_xpath(item, ".//a[contains(@class, 'info_title')]")),
"url": url,
"thumbnail": thumbnail,
"length": length,
"iframe_src": get_embeded_stream_url(url),
}
res.add(
res.types.LegacyResult(
template="videos.html",
title=extract_text(eval_xpath(item, ".//a[contains(@class, 'info_title')]")),
url=url,
thumbnail=thumbnail,
length=length,
)
)
return results
return res

View File

@@ -2,6 +2,7 @@
"""Pexels (images)"""
import re
import typing as t
from urllib.parse import urlencode
from lxml import html
@@ -46,9 +47,10 @@ CACHE: EngineCache
enable_http2 = False
def init(engine_settings):
def setup(engine_settings: dict[str, t.Any]) -> bool:
global CACHE # pylint: disable=global-statement
CACHE = EngineCache(engine_settings["name"])
return True
def _get_secret_key():

View File

@@ -21,6 +21,8 @@ Implementations
"""
import typing as t
try:
import psycopg2 # type: ignore
except ImportError:
@@ -55,15 +57,17 @@ paging = True
_connection = None
def init(engine_settings):
global _connection # pylint: disable=global-statement
def setup(engine_settings: dict[str, t.Any]) -> bool | None:
if 'query_str' not in engine_settings:
raise ValueError('query_str cannot be empty')
if not engine_settings['query_str'].lower().startswith('select '):
raise ValueError('only SELECT query is supported')
def init(_):
global _connection # pylint: disable=global-statement
_connection = psycopg2.connect(
database=database,
user=username,

View File

@@ -14,7 +14,7 @@ from lxml import html
import babel
from searx.enginelib.traits import EngineTraits
from searx.utils import eval_xpath_list, eval_xpath, extract_text, get_embeded_stream_url, extr
from searx.utils import eval_xpath_list, eval_xpath, extract_text, extr
from searx.locales import region_tag
from searx.result_types import EngineResults
@@ -61,7 +61,7 @@ video_page_map = {
}
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if privacywall_category not in ("general", "images", "videos"):
raise ValueError("invalid category: %s" % privacywall_category)
@@ -154,7 +154,6 @@ def _video_results(doc: "ElementBase") -> EngineResults:
title=extract_text(eval_xpath(result, ".//h2[contains(@class, 'video-card-title')]")) or "",
content=extract_text(eval_xpath(result, ".//p")) or "",
thumbnail=thumbnail or "",
iframe_src=get_embeded_stream_url(url) or "",
)
)

View File

@@ -1,6 +1,7 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Quark (Shenma) search engine for searxng"""
import typing as t
from urllib.parse import urlencode
from datetime import datetime
import re
@@ -43,7 +44,7 @@ def is_alibaba_captcha(html):
return bool(re.search(CAPTCHA_PATTERN, html))
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if quark_category not in ('general', 'images'):
raise SearxEngineAPIException(f"Unsupported category: {quark_category}")

View File

@@ -60,9 +60,6 @@ from searx.exceptions import (
SearxEngineTooManyRequestsException,
)
from searx.network import raise_for_httperror
from searx.utils import (
get_embeded_stream_url,
)
from searx.result_types import EngineResults
if t.TYPE_CHECKING:
@@ -299,7 +296,6 @@ def response(resp: "SXNG_Response") -> EngineResults:
title=title,
url=res_url,
content=content,
iframe_src=get_embeded_stream_url(res_url),
publishedDate=pub_date,
thumbnail=thumbnail,
template="videos.html",

View File

@@ -42,7 +42,7 @@ resulthunter_categ = "web"
"""Supported categories are ``web`` and ``images``."""
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if resulthunter_categ not in ("web", "images"):
raise ValueError("invalid category: %s" % resulthunter_categ)

View File

@@ -38,12 +38,10 @@ CACHE: EngineCache
"""Cache to store verification tokens for pagination."""
def init(_):
def setup(engine_settings: dict[str, t.Any]) -> bool:
if not base_url:
raise ValueError("base_url must be set")
def setup(engine_settings: dict[str, t.Any]) -> bool:
global CACHE # pylint: disable=global-statement
CACHE = EngineCache(engine_settings["name"])
return True
@@ -82,7 +80,7 @@ def response(resp: "SXNG_Response") -> EngineResults:
res.types.MainResult(
url=extract_text(eval_xpath(result, ".//a[contains(@class, 'title')]/@href")),
title=extract_text(eval_xpath(result, ".//a[contains(@class, 'title')]")),
content=extract_text(eval_xpath(result, ".//span[contains(@class, 'description') or @class='']")),
content=extract_text(eval_xpath(result, ".//span[contains(@class, 'description') or not(@class)]")),
)
)

View File

@@ -28,7 +28,7 @@ This is an example configuration for searching in the collection
"""
# pylint: disable=global-statement
import typing as t
from urllib.parse import urlencode
from searx.exceptions import SearxEngineAPIException
from searx.result_types import EngineResults
@@ -46,7 +46,7 @@ _search_url = ''
paging = True
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if collection == '':
raise ValueError('collection cannot be empty')

View File

@@ -1,6 +1,7 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""SoundCloud is a German audio streaming service."""
import typing as t
import re
import datetime
@@ -115,9 +116,10 @@ def response(resp):
return results
def init(engine_settings): # pylint: disable=unused-argument
def setup(engine_settings: dict[str, t.Any]) -> bool:
global CACHE # pylint: disable=global-statement
CACHE = EngineCache(engine_settings["name"]) # type:ignore
return True
def get_client_id() -> str | None:

View File

@@ -66,7 +66,7 @@ limit = 10
paging = True
def init(engine_settings):
def setup(engine_settings: dict[str, t.Any]) -> bool | None:
if 'query_str' not in engine_settings:
raise ValueError('query_str cannot be empty')

View File

@@ -165,7 +165,7 @@ CACHE: EngineCache
seconds."""
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
global CACHE # pylint: disable=global-statement
# hint: all three startpage engines (WEB, Images & News) can/should use the

View File

@@ -39,7 +39,7 @@ page_size = 10
api_url = "https://search.kompas.services"
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if startpagina_categ not in ("web", "images", "videos", "news"):
raise ValueError("invalid search type: %s" % startpagina_categ)

View File

@@ -152,7 +152,7 @@ def generate_nonce_and_signature(base_path: str, args: dict[str, t.Any]) -> tupl
maximum_page_size = {"web": 20, "images": 50, "videos": 10}
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if swisscows_category not in ("web", "images", "videos"):
raise ValueError("illegal swisscows category: %s" % swisscows_category)

View File

@@ -45,12 +45,10 @@ CACHE: EngineCache
"""Cache to store session codes (result of solved CAPTCHA)."""
def init(_):
def setup(engine_settings: dict[str, t.Any]) -> bool:
if tiger_category not in ("Websuche", "News"):
raise ValueError("invalid search category: %s" % tiger_category)
def setup(engine_settings: dict[str, t.Any]) -> bool:
global CACHE # pylint: disable=global-statement
CACHE = EngineCache(engine_settings["name"])
return True

View File

@@ -14,7 +14,7 @@ from urllib.parse import urlencode
from lxml import html
from searx.utils import eval_xpath_list, eval_xpath, extract_text, get_embeded_stream_url, ElementType
from searx.utils import eval_xpath_list, eval_xpath, extract_text, ElementType
from searx.result_types import EngineResults
from searx.enginelib import EngineAbout
@@ -44,7 +44,7 @@ tonline_channel_map = {"images": "flickr", "videos": "yt"}
language = "de"
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if tonline_categ not in ("web", "images", "videos", "news"):
raise ValueError("invalid category: %s" % tonline_categ)
@@ -126,7 +126,6 @@ def _video_results(doc: ElementType, res: EngineResults):
url=url,
title=" - ".join(extract_text(part) or "" for part in title_parts),
thumbnail=extract_text(eval_xpath(result, ".//img/@src") or "") or "",
iframe_src=get_embeded_stream_url(url) or "",
)
)

View File

@@ -83,7 +83,7 @@ show_torrent_files: bool = False
show_magnet_links: bool = True
def init(engine_settings=None): # pylint: disable=unused-argument
def setup(_: dict[str, t.Any]) -> bool | None:
"""Initialize the engine."""
if len(base_url) < 1:
raise ValueError('missing torznab base_url')

View File

@@ -75,6 +75,7 @@ Implementations
"""
import typing as t
from urllib.parse import urlencode
from dateutil.parser import parse
from searx.utils import html_to_text, humanize_number
@@ -115,7 +116,7 @@ def absolute_url(relative_url):
return f'{base_url.rstrip("/")}{relative_url}'
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if not base_url:
raise ValueError('tubearchivist engine: base_url is unset')
if not ta_token:

View File

@@ -39,7 +39,7 @@ tusk_categ = "web"
api_url = "https://api.tusksearch.com"
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if tusk_categ not in ("web", "images", "videos", "news"):
raise ValueError("invalid search type: %s" % tusk_categ)

View File

@@ -38,7 +38,7 @@ vuhuv_category = "general"
category_map = {"general": 1, "images": 2, "videos": 3}
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
if vuhuv_category not in category_map:
raise ValueError("invalid category: %s" % vuhuv_category)

View File

@@ -3,7 +3,7 @@
Wolfram|Alpha (Science)
"""
import typing as t
from json import loads
from urllib.parse import urlencode
@@ -51,7 +51,7 @@ CACHE: EngineCache
seconds."""
def init(engine_settings):
def setup(engine_settings: dict[str, t.Any]) -> bool | None:
global CACHE # pylint: disable=global-statement
CACHE = EngineCache(engine_settings["name"]) # type:ignore

View File

@@ -52,7 +52,7 @@ Implementations
"""
# pylint: disable=fixme
import typing as t
import random
from json import loads
from urllib.parse import urlencode
@@ -102,7 +102,7 @@ selected randomly.
"""
def init(_):
def setup(_: dict[str, t.Any]) -> bool | None:
valid_types = [
'text',
'image',

View File

@@ -15,6 +15,7 @@
__all__ = [
"Result",
"MainResult",
"LegacyResult",
"KeyValue",
"EngineResults",
"AnswerSet",

View File

@@ -31,6 +31,7 @@ from collections.abc import Callable
import msgspec
from searx import logger
from searx.utils import get_embedded_stream_url
log = logger.getChild("result_types")
@@ -480,6 +481,7 @@ class LegacyResult(dict[str, t.Any]):
category: str
publishedDate: datetime.datetime | None
pubdate: str = ""
iframe_src: str | None
# infobox result
urls: list[dict[str, str]]
@@ -509,6 +511,7 @@ class LegacyResult(dict[str, t.Any]):
self["score"] = self.get("score", 0)
self["category"] = self.get("category", "")
self["publishedDate"] = self.get("publishedDate")
self["iframe_src"] = self.get("iframe_src")
if "infobox" in self:
self["urls"] = self.get("urls", [])
@@ -531,6 +534,10 @@ class LegacyResult(dict[str, t.Any]):
DeprecationWarning,
)
# TODO: move into typed video results class once it is implemented # pylint: disable=fixme
if self.template == "videos.html" and self.url and not self.iframe_src:
self.iframe_src = get_embedded_stream_url(self.url)
def __getattr__(self, name: str, default: t.Any = UNSET) -> t.Any:
if default == UNSET and name not in self:
raise AttributeError(f"LegacyResult object has no field named: {name}")

View File

@@ -1296,22 +1296,6 @@ engines:
shortcut: hn
disabled: true
- name: heexy
engine: heexy
categories: general
heexy_categ: web
shortcut: he
disabled: true
inactive: true
- name: heexy images
engine: heexy
categories: images
heexy_categ: image
shortcut: hei
disabled: true
inactive: true
- name: hex
engine: hex
shortcut: hex
@@ -3360,30 +3344,20 @@ engines:
require_api_key: false
results: JSON
# - name: webcrawler
# engine: s1search
# shortcut: wc
# base_url: https://www.webcrawler.com
# disabled: true
# s1search yahoo engines / mirrors
# - name: excite
# engine: s1search
# shortcut: exc
# base_url: https://results.excite.com.s1search.co
# disabled: true
# - name: metacrawler
# engine: s1search
# shortcut: mec
# base_url: https://search.metacrawler.com
# disabled: true
- name: infospace
# s1search engines / mirrors
- name: metacrawler
engine: s1search
shortcut: ifs
base_url: https://search.infospace.com
shortcut: mec
base_url: https://search.metacrawler.com
disabled: true
inactive: true
- name: zoo search
engine: s1search
shortcut: zoo
base_url: https://search.zoo.com
disabled: true
inactive: true
# Doku engine lets you access to any Doku wiki instance:
# A public one or a privete/corporate one.

View File

@@ -592,7 +592,7 @@ def eval_xpath_getindex(
return default
def get_embeded_stream_url(url: str):
def get_embedded_stream_url(url: str):
"""
Converts a standard video URL into its embed format. Supported services include Youtube,
Facebook, Instagram, TikTok, Dailymotion, and Bilibili.

View File

@@ -152,7 +152,7 @@ commit '''
'date': r'Date: .*',
'message': r'\n\n.*$',
}
git_log_engine.init({"command": git_log_engine.command, "parse_regex": git_log_engine.parse_regex})
git_log_engine.setup({"command": git_log_engine.command, "parse_regex": git_log_engine.parse_regex})
expected_results = [
{
'commit': '35f9a8c81d162a361b826bbcd4a1081a4fbe76a7',