13 Commits

Author SHA1 Message Date
searxng-bot
c6707dc0b6 [data] update searx.data - update_engine_traits.py 2026-06-29 00:44:13 +00:00
Bnyro
8605230eb8 [fix] resulthunter: images in search results are not clickable 2026-06-28 22:31:01 +02:00
Bnyro
a54722e692 [feat] tiger: add support for related search terms / suggestions 2026-06-28 22:30:48 +02:00
Bnyro
7e9a6c2861 [fix] tiger: captcha paths have changed 2026-06-28 22:30:48 +02:00
dependabot[bot]
357662d86d [upd] github-actions: Bump docker/scout-action from 1.21.0 to 1.22.0 (#6317)
Bumps [docker/scout-action](https://github.com/docker/scout-action) from 1.21.0 to 1.22.0.
- [Release notes](https://github.com/docker/scout-action/releases)
- [Commits](cd72f264be...7520205ff6)

---
updated-dependencies:
- dependency-name: docker/scout-action
  dependency-version: 1.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-28 09:19:29 +02:00
dependabot[bot]
bfa76e4cd9 [upd] github-actions: Bump actions/cache from 5.0.5 to 6.0.0 (#6316)
Bumps [actions/cache](https://github.com/actions/cache) from 5.0.5 to 6.0.0.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](27d5ce7f10...2c8a9bd745)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: 6.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-28 09:19:14 +02:00
dependabot[bot]
a9d49a3349 [upd] github-actions: Bump actions/setup-python from 6.2.0 to 6.3.0 (#6315)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6.2.0 to 6.3.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](a309ff8b42...ece7cb06ca)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-version: 6.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-28 09:18:54 +02:00
github-actions[bot]
f8ffbf36f9 [l10n] update translations from Weblate (#6320)
6de4d96dd - 2026-06-23 - code_gremlin <code_gremlin@noreply.codeberg.org>
2026-06-26 10:24:36 +02:00
Markus Heiser
e3126b89e6 [mod] update_engine_descriptions.py - revision of the script (#6309)
* [mod] update_engine_descriptions.py - revision of the script (#6309)

The script was a bit outdated, type hints were missing, and the strings were
converted to double quotation marks

The `descriptions.items()` are sorted to avoid assigning the references to
seen_descriptions randomly (should reduce unnecessary diffs in the future).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

* [data] update searx.data - update_engine_descriptions.py (#6309)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

---------

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-06-24 12:10:28 +02:00
Markus Heiser
f9f3d089ec [fix] engine anaconda - add missing about configuration (#6308)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2026-06-24 10:45:40 +02:00
Hu Butui
e3713717f2 [fix] engines: remove terminated chinaso media engines
Signed-off-by: Hu Butui <hot123tea123@gmail.com>
2026-06-23 18:20:04 +02:00
Bnyro
952896d29e [feat] image results: automatically guess mimetype based on path 2026-06-22 12:46:22 +02:00
Bnyro
4cc32b2457 [fix] kozmonavt: remove pagination and set to inactive by default
Pagination requires a different nextpage query parameter each
day as it seems, so it's not possible to implement this in the Xpath
engine.
2026-06-22 10:06:09 +02:00
18 changed files with 2983 additions and 3548 deletions

View File

@@ -73,7 +73,7 @@ jobs:
# yamllint enable rule:line-length # yamllint enable rule:line-length
- name: Setup Python - name: Setup Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with: with:
python-version: "${{ env.PYTHON_VERSION }}" python-version: "${{ env.PYTHON_VERSION }}"
@@ -84,7 +84,7 @@ jobs:
fetch-depth: "0" fetch-depth: "0"
- name: Setup cache Python - name: Setup cache Python
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache@2c8a9bd7457de244a408f35966fab2fb45fda9c8 # v6.0.0
with: with:
key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}" key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}"
restore-keys: | restore-keys: |
@@ -96,7 +96,7 @@ jobs:
run: echo "date=$(date +'%Y%m%d')" >>$GITHUB_OUTPUT run: echo "date=$(date +'%Y%m%d')" >>$GITHUB_OUTPUT
- name: Setup cache container - name: Setup cache container
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache@2c8a9bd7457de244a408f35966fab2fb45fda9c8 # v6.0.0
with: with:
key: "container-${{ matrix.arch }}-${{ steps.date.outputs.date }}-${{ hashFiles('./requirements*.txt') }}" key: "container-${{ matrix.arch }}-${{ steps.date.outputs.date }}-${{ hashFiles('./requirements*.txt') }}"
restore-keys: | restore-keys: |

View File

@@ -41,7 +41,7 @@ jobs:
steps: steps:
- name: Setup Python - name: Setup Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with: with:
python-version: "${{ env.PYTHON_VERSION }}" python-version: "${{ env.PYTHON_VERSION }}"
@@ -51,7 +51,7 @@ jobs:
persist-credentials: "false" persist-credentials: "false"
- name: Setup cache Python - name: Setup cache Python
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache@2c8a9bd7457de244a408f35966fab2fb45fda9c8 # v6.0.0
with: with:
key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}" key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}"
restore-keys: | restore-keys: |

View File

@@ -32,7 +32,7 @@ jobs:
steps: steps:
- name: Setup Python - name: Setup Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with: with:
python-version: "${{ env.PYTHON_VERSION }}" python-version: "${{ env.PYTHON_VERSION }}"
@@ -43,7 +43,7 @@ jobs:
fetch-depth: "0" fetch-depth: "0"
- name: Setup cache Python - name: Setup cache Python
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache@2c8a9bd7457de244a408f35966fab2fb45fda9c8 # v6.0.0
with: with:
key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}" key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}"
restore-keys: | restore-keys: |

View File

@@ -34,7 +34,7 @@ jobs:
steps: steps:
- name: Setup Python - name: Setup Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with: with:
python-version: "${{ matrix.python-version }}" python-version: "${{ matrix.python-version }}"
@@ -44,7 +44,7 @@ jobs:
persist-credentials: "false" persist-credentials: "false"
- name: Setup cache Python - name: Setup cache Python
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache@2c8a9bd7457de244a408f35966fab2fb45fda9c8 # v6.0.0
with: with:
key: "python-${{ matrix.python-version }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}" key: "python-${{ matrix.python-version }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}"
restore-keys: | restore-keys: |
@@ -62,7 +62,7 @@ jobs:
runs-on: ubuntu-24.04-arm runs-on: ubuntu-24.04-arm
steps: steps:
- name: Setup Python - name: Setup Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with: with:
python-version: "${{ env.PYTHON_VERSION }}" python-version: "${{ env.PYTHON_VERSION }}"
@@ -77,13 +77,13 @@ jobs:
node-version-file: "./.nvmrc" node-version-file: "./.nvmrc"
- name: Setup cache Node.js - name: Setup cache Node.js
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache@2c8a9bd7457de244a408f35966fab2fb45fda9c8 # v6.0.0
with: with:
key: "nodejs-${{ runner.arch }}-${{ hashFiles('./.nvmrc', './package.json') }}" key: "nodejs-${{ runner.arch }}-${{ hashFiles('./.nvmrc', './package.json') }}"
path: "./client/simple/node_modules/" path: "./client/simple/node_modules/"
- name: Setup cache Python - name: Setup cache Python
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache@2c8a9bd7457de244a408f35966fab2fb45fda9c8 # v6.0.0
with: with:
key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}" key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}"
restore-keys: | restore-keys: |

View File

@@ -35,7 +35,7 @@ jobs:
steps: steps:
- name: Setup Python - name: Setup Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with: with:
python-version: "${{ env.PYTHON_VERSION }}" python-version: "${{ env.PYTHON_VERSION }}"
@@ -46,7 +46,7 @@ jobs:
fetch-depth: "0" fetch-depth: "0"
- name: Setup cache Python - name: Setup cache Python
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache@2c8a9bd7457de244a408f35966fab2fb45fda9c8 # v6.0.0
with: with:
key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}" key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}"
restore-keys: | restore-keys: |
@@ -83,7 +83,7 @@ jobs:
steps: steps:
- name: Setup Python - name: Setup Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0 uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with: with:
python-version: "${{ env.PYTHON_VERSION }}" python-version: "${{ env.PYTHON_VERSION }}"
@@ -94,7 +94,7 @@ jobs:
fetch-depth: "0" fetch-depth: "0"
- name: Setup cache Python - name: Setup cache Python
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5 uses: actions/cache@2c8a9bd7457de244a408f35966fab2fb45fda9c8 # v6.0.0
with: with:
key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}" key: "python-${{ env.PYTHON_VERSION }}-${{ runner.arch }}-${{ hashFiles('./requirements*.txt') }}"
restore-keys: | restore-keys: |

View File

@@ -29,7 +29,7 @@ jobs:
persist-credentials: "false" persist-credentials: "false"
- name: Sync GHCS from Docker Scout - name: Sync GHCS from Docker Scout
uses: docker/scout-action@cd72f264beff1cd72735de31148b9d3244a0234a # v1.21.0 uses: docker/scout-action@7520205ff60037fdc436b40b6a1d1e55a839ec2d # v1.22.0
with: with:
organization: "searxng" organization: "searxng"
dockerhub-user: "${{ secrets.DOCKER_USER }}" dockerhub-user: "${{ secrets.DOCKER_USER }}"

File diff suppressed because it is too large Load Diff

View File

@@ -7165,6 +7165,7 @@
"IR", "IR",
"IS", "IS",
"IT", "IT",
"JE",
"JM", "JM",
"JO", "JO",
"JP", "JP",
@@ -7194,6 +7195,7 @@
"MC", "MC",
"MD", "MD",
"ME", "ME",
"MF",
"MG", "MG",
"MH", "MH",
"MK", "MK",
@@ -7260,6 +7262,7 @@
"SS", "SS",
"ST", "ST",
"SV", "SV",
"SX",
"SY", "SY",
"SZ", "SZ",
"TC", "TC",
@@ -7407,6 +7410,7 @@
"th": "thai", "th": "thai",
"ti": "tigrinya", "ti": "tigrinya",
"tk": "turkmen", "tk": "turkmen",
"to": "tonga",
"tr": "turkish", "tr": "turkish",
"tt": "tatar", "tt": "tatar",
"uk": "ukrainian", "uk": "ukrainian",
@@ -7687,460 +7691,22 @@
"all_locale": null, "all_locale": null,
"custom": {}, "custom": {},
"data_type": "traits_v1", "data_type": "traits_v1",
"languages": { "languages": {},
"af": "afrikaans", "regions": {}
"am": "amharic",
"ar": "arabic",
"az": "azerbaijani",
"be": "belarusian",
"bg": "bulgarian",
"bn": "bengali",
"bs": "bosnian",
"ca": "catalan",
"cs": "czech",
"cy": "welsh",
"da": "dansk",
"de": "deutsch",
"el": "greek",
"en": "english",
"eo": "esperanto",
"es": "espanol",
"et": "estonian",
"eu": "basque",
"fa": "persian",
"fi": "suomi",
"fo": "faroese",
"fr": "francais",
"fy": "frisian",
"ga": "irish",
"gd": "gaelic",
"gl": "galician",
"gu": "gujarati",
"he": "hebrew",
"hi": "hindi",
"hr": "croatian",
"hu": "hungarian",
"ia": "interlingua",
"id": "indonesian",
"is": "icelandic",
"it": "italiano",
"ja": "nihongo",
"jv": "javanese",
"ka": "georgian",
"kn": "kannada",
"ko": "hangul",
"la": "latin",
"lt": "lithuanian",
"lv": "latvian",
"mai": "bihari",
"mk": "macedonian",
"ml": "malayalam",
"mr": "marathi",
"ms": "malay",
"mt": "maltese",
"nb": "norsk",
"ne": "nepali",
"nl": "nederlands",
"oc": "occitan",
"pa": "punjabi",
"pl": "polski",
"pt": "portugues",
"pt_BR": "brazilian",
"ro": "romanian",
"ru": "russian",
"si": "sinhalese",
"sk": "slovak",
"sl": "slovenian",
"sq": "albanian",
"sr": "serbian",
"su": "sudanese",
"sv": "svenska",
"sw": "swahili",
"ta": "tamil",
"te": "telugu",
"th": "thai",
"ti": "tigrinya",
"tl": "tagalog",
"tr": "turkce",
"uk": "ukrainian",
"ur": "urdu",
"uz": "uzbek",
"vi": "vietnamese",
"xh": "xhosa",
"zh": "jiantizhongwen",
"zh_Hant": "fantizhengwen",
"zu": "zulu"
},
"regions": {
"ar-EG": "ar_EG",
"bg-BG": "bg_BG",
"ca-ES": "ca_ES",
"cs-CZ": "cs_CZ",
"da-DK": "da_DK",
"de-AT": "de_AT",
"de-CH": "de_CH",
"de-DE": "de_DE",
"el-GR": "el_GR",
"en-AU": "en_AU",
"en-CA": "en_CA",
"en-GB": "en-GB_GB",
"en-ID": "en_ID",
"en-IE": "en_IE",
"en-IN": "en_IN",
"en-MY": "en_MY",
"en-NZ": "en_NZ",
"en-PH": "en_PH",
"en-SG": "en_SG",
"en-US": "en_US",
"en-ZA": "en_ZA",
"es-AR": "es_AR",
"es-CL": "es_CL",
"es-CO": "es_CO",
"es-ES": "es_ES",
"es-MX": "es_MX",
"es-PE": "es_PE",
"es-US": "es_US",
"es-UY": "es_UY",
"es-VE": "es_VE",
"et-EE": "et_EE",
"fi-FI": "fi_FI",
"fil-PH": "fil_PH",
"fr-BE": "fr_BE",
"fr-CA": "fr_CA",
"fr-CH": "fr_CH",
"fr-FR": "fr_FR",
"hi-IN": "hi_IN",
"hu-HU": "hu_HU",
"id-ID": "id_ID",
"it-CH": "it_CH",
"it-IT": "it_IT",
"ja-JP": "ja_JP",
"ko-KR": "ko_KR",
"ms-MY": "ms_MY",
"ms-SG": "ms_SG",
"nb-NO": "no_NO",
"nl-BE": "nl_BE",
"nl-NL": "nl_NL",
"pl-PL": "pl_PL",
"pt-BR": "pt-BR_BR",
"pt-PT": "pt_PT",
"ro-RO": "ro_RO",
"ru-BY": "ru_BY",
"ru-RU": "ru_RU",
"sv-SE": "sv_SE",
"tr-TR": "tr_TR",
"uk-UA": "uk_UA",
"vi-VN": "vi_VN",
"zh-CN": "zh-CN_CN",
"zh-HK": "zh-TW_HK",
"zh-TW": "zh-TW_TW"
}
}, },
"startpage images": { "startpage images": {
"all_locale": null, "all_locale": null,
"custom": {}, "custom": {},
"data_type": "traits_v1", "data_type": "traits_v1",
"languages": { "languages": {},
"af": "afrikaans", "regions": {}
"am": "amharic",
"ar": "arabic",
"az": "azerbaijani",
"be": "belarusian",
"bg": "bulgarian",
"bn": "bengali",
"bs": "bosnian",
"ca": "catalan",
"cs": "czech",
"cy": "welsh",
"da": "dansk",
"de": "deutsch",
"el": "greek",
"en": "english",
"eo": "esperanto",
"es": "espanol",
"et": "estonian",
"eu": "basque",
"fa": "persian",
"fi": "suomi",
"fo": "faroese",
"fr": "francais",
"fy": "frisian",
"ga": "irish",
"gd": "gaelic",
"gl": "galician",
"gu": "gujarati",
"he": "hebrew",
"hi": "hindi",
"hr": "croatian",
"hu": "hungarian",
"ia": "interlingua",
"id": "indonesian",
"is": "icelandic",
"it": "italiano",
"ja": "nihongo",
"jv": "javanese",
"ka": "georgian",
"kn": "kannada",
"ko": "hangul",
"la": "latin",
"lt": "lithuanian",
"lv": "latvian",
"mai": "bihari",
"mk": "macedonian",
"ml": "malayalam",
"mr": "marathi",
"ms": "malay",
"mt": "maltese",
"nb": "norsk",
"ne": "nepali",
"nl": "nederlands",
"oc": "occitan",
"pa": "punjabi",
"pl": "polski",
"pt": "portugues",
"pt_BR": "brazilian",
"ro": "romanian",
"ru": "russian",
"si": "sinhalese",
"sk": "slovak",
"sl": "slovenian",
"sq": "albanian",
"sr": "serbian",
"su": "sudanese",
"sv": "svenska",
"sw": "swahili",
"ta": "tamil",
"te": "telugu",
"th": "thai",
"ti": "tigrinya",
"tl": "tagalog",
"tr": "turkce",
"uk": "ukrainian",
"ur": "urdu",
"uz": "uzbek",
"vi": "vietnamese",
"xh": "xhosa",
"zh": "jiantizhongwen",
"zh_Hant": "fantizhengwen",
"zu": "zulu"
},
"regions": {
"ar-EG": "ar_EG",
"bg-BG": "bg_BG",
"ca-ES": "ca_ES",
"cs-CZ": "cs_CZ",
"da-DK": "da_DK",
"de-AT": "de_AT",
"de-CH": "de_CH",
"de-DE": "de_DE",
"el-GR": "el_GR",
"en-AU": "en_AU",
"en-CA": "en_CA",
"en-GB": "en-GB_GB",
"en-ID": "en_ID",
"en-IE": "en_IE",
"en-IN": "en_IN",
"en-MY": "en_MY",
"en-NZ": "en_NZ",
"en-PH": "en_PH",
"en-SG": "en_SG",
"en-US": "en_US",
"en-ZA": "en_ZA",
"es-AR": "es_AR",
"es-CL": "es_CL",
"es-CO": "es_CO",
"es-ES": "es_ES",
"es-MX": "es_MX",
"es-PE": "es_PE",
"es-US": "es_US",
"es-UY": "es_UY",
"es-VE": "es_VE",
"et-EE": "et_EE",
"fi-FI": "fi_FI",
"fil-PH": "fil_PH",
"fr-BE": "fr_BE",
"fr-CA": "fr_CA",
"fr-CH": "fr_CH",
"fr-FR": "fr_FR",
"hi-IN": "hi_IN",
"hu-HU": "hu_HU",
"id-ID": "id_ID",
"it-CH": "it_CH",
"it-IT": "it_IT",
"ja-JP": "ja_JP",
"ko-KR": "ko_KR",
"ms-MY": "ms_MY",
"ms-SG": "ms_SG",
"nb-NO": "no_NO",
"nl-BE": "nl_BE",
"nl-NL": "nl_NL",
"pl-PL": "pl_PL",
"pt-BR": "pt-BR_BR",
"pt-PT": "pt_PT",
"ro-RO": "ro_RO",
"ru-BY": "ru_BY",
"ru-RU": "ru_RU",
"sv-SE": "sv_SE",
"tr-TR": "tr_TR",
"uk-UA": "uk_UA",
"vi-VN": "vi_VN",
"zh-CN": "zh-CN_CN",
"zh-HK": "zh-TW_HK",
"zh-TW": "zh-TW_TW"
}
}, },
"startpage news": { "startpage news": {
"all_locale": null, "all_locale": null,
"custom": {}, "custom": {},
"data_type": "traits_v1", "data_type": "traits_v1",
"languages": { "languages": {},
"af": "afrikaans", "regions": {}
"am": "amharic",
"ar": "arabic",
"az": "azerbaijani",
"be": "belarusian",
"bg": "bulgarian",
"bn": "bengali",
"bs": "bosnian",
"ca": "catalan",
"cs": "czech",
"cy": "welsh",
"da": "dansk",
"de": "deutsch",
"el": "greek",
"en": "english",
"eo": "esperanto",
"es": "espanol",
"et": "estonian",
"eu": "basque",
"fa": "persian",
"fi": "suomi",
"fo": "faroese",
"fr": "francais",
"fy": "frisian",
"ga": "irish",
"gd": "gaelic",
"gl": "galician",
"gu": "gujarati",
"he": "hebrew",
"hi": "hindi",
"hr": "croatian",
"hu": "hungarian",
"ia": "interlingua",
"id": "indonesian",
"is": "icelandic",
"it": "italiano",
"ja": "nihongo",
"jv": "javanese",
"ka": "georgian",
"kn": "kannada",
"ko": "hangul",
"la": "latin",
"lt": "lithuanian",
"lv": "latvian",
"mai": "bihari",
"mk": "macedonian",
"ml": "malayalam",
"mr": "marathi",
"ms": "malay",
"mt": "maltese",
"nb": "norsk",
"ne": "nepali",
"nl": "nederlands",
"oc": "occitan",
"pa": "punjabi",
"pl": "polski",
"pt": "portugues",
"pt_BR": "brazilian",
"ro": "romanian",
"ru": "russian",
"si": "sinhalese",
"sk": "slovak",
"sl": "slovenian",
"sq": "albanian",
"sr": "serbian",
"su": "sudanese",
"sv": "svenska",
"sw": "swahili",
"ta": "tamil",
"te": "telugu",
"th": "thai",
"ti": "tigrinya",
"tl": "tagalog",
"tr": "turkce",
"uk": "ukrainian",
"ur": "urdu",
"uz": "uzbek",
"vi": "vietnamese",
"xh": "xhosa",
"zh": "jiantizhongwen",
"zh_Hant": "fantizhengwen",
"zu": "zulu"
},
"regions": {
"ar-EG": "ar_EG",
"bg-BG": "bg_BG",
"ca-ES": "ca_ES",
"cs-CZ": "cs_CZ",
"da-DK": "da_DK",
"de-AT": "de_AT",
"de-CH": "de_CH",
"de-DE": "de_DE",
"el-GR": "el_GR",
"en-AU": "en_AU",
"en-CA": "en_CA",
"en-GB": "en-GB_GB",
"en-ID": "en_ID",
"en-IE": "en_IE",
"en-IN": "en_IN",
"en-MY": "en_MY",
"en-NZ": "en_NZ",
"en-PH": "en_PH",
"en-SG": "en_SG",
"en-US": "en_US",
"en-ZA": "en_ZA",
"es-AR": "es_AR",
"es-CL": "es_CL",
"es-CO": "es_CO",
"es-ES": "es_ES",
"es-MX": "es_MX",
"es-PE": "es_PE",
"es-US": "es_US",
"es-UY": "es_UY",
"es-VE": "es_VE",
"et-EE": "et_EE",
"fi-FI": "fi_FI",
"fil-PH": "fil_PH",
"fr-BE": "fr_BE",
"fr-CA": "fr_CA",
"fr-CH": "fr_CH",
"fr-FR": "fr_FR",
"hi-IN": "hi_IN",
"hu-HU": "hu_HU",
"id-ID": "id_ID",
"it-CH": "it_CH",
"it-IT": "it_IT",
"ja-JP": "ja_JP",
"ko-KR": "ko_KR",
"ms-MY": "ms_MY",
"ms-SG": "ms_SG",
"nb-NO": "no_NO",
"nl-BE": "nl_BE",
"nl-NL": "nl_NL",
"pl-PL": "pl_PL",
"pt-BR": "pt-BR_BR",
"pt-PT": "pt_PT",
"ro-RO": "ro_RO",
"ru-BY": "ru_BY",
"ru-RU": "ru_RU",
"sv-SE": "sv_SE",
"tr-TR": "tr_TR",
"uk-UA": "uk_UA",
"vi-VN": "vi_VN",
"zh-CN": "zh-CN_CN",
"zh-HK": "zh-TW_HK",
"zh-TW": "zh-TW_TW"
}
}, },
"wikidata": { "wikidata": {
"all_locale": null, "all_locale": null,
@@ -8394,6 +7960,7 @@
"inh", "inh",
"io", "io",
"is", "is",
"isv",
"it", "it",
"iu", "iu",
"ja", "ja",
@@ -8442,6 +8009,7 @@
"ltg", "ltg",
"lv", "lv",
"mad", "mad",
"mag",
"mai", "mai",
"map-bms", "map-bms",
"mdf", "mdf",
@@ -9585,4 +9153,4 @@
}, },
"regions": {} "regions": {}
} }
} }

View File

@@ -207,6 +207,7 @@ class EngineAbout(msgspec.Struct, kw_only=True):
use_official_api: bool = False use_official_api: bool = False
"""SearXNG engine makes use of the official API or not""" """SearXNG engine makes use of the official API or not"""
require_api_key: bool = False require_api_key: bool = False
"""API requires a key or not.""" """API requires a key or not."""
@@ -217,8 +218,7 @@ class EngineAbout(msgspec.Struct, kw_only=True):
"""Brief description of the engine and where it gets its data from. """Brief description of the engine and where it gets its data from.
This value should only be set as long as no description of the data source This value should only be set as long as no description of the data source
is available via a :py:obj:`EngineAbout.wikidata_id`. is available via a :py:obj:`EngineAbout.wikidata_id`."""
"""
language: str = "" language: str = ""
"""Deprecated! Migrate your setting from `engine.about.language` to """Deprecated! Migrate your setting from `engine.about.language` to

View File

@@ -19,11 +19,9 @@ Configuration
The engine has the following additional settings: The engine has the following additional settings:
- :py:obj:`chinaso_category` (:py:obj:`ChinasoCategoryType`)
- :py:obj:`chinaso_news_source` (:py:obj:`ChinasoNewsSourceType`) - :py:obj:`chinaso_news_source` (:py:obj:`ChinasoNewsSourceType`)
In the example below, all three ChinaSO engines are using the :ref:`network In the example below, ChinaSO is configured for news search.
<engine network>` from the ``chinaso news`` engine.
.. code:: yaml .. code:: yaml
@@ -31,23 +29,8 @@ In the example below, all three ChinaSO engines are using the :ref:`network
engine: chinaso engine: chinaso
shortcut: chinaso shortcut: chinaso
categories: [news] categories: [news]
chinaso_category: news
chinaso_news_source: all chinaso_news_source: all
- name: chinaso images
engine: chinaso
network: chinaso news
shortcut: chinasoi
categories: [images]
chinaso_category: images
- name: chinaso videos
engine: chinaso
network: chinaso news
shortcut: chinasov
categories: [videos]
chinaso_category: videos
Implementations Implementations
=============== ===============
@@ -78,19 +61,6 @@ results_per_page = 10
categories = [] categories = []
language = "zh" language = "zh"
ChinasoCategoryType = t.Literal['news', 'videos', 'images']
"""ChinaSo supports news, videos, images search.
- ``news``: search for news
- ``videos``: search for videos
- ``images``: search for images
In the category ``news`` you can additionally filter by option
:py:obj:`chinaso_news_source`.
"""
chinaso_category = 'news'
"""Configure ChinaSo category (:py:obj:`ChinasoCategoryType`)."""
ChinasoNewsSourceType = t.Literal['CENTRAL', 'LOCAL', 'BUSINESS', 'EPAPER', 'all'] ChinasoNewsSourceType = t.Literal['CENTRAL', 'LOCAL', 'BUSINESS', 'EPAPER', 'all']
"""Filtering ChinaSo-News results by source: """Filtering ChinaSo-News results by source:
@@ -109,39 +79,24 @@ base_url = "https://www.chinaso.com"
def init(_): def init(_):
if chinaso_category not in ('news', 'videos', 'images'): if chinaso_news_source not in t.get_args(ChinasoNewsSourceType):
raise ValueError(f"Unsupported category: {chinaso_category}")
if chinaso_category == 'news' and chinaso_news_source not in t.get_args(ChinasoNewsSourceType):
raise ValueError(f"Unsupported news source: {chinaso_news_source}") raise ValueError(f"Unsupported news source: {chinaso_news_source}")
def request(query, params): def request(query, params):
query_params = {"q": query} query_params = {'q': query, 'pn': params["pageno"], 'ps': results_per_page}
if time_range_dict.get(params['time_range']): if time_range_dict.get(params['time_range']):
query_params["stime"] = time_range_dict[params['time_range']] query_params["stime"] = time_range_dict[params['time_range']]
query_params["etime"] = 'now' query_params["etime"] = 'now'
category_config = {
'news': {'endpoint': '/v5/general/v1/web/search', 'params': {'pn': params["pageno"], 'ps': results_per_page}},
'images': {
'endpoint': '/v5/general/v1/search/image',
'params': {'start_index': (params["pageno"] - 1) * results_per_page, 'rn': results_per_page},
},
'videos': {
'endpoint': '/v5/general/v1/search/video',
'params': {'start_index': (params["pageno"] - 1) * results_per_page, 'rn': results_per_page},
},
}
if chinaso_news_source != 'all': if chinaso_news_source != 'all':
if chinaso_news_source == 'EPAPER': if chinaso_news_source == 'EPAPER':
category_config['news']['params']["type"] = 'EPAPER' query_params["type"] = 'EPAPER'
else: else:
category_config['news']['params']["cate"] = chinaso_news_source query_params["cate"] = chinaso_news_source
query_params.update(category_config[chinaso_category]['params']) params["url"] = f"{base_url}/v5/general/v1/web/search?{urlencode(query_params)}"
params["url"] = f"{base_url}{category_config[chinaso_category]['endpoint']}?{urlencode(query_params)}"
cookie = { cookie = {
"uid": base64.b64encode(secrets.token_bytes(16)).decode("utf-8"), "uid": base64.b64encode(secrets.token_bytes(16)).decode("utf-8"),
} }
@@ -163,12 +118,6 @@ def response(resp):
if not data["data"]: if not data["data"]:
return [] return []
parsers = {'news': parse_news, 'images': parse_images, 'videos': parse_videos}
return parsers[chinaso_category](data)
def parse_news(data):
results = [] results = []
if not data.get("data", {}).get("data"): if not data.get("data", {}).get("data"):
raise SearxEngineAPIException("Invalid response") raise SearxEngineAPIException("Invalid response")
@@ -190,47 +139,3 @@ def parse_news(data):
} }
) )
return results return results
def parse_images(data):
results = []
if not data.get("data", {}).get("arrRes"):
raise SearxEngineAPIException("Invalid response")
for entry in data["data"]["arrRes"]:
results.append(
{
'url': entry["web_url"],
'title': html_to_text(entry["title"]),
'content': html_to_text(entry.get("ImageInfo", "")),
'template': 'images.html',
'img_src': entry["url"].replace("http://", "https://"),
'thumbnail_src': entry["largeimage"].replace("http://", "https://"),
}
)
return results
def parse_videos(data):
results = []
if not data.get("data", {}).get("arrRes"):
raise SearxEngineAPIException("Invalid response")
for entry in data["data"]["arrRes"]:
published_date = None
if entry.get("VideoPubDate"):
try:
published_date = datetime.fromtimestamp(int(entry["VideoPubDate"]))
except (ValueError, TypeError):
pass
results.append(
{
'url': entry["url"],
'title': html_to_text(entry["raw_title"]),
'template': 'videos.html',
'publishedDate': published_date,
'thumbnail': entry["image_src"].replace("http://", "https://"),
}
)
return results

View File

@@ -101,7 +101,7 @@ def _image_results(doc: "ElementBase") -> EngineResults:
res.types.Image( res.types.Image(
url=extract_text(eval_xpath(result, "./@href")) or "", url=extract_text(eval_xpath(result, "./@href")) or "",
title=extract_text(eval_xpath(result, "./img/@alt")) or "", title=extract_text(eval_xpath(result, "./img/@alt")) or "",
thumbnail_src=extract_text(eval_xpath(result, "./img/@src")) or "", img_src=extract_text(eval_xpath(result, "./img/@src")) or "",
), ),
) )
) )

View File

@@ -73,7 +73,7 @@ def _obtain_session_code() -> str:
if cached_session: if cached_session:
return cached_session return cached_session
results_page = get(f"{base_url}/_internCode.aspx") results_page = get(f"{base_url}/checkCode.aspx")
doc = html.fromstring(results_page.text) doc = html.fromstring(results_page.text)
extra_data: dict[str, str] = {} extra_data: dict[str, str] = {}
@@ -107,7 +107,7 @@ def _obtain_session_code() -> str:
} }
challenge_response = post( challenge_response = post(
f"{base_url}/_internCode.aspx", f"{base_url}/checkCode.aspx",
cookies=results_page.cookies, cookies=results_page.cookies,
data=data, data=data,
) )
@@ -125,7 +125,9 @@ def request(query: str, params: "OnlineParams"):
code = _obtain_session_code() code = _obtain_session_code()
args = {"w": query, "page": params["pageno"]} args = {"w": query, "page": params["pageno"]}
params["url"] = f"{base_url}/{tiger_category}?{urlencode(args)}" params["url"] = f"{base_url}/{tiger_category}?{urlencode(args)}"
params["cookies"]["Tiger.ch"] = f"Code={code}" # Setting Checked=1 shows related search terms / suggestions
# Language and country could be set with Lng= and Land= in the future
params["cookies"]["Tiger.ch"] = f"Tiger.ch=&Code={code}&Checked=1"
def response(resp: "SXNG_Response") -> EngineResults: def response(resp: "SXNG_Response") -> EngineResults:
@@ -144,6 +146,9 @@ def response(resp: "SXNG_Response") -> EngineResults:
content=extract_text(eval_xpath(result, ".//*[contains(@class, 'webbodynopic')]")) or "", content=extract_text(eval_xpath(result, ".//*[contains(@class, 'webbodynopic')]")) or "",
) )
) )
for suggestion in eval_xpath_list(doc, "//a[contains(@class, 'linkAnders')]"):
res.add(res.types.LegacyResult(suggestion=extract_text(suggestion)))
elif tiger_category == "News": elif tiger_category == "News":
for result in eval_xpath_list(doc, "//div[@id='panNews']/div"): for result in eval_xpath_list(doc, "//div[@id='panNews']/div"):
publishedDate = None publishedDate = None

View File

@@ -14,6 +14,7 @@ template.
# pylint: disable=too-few-public-methods # pylint: disable=too-few-public-methods
__all__ = ["Image", "ImageRef"] __all__ = ["Image", "ImageRef"]
import mimetypes
import types import types
import typing as t import typing as t
from collections.abc import Callable from collections.abc import Callable
@@ -71,7 +72,7 @@ class Image(MainResult, kw_only=True):
"""The resolution of the image (e.g. ``1920 x 1080`` pixel)""" """The resolution of the image (e.g. ``1920 x 1080`` pixel)"""
img_format: str = "" img_format: str = ""
"""The format of the image (e.g. ``png``).""" """The format of the image :py:obj:`.MainResult.img_src` (e.g. ``png``)."""
source: str = "" source: str = ""
"""Source of the image.""" """Source of the image."""
@@ -83,6 +84,19 @@ class Image(MainResult, kw_only=True):
formats: list[ImageRef] = [] formats: list[ImageRef] = []
"""List of links to alternative image formats.""" """List of links to alternative image formats."""
def __post_init__(self):
super().__post_init__()
if not self.img_format:
# automatically guess the image format based on the path of the image
mimetype = mimetypes.guess_type(self.img_src)[0]
if mimetype:
subtype = mimetype.split("/")[-1]
if subtype in MIMESUB:
self.img_format = MIMESUB[subtype]
else:
self.img_format = subtype.upper()
def filter_urls(self, filter_func: "Callable[[Result | LegacyResult, str, str], str | bool ]"): def filter_urls(self, filter_func: "Callable[[Result | LegacyResult, str, str], str | bool ]"):
for _ref in self.formats[:]: for _ref in self.formats[:]:

View File

@@ -443,6 +443,12 @@ engines:
timeout: 6.0 timeout: 6.0
shortcut: conda shortcut: conda
disabled: true disabled: true
about:
website: https://anaconda.org/search
wikidata_id: Q18209310
official_api_documentation: https://api.anaconda.org/search
use_official_api: false
results: HTML
- name: arch linux wiki - name: arch linux wiki
engine: archlinux engine: archlinux
@@ -644,29 +650,10 @@ engines:
engine: chinaso engine: chinaso
shortcut: chinaso shortcut: chinaso
categories: [news] categories: [news]
chinaso_category: news
chinaso_news_source: all chinaso_news_source: all
disabled: true disabled: true
inactive: true inactive: true
- name: chinaso images
engine: chinaso
network: chinaso news
shortcut: chinasoi
categories: [images]
chinaso_category: images
disabled: true
inactive: true
- name: chinaso videos
engine: chinaso
network: chinaso news
shortcut: chinasov
categories: [videos]
chinaso_category: videos
disabled: true
inactive: true
- name: cl0q - name: cl0q
engine: json_engine engine: json_engine
shortcut: cl shortcut: cl
@@ -1424,12 +1411,10 @@ engines:
- name: kozmonavt - name: kozmonavt
engine: xpath engine: xpath
paging: true search_url: https://kozmonavt.su/s?q={query}
# pageno query param key is U+1D5D1 (looks like an "x")
search_url: https://kozmonavt.su/s?q={query}&%F0%9D%97%91={pageno}
shortcut: koz shortcut: koz
disabled: true disabled: true
# inactive: true inactive: true
results_xpath: //div[contains(@class, 'list')]/section results_xpath: //div[contains(@class, 'list')]/section
url_xpath: concat('https://', substring-after(.//a/@href, '?q=')) url_xpath: concat('https://', substring-after(.//a/@href, '?q='))
title_xpath: .//a title_xpath: .//a

View File

@@ -9,7 +9,6 @@
''' '''
sxng_locales = ( sxng_locales = (
('af', 'Afrikaans', '', 'Afrikaans', '\U0001f310'),
('ar', 'العربية', '', 'Arabic', '\U0001f310'), ('ar', 'العربية', '', 'Arabic', '\U0001f310'),
('ar-SA', 'العربية', 'المملكة العربية السعودية', 'Arabic', '\U0001f1f8\U0001f1e6'), ('ar-SA', 'العربية', 'المملكة العربية السعودية', 'Arabic', '\U0001f1f8\U0001f1e6'),
('bg', 'Български', '', 'Bulgarian', '\U0001f310'), ('bg', 'Български', '', 'Bulgarian', '\U0001f310'),
@@ -46,7 +45,6 @@ sxng_locales = (
('es-ES', 'Español', 'España', 'Spanish', '\U0001f1ea\U0001f1f8'), ('es-ES', 'Español', 'España', 'Spanish', '\U0001f1ea\U0001f1f8'),
('es-MX', 'Español', 'México', 'Spanish', '\U0001f1f2\U0001f1fd'), ('es-MX', 'Español', 'México', 'Spanish', '\U0001f1f2\U0001f1fd'),
('es-PE', 'Español', 'Perú', 'Spanish', '\U0001f1f5\U0001f1ea'), ('es-PE', 'Español', 'Perú', 'Spanish', '\U0001f1f5\U0001f1ea'),
('es-VE', 'Español', 'Venezuela', 'Spanish', '\U0001f1fb\U0001f1ea'),
('et', 'Eesti', '', 'Estonian', '\U0001f310'), ('et', 'Eesti', '', 'Estonian', '\U0001f310'),
('et-EE', 'Eesti', 'Eesti', 'Estonian', '\U0001f1ea\U0001f1ea'), ('et-EE', 'Eesti', 'Eesti', 'Estonian', '\U0001f1ea\U0001f1ea'),
('fi', 'Suomi', '', 'Finnish', '\U0001f310'), ('fi', 'Suomi', '', 'Finnish', '\U0001f310'),
@@ -58,7 +56,6 @@ sxng_locales = (
('fr-CA', 'Français', 'Canada', 'French', '\U0001f1e8\U0001f1e6'), ('fr-CA', 'Français', 'Canada', 'French', '\U0001f1e8\U0001f1e6'),
('fr-CH', 'Français', 'Suisse', 'French', '\U0001f1e8\U0001f1ed'), ('fr-CH', 'Français', 'Suisse', 'French', '\U0001f1e8\U0001f1ed'),
('fr-FR', 'Français', 'France', 'French', '\U0001f1eb\U0001f1f7'), ('fr-FR', 'Français', 'France', 'French', '\U0001f1eb\U0001f1f7'),
('gl', 'Galego', '', 'Galician', '\U0001f310'),
('hi', 'हिन्दी', '', 'Hindi', '\U0001f310'), ('hi', 'हिन्दी', '', 'Hindi', '\U0001f310'),
('hi-IN', 'हिन्दी', 'भारत', 'Hindi', '\U0001f1ee\U0001f1f3'), ('hi-IN', 'हिन्दी', 'भारत', 'Hindi', '\U0001f1ee\U0001f1f3'),
('hr', 'Hrvatski', '', 'Croatian', '\U0001f310'), ('hr', 'Hrvatski', '', 'Croatian', '\U0001f310'),
@@ -74,6 +71,8 @@ sxng_locales = (
('ja-JP', '日本語', '日本', 'Japanese', '\U0001f1ef\U0001f1f5'), ('ja-JP', '日本語', '日本', 'Japanese', '\U0001f1ef\U0001f1f5'),
('ko', '한국어', '', 'Korean', '\U0001f310'), ('ko', '한국어', '', 'Korean', '\U0001f310'),
('ko-KR', '한국어', '대한민국', 'Korean', '\U0001f1f0\U0001f1f7'), ('ko-KR', '한국어', '대한민국', 'Korean', '\U0001f1f0\U0001f1f7'),
('mi', 'Māori', '', 'Māori', '\U0001f310'),
('mi-NZ', 'Māori', 'Aotearoa', 'Māori', '\U0001f1f3\U0001f1ff'),
('nb', 'Norsk Bokmål', '', 'Norwegian Bokmål', '\U0001f310'), ('nb', 'Norsk Bokmål', '', 'Norwegian Bokmål', '\U0001f310'),
('nb-NO', 'Norsk Bokmål', 'Norge', 'Norwegian Bokmål', '\U0001f1f3\U0001f1f4'), ('nb-NO', 'Norsk Bokmål', 'Norge', 'Norwegian Bokmål', '\U0001f1f3\U0001f1f4'),
('nl', 'Nederlands', '', 'Dutch', '\U0001f310'), ('nl', 'Nederlands', '', 'Dutch', '\U0001f310'),
@@ -90,8 +89,6 @@ sxng_locales = (
('ro-RO', 'Română', 'România', 'Romanian', '\U0001f1f7\U0001f1f4'), ('ro-RO', 'Română', 'România', 'Romanian', '\U0001f1f7\U0001f1f4'),
('ru', 'Русский', '', 'Russian', '\U0001f310'), ('ru', 'Русский', '', 'Russian', '\U0001f310'),
('ru-RU', 'Русский', 'Россия', 'Russian', '\U0001f1f7\U0001f1fa'), ('ru-RU', 'Русский', 'Россия', 'Russian', '\U0001f1f7\U0001f1fa'),
('sk', 'Slovenčina', '', 'Slovak', '\U0001f310'),
('sq', 'Shqip', '', 'Albanian', '\U0001f310'),
('sv', 'Svenska', '', 'Swedish', '\U0001f310'), ('sv', 'Svenska', '', 'Swedish', '\U0001f310'),
('sv-FI', 'Svenska', 'Finland', 'Swedish', '\U0001f1eb\U0001f1ee'), ('sv-FI', 'Svenska', 'Finland', 'Swedish', '\U0001f1eb\U0001f1ee'),
('sv-SE', 'Svenska', 'Sverige', 'Swedish', '\U0001f1f8\U0001f1ea'), ('sv-SE', 'Svenska', 'Sverige', 'Swedish', '\U0001f1f8\U0001f1ea'),
@@ -99,7 +96,6 @@ sxng_locales = (
('th-TH', 'ไทย', 'ไทย', 'Thai', '\U0001f1f9\U0001f1ed'), ('th-TH', 'ไทย', 'ไทย', 'Thai', '\U0001f1f9\U0001f1ed'),
('tr', 'Türkçe', '', 'Turkish', '\U0001f310'), ('tr', 'Türkçe', '', 'Turkish', '\U0001f310'),
('tr-TR', 'Türkçe', 'Türkiye', 'Turkish', '\U0001f1f9\U0001f1f7'), ('tr-TR', 'Türkçe', 'Türkiye', 'Turkish', '\U0001f1f9\U0001f1f7'),
('uk', 'Українська', '', 'Ukrainian', '\U0001f310'),
('vi', 'Tiếng Việt', '', 'Vietnamese', '\U0001f310'), ('vi', 'Tiếng Việt', '', 'Vietnamese', '\U0001f310'),
('vi-VN', 'Tiếng Việt', 'Việt Nam', 'Vietnamese', '\U0001f1fb\U0001f1f3'), ('vi-VN', 'Tiếng Việt', 'Việt Nam', 'Vietnamese', '\U0001f1fb\U0001f1f3'),
('zh', '中文', '', 'Chinese', '\U0001f310'), ('zh', '中文', '', 'Chinese', '\U0001f310'),

View File

@@ -33,22 +33,24 @@
# IcewindX <icewindx@noreply.codeberg.org>, 2025. # IcewindX <icewindx@noreply.codeberg.org>, 2025.
# 0ko <0ko@noreply.codeberg.org>, 2025. # 0ko <0ko@noreply.codeberg.org>, 2025.
# greatdng <greatdng@noreply.codeberg.org>, 2026. # greatdng <greatdng@noreply.codeberg.org>, 2026.
# code_gremlin <code_gremlin@noreply.codeberg.org>, 2026.
msgid "" msgid ""
msgstr "" msgstr ""
"Project-Id-Version: searx\n" "Project-Id-Version: searx\n"
"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n" "Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
"POT-Creation-Date: 2026-06-13 11:31+0000\n" "POT-Creation-Date: 2026-06-13 11:31+0000\n"
"PO-Revision-Date: 2026-05-19 12:08+0000\n" "PO-Revision-Date: 2026-06-24 10:07+0000\n"
"Last-Translator: return42 <return42@noreply.codeberg.org>\n" "Last-Translator: code_gremlin <code_gremlin@noreply.codeberg.org>\n"
"Language-Team: Russian <https://translate.codeberg.org/projects/searxng/"
"searxng/ru/>\n"
"Language: ru\n" "Language: ru\n"
"Language-Team: Russian "
"<https://translate.codeberg.org/projects/searxng/searxng/ru/>\n"
"Plural-Forms: nplurals=4; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && "
"n%10<=4 && (n%100<12 || n%100>14) ? 1 : n%10==0 || (n%10>=5 && n%10<=9) "
"|| (n%100>=11 && n%100<=14)? 2 : 3);\n"
"MIME-Version: 1.0\n" "MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n" "Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n" "Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=4; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && "
"n%10<=4 && (n%100<12 || n%100>14) ? 1 : n%10==0 || (n%10>=5 && n%10<=9) || "
"(n%100>=11 && n%100<=14)? 2 : 3);\n"
"X-Generator: Weblate 2026.6.1\n"
"Generated-By: Babel 2.18.0\n" "Generated-By: Babel 2.18.0\n"
#. CONSTANT_NAMES['NO_SUBGROUPING'] #. CONSTANT_NAMES['NO_SUBGROUPING']
@@ -1667,11 +1669,11 @@ msgstr "Разрешение"
#: searx/templates/simple/result_templates/images.html:55 #: searx/templates/simple/result_templates/images.html:55
msgid "Image formats" msgid "Image formats"
msgstr "" msgstr "Форматы изображений"
#: searx/templates/simple/result_templates/images.html:56 #: searx/templates/simple/result_templates/images.html:56
msgid "original format" msgid "original format"
msgstr "" msgstr "формат оригинала"
#: searx/templates/simple/result_templates/images.html:63 #: searx/templates/simple/result_templates/images.html:63
msgid "Engine" msgid "Engine"
@@ -2441,4 +2443,3 @@ msgstr "скрыть видео"
#~ msgid "Format" #~ msgid "Format"
#~ msgstr "Формат" #~ msgstr "Формат"

View File

@@ -15,25 +15,26 @@ from os.path import join
from lxml.html import fromstring from lxml.html import fromstring
import searx.engines
from searx.engines import wikidata, set_loggers from searx.engines import wikidata, set_loggers
from searx.utils import extract_text, searxng_useragent from searx.utils import extract_text
from searx.locales import LOCALE_NAMES, locales_initialize, match_locale from searx.locales import LOCALE_NAMES, locales_initialize, match_locale
from searx import searx_dir from searx import searx_dir
from searx.utils import gen_useragent from searx.utils import gen_useragent
import searx.search import searx.search
import searx.network import searx.network
from searx.data import data_dir from searx.data import data_dir, ENGINE_DESCRIPTIONS
DATA_FILE = data_dir / 'engine_descriptions.json' DATA_FILE = data_dir / "engine_descriptions.json"
set_loggers(wikidata, 'wikidata') set_loggers(wikidata, "wikidata")
locales_initialize() locales_initialize()
# you can run the query in https://query.wikidata.org # you can run the query in https://query.wikidata.org
# replace %IDS% by Wikidata entities separated by spaces with the prefix wd: # replace %IDS% by Wikidata entities separated by spaces with the prefix wd:
# for example wd:Q182496 wd:Q1540899 # for example wd:Q182496 wd:Q1540899
# replace %LANGUAGES_SPARQL% by languages # replace %LANGUAGES_SPARQL% by languages
SPARQL_WIKIPEDIA_ARTICLE = """ SPARQL_WIKIPEDIA_ARTICLE: str = """
SELECT DISTINCT ?item ?name ?article ?lang SELECT DISTINCT ?item ?name ?article ?lang
WHERE { WHERE {
hint:Query hint:optimizer "None". hint:Query hint:optimizer "None".
@@ -48,7 +49,7 @@ WHERE {
ORDER BY ?item ?lang ORDER BY ?item ?lang
""" """
SPARQL_DESCRIPTION = """ SPARQL_DESCRIPTION: str = """
SELECT DISTINCT ?item ?itemDescription SELECT DISTINCT ?item ?itemDescription
WHERE { WHERE {
VALUES ?item { %IDS% } VALUES ?item { %IDS% }
@@ -58,43 +59,43 @@ WHERE {
ORDER BY ?itemLang ORDER BY ?itemLang
""" """
NOT_A_DESCRIPTION = [ NOT_A_DESCRIPTION: list[str] = [
'web site', "web site",
'site web', "site web",
'komputa serĉilo', "komputa serĉilo",
'interreta serĉilo', "interreta serĉilo",
'bilaketa motor', "bilaketa motor",
'web search engine', "web search engine",
'wikimedia täpsustuslehekülg', "wikimedia täpsustuslehekülg",
] ]
SKIP_ENGINE_SOURCE = [ SKIP_ENGINE_SOURCE: list[tuple[str, str]] = [
# fmt: off # fmt: off
('gitlab', 'wikidata') ("gitlab", "wikidata")
# descriptions are about wikipedia disambiguation pages # descriptions are about wikipedia disambiguation pages
# fmt: on # fmt: on
] ]
WIKIPEDIA_LANGUAGES = {} WIKIPEDIA_LANGUAGES: dict[str, str] = {}
LANGUAGES_SPARQL = '' LANGUAGES_SPARQL: str = ""
IDS = None IDS: str = ""
WIKIPEDIA_LANGUAGE_VARIANTS = {'zh_Hant': 'zh-tw'} WIKIPEDIA_LANGUAGE_VARIANTS: dict[str, str] = {"zh_Hant": "zh-tw"}
# descriptions[engine][language] = [description, source]
descriptions: dict[str, dict[str, list[str]]] = {}
wd_to_engine_name: dict[str, set[str]] = {}
descriptions = {} def normalize_description(description: str):
wd_to_engine_name = {}
def normalize_description(description):
for c in [chr(c) for c in range(0, 31)]: for c in [chr(c) for c in range(0, 31)]:
description = description.replace(c, ' ') description = description.replace(c, " ")
description = ' '.join(description.strip().split()) description = " ".join(description.strip().split())
return description return description
def update_description(engine_name, lang, description, source, replace=True): def update_description(engine_name: str, lang: str, description: str, source: str, replace: bool = True) -> None:
if not isinstance(description, str): if not isinstance(description, str):
return return # pyright: ignore[reportUnreachable]
description = normalize_description(description) description = normalize_description(description)
if description.lower() == engine_name.lower(): if description.lower() == engine_name.lower():
return return
@@ -102,54 +103,71 @@ def update_description(engine_name, lang, description, source, replace=True):
return return
if (engine_name, source) in SKIP_ENGINE_SOURCE: if (engine_name, source) in SKIP_ENGINE_SOURCE:
return return
if ' ' not in description: if " " not in description:
# skip unique word description (like "website") # skip unique word description (like "website")
return return
if replace or lang not in descriptions[engine_name]: if replace or lang not in descriptions[engine_name]:
descriptions[engine_name][lang] = [description, source] descriptions[engine_name][lang] = [description, source]
def get_wikipedia_summary(wikipedia_url, searxng_locale): def get_wikipedia_summary(wikipedia_url: str, searxng_locale: str):
# get the REST API URL from the HTML URL # get the REST API URL from the HTML URL
# Headers # Headers
headers = {'User-Agent': searxng_useragent()} headers = {
"User-Agent": gen_useragent(),
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Accept-Encoding": "gzip, deflate",
"DNT": "1",
"Upgrade-Insecure-Requests": "1",
"Sec-GPC": "1",
"Cache-Control": "max-age=0",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "same-origin",
"Sec-Fetch-User": "?1",
}
if searxng_locale in WIKIPEDIA_LANGUAGE_VARIANTS: if searxng_locale in WIKIPEDIA_LANGUAGE_VARIANTS:
headers['Accept-Language'] = WIKIPEDIA_LANGUAGE_VARIANTS.get(searxng_locale) headers["Accept-Language"] = WIKIPEDIA_LANGUAGE_VARIANTS[searxng_locale]
# URL path : from HTML URL to REST API URL # URL path : from HTML URL to REST API URL
parsed_url = urlparse(wikipedia_url) parsed_url = urlparse(wikipedia_url)
# remove the /wiki/ prefix # remove the /wiki/ prefix
article_name = parsed_url.path.split('/wiki/')[1] article_name = parsed_url.path.split("/wiki/")[1]
# article_name is already encoded but not the / which is required for the REST API call # article_name is already encoded but not the / which is required for the REST API call
encoded_article_name = article_name.replace('/', '%2F') encoded_article_name = article_name.replace("/", "%2F")
path = '/api/rest_v1/page/summary/' + encoded_article_name path = "/api/rest_v1/page/summary/" + encoded_article_name
wikipedia_rest_url = parsed_url._replace(path=path).geturl() wikipedia_rest_url = parsed_url._replace(path=path).geturl()
try: try:
response = searx.network.get(wikipedia_rest_url, headers=headers, timeout=10) response = searx.network.get(wikipedia_rest_url, headers=headers, timeout=10)
response.raise_for_status() response.raise_for_status()
except Exception as e: # pylint: disable=broad-except except Exception as e: # pylint: disable=broad-except
print(" ", wikipedia_url, e) print(" get_wikipedia_summary: ", wikipedia_rest_url, e)
return None return None
api_result = json.loads(response.text) api_result = json.loads(response.text)
return api_result.get('extract') return api_result.get("extract")
def get_website_description(url, lang1, lang2=None): def get_website_description(url: str, lang1: str | None, lang2: str | None = None):
headers = { headers = {
'User-Agent': gen_useragent(), "User-Agent": gen_useragent(),
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
'DNT': '1', "Accept-Encoding": "gzip, deflate",
'Upgrade-Insecure-Requests': '1', "DNT": "1",
'Sec-GPC': '1', "Upgrade-Insecure-Requests": "1",
'Cache-Control': 'max-age=0', "Sec-GPC": "1",
"Cache-Control": "max-age=0",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "same-origin",
"Sec-Fetch-User": "?1",
} }
if lang1 is not None: if lang1 is not None:
lang_list = [lang1] lang_list = [lang1]
if lang2 is not None: if lang2 is not None:
lang_list.append(lang2) lang_list.append(lang2)
headers['Accept-Language'] = f'{",".join(lang_list)};q=0.8' headers["Accept-Language"] = f'{",".join(lang_list)};q=0.8'
try: try:
response = searx.network.get(url, headers=headers, timeout=10) response = searx.network.get(url, headers=headers, timeout=10)
response.raise_for_status() response.raise_for_status()
@@ -167,7 +185,7 @@ def get_website_description(url, lang1, lang2=None):
if not description: if not description:
description = extract_text(html.xpath('/html/head/title')) description = extract_text(html.xpath('/html/head/title'))
lang = extract_text(html.xpath('/html/@lang')) lang = extract_text(html.xpath('/html/@lang'))
if lang is None and len(lang1) > 0: if lang is None and lang1 and len(lang1) > 0:
lang = lang1 lang = lang1
lang = lang or 'en' lang = lang or 'en'
lang = lang.split('_')[0] lang = lang.split('_')[0]
@@ -178,15 +196,15 @@ def get_website_description(url, lang1, lang2=None):
def initialize(): def initialize():
global IDS, LANGUAGES_SPARQL global IDS, LANGUAGES_SPARQL
searx.search.initialize() searx.search.initialize()
wikipedia_engine = searx.engines.engines['wikipedia'] wikipedia_engine = searx.engines.engines["wikipedia"]
locale2lang = {'nl-BE': 'nl'} locale2lang = {"nl-BE": "nl"}
for sxng_ui_lang in LOCALE_NAMES: for sxng_ui_lang in LOCALE_NAMES:
sxng_ui_alias = locale2lang.get(sxng_ui_lang, sxng_ui_lang) sxng_ui_alias = locale2lang.get(sxng_ui_lang, sxng_ui_lang)
wiki_lang = None wiki_lang = None
if sxng_ui_alias in wikipedia_engine.traits.custom['WIKIPEDIA_LANGUAGES']: if sxng_ui_alias in wikipedia_engine.traits.custom["WIKIPEDIA_LANGUAGES"]:
wiki_lang = sxng_ui_alias wiki_lang = sxng_ui_alias
if not wiki_lang: if not wiki_lang:
wiki_lang = wikipedia_engine.traits.get_language(sxng_ui_alias) wiki_lang = wikipedia_engine.traits.get_language(sxng_ui_alias)
@@ -195,70 +213,84 @@ def initialize():
continue continue
WIKIPEDIA_LANGUAGES[sxng_ui_lang] = wiki_lang WIKIPEDIA_LANGUAGES[sxng_ui_lang] = wiki_lang
LANGUAGES_SPARQL = ', '.join(f"'{l}'" for l in set(WIKIPEDIA_LANGUAGES.values())) LANGUAGES_SPARQL = ", ".join(f"'{l}'" for l in set(WIKIPEDIA_LANGUAGES.values()))
for engine_name, engine in searx.engines.engines.items(): for engine_name, engine in searx.engines.engines.items():
descriptions[engine_name] = {} descriptions[engine_name] = {}
wikidata_id = getattr(engine, "about", {}).get('wikidata_id') if engine.about.wikidata_id:
if wikidata_id is not None: wd_to_engine_name.setdefault(engine.about.wikidata_id, set()).add(engine_name)
wd_to_engine_name.setdefault(wikidata_id, set()).add(engine_name)
IDS = ' '.join(list(map(lambda wd_id: 'wd:' + wd_id, wd_to_engine_name.keys()))) IDS = " ".join(list(map(lambda wd_id: "wd:" + wd_id, wd_to_engine_name.keys())))
def fetch_wikidata_descriptions(): def fetch_wikidata_descriptions():
print('Fetching wikidata descriptions') print("Fetching wikidata descriptions")
searx.network.set_timeout_for_thread(60) searx.network.set_timeout_for_thread(60)
result = wikidata.send_wikidata_query( result = wikidata.send_wikidata_query(
SPARQL_DESCRIPTION.replace('%IDS%', IDS).replace('%LANGUAGES_SPARQL%', LANGUAGES_SPARQL) SPARQL_DESCRIPTION.replace("%IDS%", IDS).replace("%LANGUAGES_SPARQL%", LANGUAGES_SPARQL)
) )
if result is not None: if not result:
for binding in result['results']['bindings']: print("ERROR: fetching wikiDATA descriptions - SPARQL_DESCRIPTION query without results.")
wikidata_id = binding['item']['value'].replace('http://www.wikidata.org/entity/', '') return
wikidata_lang = binding['itemDescription']['xml:lang']
desc = binding['itemDescription']['value'] for binding in result["results"]["bindings"]:
for engine_name in wd_to_engine_name[wikidata_id]: wikidata_id = binding["item"]["value"].replace("http://www.wikidata.org/entity/", "")
for searxng_locale in LOCALE_NAMES: wikidata_lang = binding["itemDescription"]["xml:lang"]
if WIKIPEDIA_LANGUAGES[searxng_locale] != wikidata_lang: desc = binding["itemDescription"]["value"]
continue for engine_name in wd_to_engine_name[wikidata_id]:
print( for searxng_locale in LOCALE_NAMES:
f" engine: {engine_name:20} / wikidata_lang: {wikidata_lang:5}", if WIKIPEDIA_LANGUAGES[searxng_locale] != wikidata_lang:
f"/ len(wikidata_desc): {len(desc)}", continue
) print(
update_description(engine_name, searxng_locale, desc, 'wikidata') f" engine: {engine_name:20} / wikidata_lang: {wikidata_lang:5}",
f"/ len(wikidata_desc): {len(desc)}",
)
update_description(engine_name, searxng_locale, desc, "wikidata")
def fetch_wikipedia_descriptions(): def fetch_wikipedia_descriptions():
print('Fetching wikipedia descriptions') print("Fetching wikipedia descriptions")
result = wikidata.send_wikidata_query( result = wikidata.send_wikidata_query(
SPARQL_WIKIPEDIA_ARTICLE.replace('%IDS%', IDS).replace('%LANGUAGES_SPARQL%', LANGUAGES_SPARQL) SPARQL_WIKIPEDIA_ARTICLE.replace("%IDS%", IDS).replace("%LANGUAGES_SPARQL%", LANGUAGES_SPARQL)
) )
if result is not None: if not result:
for binding in result['results']['bindings']: print("ERROR: fetching wikiPEDIA descriptions - SPARQL_WIKIPEDIA_ARTICLE query without results.")
wikidata_id = binding['item']['value'].replace('http://www.wikidata.org/entity/', '') return
wikidata_lang = binding['name']['xml:lang']
wikipedia_url = binding['article']['value'] # for example the URL https://de.wikipedia.org/wiki/PubMed # pylint: disable=too-many-nested-blocks
for engine_name in wd_to_engine_name[wikidata_id]: for binding in result["results"]["bindings"]:
for searxng_locale in LOCALE_NAMES: wikidata_id = binding["item"]["value"].replace("http://www.wikidata.org/entity/", "")
if WIKIPEDIA_LANGUAGES[searxng_locale] != wikidata_lang: wikidata_lang = binding["name"]["xml:lang"]
continue wikipedia_url = binding["article"]["value"] # for example the URL https://de.wikipedia.org/wiki/PubMed
desc = get_wikipedia_summary(wikipedia_url, searxng_locale) for engine_name in wd_to_engine_name[wikidata_id]:
if not desc: for searxng_locale in LOCALE_NAMES:
continue if WIKIPEDIA_LANGUAGES[searxng_locale] != wikidata_lang:
print( continue
f" engine: {engine_name:20} / wikidata_lang: {wikidata_lang:5}", desc = get_wikipedia_summary(wikipedia_url, searxng_locale)
f"/ len(wikipedia_desc): {len(desc)}", if not desc:
) if descriptions.get(searxng_locale, {}).get(engine_name) is None:
update_description(engine_name, searxng_locale, desc, 'wikipedia') _descr = ENGINE_DESCRIPTIONS.get(searxng_locale, {}).get(engine_name)
if _descr is not None:
if len(_descr) == 2 and _descr[1] == 'ref':
ref_engine, ref_lang = _descr[0].split(':')
_descr = ENGINE_DESCRIPTIONS[ref_lang][ref_engine]
update_description(engine_name, searxng_locale, _descr[0], _descr[1])
continue
print(
f" engine: {engine_name:20} / wikidata_lang: {wikidata_lang:5}",
f"/ len(wikipedia_desc): {len(desc)}",
)
update_description(engine_name, searxng_locale, desc, "wikipedia")
def normalize_url(url): def normalize_url(url: str):
url = url.replace('{language}', 'en') url = url.replace("{language}", "en")
url = urlparse(url)._replace(path='/', params='', query='', fragment='').geturl() url = urlparse(url)._replace(path="/", params="", query="", fragment="").geturl()
url = url.replace('https://api.', 'https://') url = url.replace("https://api.", "https://")
return url return url
def fetch_website_description(engine_name, website): def fetch_website_description(engine_name: str, website: str):
print(f"- fetch website descr: {engine_name} / {website}") print(f"- fetch website descr: {engine_name} / {website}")
default_lang, default_description = get_website_description(website, None, None) default_lang, default_description = get_website_description(website, None, None)
@@ -268,11 +300,11 @@ def fetch_website_description(engine_name, website):
# to specify an order in where the most common languages are in front of the # to specify an order in where the most common languages are in front of the
# language list .. # language list ..
languages = ['en', 'es', 'pt', 'ru', 'tr', 'fr'] languages = ["en", "es", "pt", "ru", "tr", "fr"]
languages = languages + [l for l in LOCALE_NAMES if l not in languages] languages = languages + [l for l in LOCALE_NAMES if l not in languages]
previous_matched_lang = None previous_matched_lang: str | None = None
previous_count = 0 previous_count: int = 0
for lang in languages: for lang in languages:
@@ -307,19 +339,15 @@ def fetch_website_description(engine_name, website):
f" / fetched lang: {fetched_lang:7} / len(desc): {len(desc)}" f" / fetched lang: {fetched_lang:7} / len(desc): {len(desc)}"
) )
matched_lang = match_locale(fetched_lang, LOCALE_NAMES.keys(), fallback=lang) matched_lang = match_locale(fetched_lang, list(LOCALE_NAMES.keys())) or lang
update_description(engine_name, matched_lang, desc, website, replace=False) update_description(engine_name, matched_lang, desc, website, replace=False)
def fetch_website_descriptions(): def fetch_website_descriptions():
print('Fetching website descriptions') print("Fetching website descriptions")
for engine_name, engine in searx.engines.engines.items(): for engine_name, engine in searx.engines.engines.items():
website = getattr(engine, "about", {}).get('website') website = engine.about.website or getattr(engine, "search_url", "") or getattr(engine, "base_url", "")
if website is None and hasattr(engine, "search_url"): if website:
website = normalize_url(getattr(engine, "search_url"))
if website is None and hasattr(engine, "base_url"):
website = normalize_url(getattr(engine, "base_url"))
if website is not None:
fetch_website_description(engine_name, website) fetch_website_description(engine_name, website)
@@ -328,30 +356,32 @@ def get_engine_descriptions_filename():
def get_output(): def get_output():
"""Summary of the results, once known descriptions are not duplicated,
instead a reference is provided.
- from: ``descriptions[engine][language] = [description, source]``
- to: ``output[language][engine] = description_and_source``
``description_and_source`` can be:
- ``[description, source]``
- ``description`` (if source = "wikipedia")
- ``[f"engine:lang", "ref"]`` reference to another existing description
""" """
From descriptions[engine][language] = [description, source] output: dict[str, dict[str, list[str] | str]] = {locale: {} for locale in LOCALE_NAMES}
To seen_descriptions: dict[str, tuple[str, str]] = {}
* output[language][engine] = description_and_source for engine_name, lang_descriptions in sorted(descriptions.items()):
* description_and_source can be: for language, descr in sorted(lang_descriptions.items()):
* [description, source] if descr[0] in seen_descriptions:
* description (if source = "wikipedia") ref = seen_descriptions[descr[0]]
* [f"engine:lang", "ref"] (reference to another existing description) descr = [f"{ref[0]}:{ref[1]}", "ref"]
"""
output = {locale: {} for locale in LOCALE_NAMES}
seen_descriptions = {}
for engine_name, lang_descriptions in descriptions.items():
for language, description in lang_descriptions.items():
if description[0] in seen_descriptions:
ref = seen_descriptions[description[0]]
description = [f'{ref[0]}:{ref[1]}', 'ref']
else: else:
seen_descriptions[description[0]] = (engine_name, language) seen_descriptions[descr[0]] = (engine_name, language)
if description[1] == 'wikipedia': if descr[1] == "wikipedia":
description = description[0] descr = descr[0]
output.setdefault(language, {}).setdefault(engine_name, description) output.setdefault(language, {}).setdefault(engine_name, descr)
return output return output
@@ -363,8 +393,8 @@ def main():
fetch_website_descriptions() fetch_website_descriptions()
output = get_output() output = get_output()
with DATA_FILE.open('w', encoding='utf8') as f: with DATA_FILE.open("w", encoding="utf8") as f:
f.write(json.dumps(output, indent=1, separators=(',', ':'), sort_keys=True, ensure_ascii=False)) f.write(json.dumps(output, indent=1, separators=(",", ":"), sort_keys=True, ensure_ascii=False))
if __name__ == "__main__": if __name__ == "__main__":