Refactor engines that parse ISO 8601 dates with strptime to use
fromisoformat instead. In most cases this is a direct replacement of
strptime(text, "format") with fromisoformat(text).
For engines where the source has a trailing "Z" that strptime consumed
as a literal (e.g. "%Y-%m-%dT%H:%M:%S.%fZ" in huggingface.py), add
rstrip("Z") to keep the output naive and preserve the existing behavior.
In sogou.py the date is extracted with a regular expression, which can
yield strings like "2026-7-11". strptime accepts this via its format
string, but fromisoformat does not. To preserve the existing behavior
and satisfy the format fromisoformat expects, add zero-padding for the
month and day.
Closes: #6098
---------
Signed-off-by: OneVth <onebrotravel@gmail.com>
* [mod] show img_load_error.svg if thumbnail fails to load (#6368)
Brave and a few other engines occasionally provide dead URLs for the thumbnails,
which looks pretty ugly in the UI. As already with the image search, in case of
an error img_load_error.svg should be displayed here.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* [build] /static (#6368)
---------
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
`engines` only contains active engines. Since `google` was set to
inactive, it's no longer part of the , making the engine crash.
With this workaround, we directly load the engine traits from the data
dictionary instead of going the intermediate step with the Google engine.
I was testing the new Kagi engine and found for some queries I was getting a
`KeyError` exception from the result parsing.
This PR ensures we only assumes the `url` key exists, and we use `.get()` to
retrieve values for keys that may not be present.
Their API documentation [1] clarifies that only the `url` and `title` properties
are required/guaranteed in the search result object, but this is not correct:
File "/home/patrick/code/searxng/searx/engines/kagi.py", line 161, in response
title=html.unescape(result["title"]),
~~~~~~^^^^^^^^^
KeyError: 'title
Heard back from Kagi support:
> The image results "title' should be marked as optional, as many images likely
> don't have titles - as you've noticed. The only required field there should be
> the "url" field.
[1] https://kagi.redocly.app/api/docs/openapi/search/search#search/search/t=response&c=200&path=data/search
[2] https://github.com/searxng/searxng/issues/2247#issuecomment-4692976877
The GSA headers that were introduced in PR #5644 unfortunately no longer
work (#6359).
The Google engines
- google.py
- google_videos.py
do not work anymore either, but we'll leave it in the code for now:
- In google.py, central functions like get_google_info(..) are provided, which
are also used by other modules.
- We will probably need a Google HTML (and video) engine again very soon.
Related:
- https://github.com/searxng/searxng/issues/6359
- https://github.com/searxng/searxng/pull/6364
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Adds a new google cse engine as an alternative to the currently broken html
google engine.
It works by generating a token which is cached for 1 hour and it only uses the
public CSE id from blackle.com. We do have two others but I think just
hardcoding one will be enough.
Will be broken in 2027 due to google deprecating it but I it will be good while
it lasts.
I found I can get up to 20 results per page. Setting safesearch and language
works and I just copied the traits from the google engine.
Also set old google engine as inactive as it is completely broken for now.
We're now shuffling the order of query arguments, as
it seems that Qwant actively blocks the query order of
SearXNG.
tgp seems to be short for "test group" (look at the XHR requests the
browser sends to "https://www.qwant.com/action/ui") - its actual value
doesn't matter, as long as it's sent and at the correct position in the
query params and doesn't change too frequently
Naver redesigned their search results page, replacing the old CSS classes
(lst_total, link_tit, total_dsc_wrap, api_txt_lines) with a new layout
using fds-web-doc-root containers and sds-comps design system.
Updated parse_general():
- Result container: fds-web-doc-root + fds-web-normal-doc-root
- Title: sds-comps-text-type-headline1 span
- URL: first external http link (with try/except guard)
- Content: sds-comps-text-type-body1 span
- Thumbnail: profile-thumbnail img src
- Added guard: skip results without title or URL
Baidu image search returns antiFlag=1 when the acjson API is called
without session cookies from image.baidu.com.
Warm up cookies with a GET to the image homepage, cache them for one
hour via EngineCache, and pass them to image search requests. Raise
SearxEngineAccessDeniedException when antiFlag is still returned.
Signed-off-by: Hu Butui <hot123tea123@gmail.com>
* [mod] update_engine_descriptions.py - revision of the script (#6309)
The script was a bit outdated, type hints were missing, and the strings were
converted to double quotation marks
The `descriptions.items()` are sorted to avoid assigning the references to
seen_descriptions randomly (should reduce unnecessary diffs in the future).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* [data] update searx.data - update_engine_descriptions.py (#6309)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
---------
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
