2 Commits

Author SHA1 Message Date
Markus Heiser
ee6d4f322f [mod] engine: reuters - REST-API for Reuter's thumbnail, height:80
The size of the full-size images from ``thumbnail.url`` is usually several
MB. By reducing the full-size image to 80 pixels, the data size for a thumb is
reduced from MB to a few KB.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-18 14:43:35 +02:00
Bnyro
3725aef6f3 [fix] reuters: crash on empty results pages & date parsing
1. On empty result list, return empty EngineResults (#5330)

2. Use ``dateutil.parser`` to avoid ``ValueError``:

    ERROR   searx.engines.reuters : exception : Invalid isoformat string: '2022-06-08T16:07:54Z'
      File "searx/engines/reuters.py", line 91, in response
        publishedDate=datetime.fromisoformat(result["display_time"]),
    ValueError: Invalid isoformat string: '2022-06-08T16:07:54Z'

Closes: https://github.com/searxng/searxng/issues/5330
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2025-10-18 14:43:35 +02:00

View File

@@ -17,7 +17,6 @@ The engine has the following additional settings:
shortcut: reu
sort_order: "relevance"
Implementations
===============
@@ -26,6 +25,7 @@ Implementations
from json import dumps
from urllib.parse import quote_plus
from datetime import datetime, timedelta
from dateutil import parser
from searx.result_types import EngineResults
@@ -76,15 +76,62 @@ def request(query, params):
def response(resp) -> EngineResults:
res = EngineResults()
for result in resp.json().get("result", {}).get("articles", []):
resp_json = resp.json()
if not resp_json.get("result"):
return res
for result in resp_json["result"].get("articles", []):
res.add(
res.types.MainResult(
url=base_url + result["canonical_url"],
title=result["web"],
content=result["description"],
thumbnail=result.get("thumbnail", {}).get("url", ""),
thumbnail=resize_url(result.get("thumbnail", {}), height=80),
metadata=result.get("kicker", {}).get("name"),
publishedDate=datetime.fromisoformat(result["display_time"]),
publishedDate=parser.isoparse(result["display_time"]),
)
)
return res
def resize_url(thumbnail: dict[str, str], width: int = 0, height: int = 0) -> str:
"""Generates a URL for Reuter's thumbnail with the dimensions *width* and
*height*. If no URL can be generated from the *thumbnail data*, an empty
string will be returned.
width: default is *unset* (``0``)
Image width in pixels (negative values are ignored). If only width is
specified, the height matches the original aspect ratio.
height: default is *unset* (``0``)
Image height in pixels (negative values are ignored). If only height is
specified, the width matches the original aspect ratio.
The file size of a full-size image is usually several MB; when reduced to a
height of, for example, 80 points, only a few KB remain!
Fields of the *thumbnail data* (``result.articles.[<int>].thumbnail``):
thumbnail.url:
Is a full-size image (>MB).
thumbnail.width & .height:
Dimensions of the full-size image.
thumbnail.resizer_url:
Reuters has a *resizer* `REST-API for the images`_, this is the URL of the
service. This URL includes the ``&auth`` argument, other arguments are
``&width=<int>`` and ``&height=<int>``.
.. _REST-API for the images:
https://dev.arcxp.com/photo-center/image-resizer/resizer-v2-how-to-transform-images/#query-parameters
"""
url = thumbnail.get("resizer_url")
if not url:
return ""
if int(width) > 0:
url += f"&width={int(width)}"
if int(height) > 0:
url += f"&height={int(height)}"
return url