[chore] Google (HTML) engine: remove obsolete GSA User-Agents (#6366)

The GSA headers that were introduced in PR #5644 unfortunately no longer
work (#6359).

The Google engines

- google.py
- google_videos.py

do not work anymore either, but we'll leave it in the code for now:

- In google.py, central functions like get_google_info(..) are provided, which
  are also used by other modules.
- We will probably need a Google HTML (and video) engine again very soon.

Related:

- https://github.com/searxng/searxng/issues/6359
- https://github.com/searxng/searxng/pull/6364

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
Markus Heiser
2026-07-05 12:07:23 +02:00
committed by GitHub
parent 888364c1ce
commit fd5eb84a37
9 changed files with 4 additions and 5391 deletions

View File

@@ -6,7 +6,7 @@ make data.all
"""
# pylint: disable=invalid-name
__all__ = ["ahmia_blacklist_loader", "gsa_useragents_loader", "data_dir", "get_cache"]
__all__ = ["ahmia_blacklist_loader", "data_dir", "get_cache"]
import json
import typing as t
@@ -63,7 +63,6 @@ lazy_globals = {
"ENGINE_TRAITS": None,
"LOCALES": None,
"TRACKER_PATTERNS": TrackerPatternsDB(),
"GSA_USER_AGENTS": None,
}
data_json_files = {
@@ -106,24 +105,3 @@ def ahmia_blacklist_loader() -> list[str]:
"""
with open(data_dir / 'ahmia_blacklist.txt', encoding='utf-8') as f:
return f.read().split()
def gsa_useragents_loader() -> list[str]:
"""Load data from `gsa_useragents.txt` and return a list of user agents
suitable for Google. The user agents are fetched by::
searxng_extra/update/update_gsa_useragents.py
This function is used by :py:mod:`searx.engines.google`.
"""
data = lazy_globals["GSA_USER_AGENTS"]
if data is not None:
return data
log.debug("init searx.data.%s", "GSA_USER_AGENTS")
with open(data_dir / 'gsa_useragents.txt', encoding='utf-8') as f:
lazy_globals["GSA_USER_AGENTS"] = f.read().splitlines()
return lazy_globals["GSA_USER_AGENTS"]

File diff suppressed because it is too large Load Diff