[feat] sourcehut engine: implement as custom module, fix user agent

SourceHut uses a foss bot protection tool called `go-away` (which I can
recommend BTW).  It blocks common crawler user agents, such as the standard
Firefox user agent.  Hence, we're now using our custom SearXNG user agent to
clarify we're not a crawler.

Closes: https://github.com/searxng/searxng/issues/5270
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
Bnyro
2025-11-04 18:00:02 +01:00
parent 3763b4bff4
commit 1be19f8b58
3 changed files with 101 additions and 16 deletions

View File

@@ -2677,23 +2677,10 @@ engines:
- name: sourcehut
shortcut: srht
engine: xpath
paging: true
search_url: https://sr.ht/projects?page={pageno}&search={query}
results_xpath: (//div[@class="event-list"])[1]/div[@class="event"]
url_xpath: ./h4/a[2]/@href
title_xpath: ./h4/a[2]
content_xpath: ./p
first_page_num: 1
categories: [it, repos]
engine: sourcehut
# https://docs.searxng.org/dev/engines/online/sourcehut.html
# sourcehut_sort_order: longest-active
disabled: true
about:
website: https://sr.ht
wikidata_id: Q78514485
official_api_documentation: https://man.sr.ht/
use_official_api: false
require_api_key: false
results: HTML
- name: goo
shortcut: goo