mirror of
https://github.com/searxng/searxng.git
synced 2025-12-22 19:50:00 +00:00
[feat] sourcehut engine: implement as custom module, fix user agent
SourceHut uses a foss bot protection tool called `go-away` (which I can recommend BTW). It blocks common crawler user agents, such as the standard Firefox user agent. Hence, we're now using our custom SearXNG user agent to clarify we're not a crawler. Closes: https://github.com/searxng/searxng/issues/5270 Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
@@ -2677,23 +2677,10 @@ engines:
|
||||
|
||||
- name: sourcehut
|
||||
shortcut: srht
|
||||
engine: xpath
|
||||
paging: true
|
||||
search_url: https://sr.ht/projects?page={pageno}&search={query}
|
||||
results_xpath: (//div[@class="event-list"])[1]/div[@class="event"]
|
||||
url_xpath: ./h4/a[2]/@href
|
||||
title_xpath: ./h4/a[2]
|
||||
content_xpath: ./p
|
||||
first_page_num: 1
|
||||
categories: [it, repos]
|
||||
engine: sourcehut
|
||||
# https://docs.searxng.org/dev/engines/online/sourcehut.html
|
||||
# sourcehut_sort_order: longest-active
|
||||
disabled: true
|
||||
about:
|
||||
website: https://sr.ht
|
||||
wikidata_id: Q78514485
|
||||
official_api_documentation: https://man.sr.ht/
|
||||
use_official_api: false
|
||||
require_api_key: false
|
||||
results: HTML
|
||||
|
||||
- name: goo
|
||||
shortcut: goo
|
||||
|
||||
Reference in New Issue
Block a user