Compare commits

...

3 Commits

Author SHA1 Message Date
Markus Heiser 3c9d559834
Merge ca85d20204 into 10d3af84b8 2024-11-18 09:11:31 +01:00
Markus Heiser 10d3af84b8 [fix] engine: duckduckgo - don't quote query string
The query string send to DDG must not be qouted.

The query string was URL-qouted in #4011, but the URL-qouted query string result
in unexpected *URL decoded* and other garbish results as reported in #4019
and #4020.  To test compare the results of a query like::

    !ddg Häuser und Straßen :de
    !ddg Häuser und Straßen :all
    !ddg 房屋和街道 :all
    !ddg 房屋和街道 :zh

Closed:

- [#4019] https://github.com/searxng/searxng/issues/4019
- [#4020] https://github.com/searxng/searxng/issues/4020

Related:

- [#4011] https://github.com/searxng/searxng/pull/4011

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-11-17 18:14:22 +01:00
Markus Heiser ca85d20204 [mod] set HTTP GET method by default
HTTP GET vs POST
================

Already diskussed in "Method POST harms UX without providing a tangible privacy
benefit" [1].  At that time we agreed to POST.  However, as we are having more
and more drawbacks with POST, I suggest that we reconsider our previous
decision. The latest cause was [2], but we also have other problems that
negatively affect the UI [1]:

* Open Link in New Tab: does not work for our search result tabs because they
  aren't actual links.

* Bookmarking: a search becomes more difficult with POST. You cannot just press
  a browser keyboard shortcut `Ctrl+D` ... no you have to copy the Search URL from
  the sidebar. This can pose a real struggle for less technically-minded users.

* Sharing: a search with somebody else becomes more difficult with POST (for
  the same reason).

Lock HTTP method in the preferences
===================================

If the user changes the HTTP method in his settings, e.g. from GET to POST, but
has not removed the SearXNG instance from the WEB browser and added it again,
the WEB browser will continue to work with the old setting (GET), while entries
in the HTML form use the newly set method (POST). Not realted to this commit,
but this complication is also known from autocomplete[3].

Only very few maintainers are aware of this fact and probably none of the users
know about it.  We should provide a setup in our defaults that is manageable in
its entirety and comprehensible for the user.  For this reason, the option to
select the HTTP method in the preferences is also disabled in this commit.

[1] https://github.com/searxng/searxng/issues/711
[2] https://github.com/searxng/searxng/issues/3590
[3] https://github.com/searxng/searxng/pull/2333#issuecomment-1565392120

Closes: https://github.com/searxng/searxng/issues/3590
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-06-28 15:14:33 +02:00
3 changed files with 19 additions and 8 deletions

View File

@ -14,6 +14,7 @@
limiter: false limiter: false
public_instance: false public_instance: false
image_proxy: false image_proxy: false
method: "GET"
default_http_headers: default_http_headers:
X-Content-Type-Options : nosniff X-Content-Type-Options : nosniff
X-Download-Options : noopen X-Download-Options : noopen
@ -50,8 +51,14 @@
``image_proxy`` : ``$SEARXNG_IMAGE_PROXY`` ``image_proxy`` : ``$SEARXNG_IMAGE_PROXY``
Allow your instance of SearXNG of being able to proxy images. Uses memory space. Allow your instance of SearXNG of being able to proxy images. Uses memory space.
``method`` : ``GET`` | ``POST``
HTTP method. By defaults ``GET`` is used / The ``POST`` method has the
advantage with some WEB browsers that the history is not easy to read, but
there are also various disadvantages that sometimes severely restrict the ease
of use for the user (e.g. back button to jump back to the previous search
page and drag & drop of search term to new tabs do not work as expected).
.. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers .. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
``default_http_headers`` : ``default_http_headers`` :
Set additional HTTP headers, see `#755 <https://github.com/searx/searx/issues/715>`__ Set additional HTTP headers, see `#755 <https://github.com/searx/searx/issues/715>`__

View File

@ -6,7 +6,7 @@ DuckDuckGo Lite
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
import re import re
from urllib.parse import urlencode, quote_plus from urllib.parse import urlencode
import json import json
import babel import babel
import lxml.html import lxml.html
@ -263,7 +263,7 @@ def request(query, params):
params['url'] = url params['url'] = url
params['method'] = 'POST' params['method'] = 'POST'
params['data']['q'] = quote_plus(query) params['data']['q'] = query
# The API is not documented, so we do some reverse engineering and emulate # The API is not documented, so we do some reverse engineering and emulate
# what https://html.duckduckgo.com/html does when you press "next Page" link # what https://html.duckduckgo.com/html does when you press "next Page" link
@ -381,7 +381,11 @@ def response(resp):
zero_click_info_xpath = '//div[@id="zero_click_abstract"]' zero_click_info_xpath = '//div[@id="zero_click_abstract"]'
zero_click = extract_text(eval_xpath(doc, zero_click_info_xpath)).strip() zero_click = extract_text(eval_xpath(doc, zero_click_info_xpath)).strip()
if zero_click and "Your IP address is" not in zero_click and "Your user agent:" not in zero_click: if zero_click and (
"Your IP address is" not in zero_click
and "Your user agent:" not in zero_click
and "URL Decoded:" not in zero_click
):
current_query = resp.search_params["data"].get("q") current_query = resp.search_params["data"].get("q")
results.append( results.append(

View File

@ -98,7 +98,7 @@ server:
http_protocol_version: "1.0" http_protocol_version: "1.0"
# POST queries are more secure as they don't show up in history but may cause # POST queries are more secure as they don't show up in history but may cause
# problems when using Firefox containers # problems when using Firefox containers
method: "POST" method: "GET"
default_http_headers: default_http_headers:
X-Content-Type-Options: nosniff X-Content-Type-Options: nosniff
X-Download-Options: noopen X-Download-Options: noopen
@ -145,11 +145,11 @@ ui:
# Lock arbitrary settings on the preferences page. To find the ID of the user # Lock arbitrary settings on the preferences page. To find the ID of the user
# setting you want to lock, check the ID of the form on the page "preferences". # setting you want to lock, check the ID of the form on the page "preferences".
# #
# preferences: preferences:
# lock: lock:
- method
# - language # - language
# - autocomplete # - autocomplete
# - method
# - query_in_title # - query_in_title
# searx supports result proxification using an external service: # searx supports result proxification using an external service: