jandieman
/
searxng
mirror of https://github.com/searxng/searxng.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
4,013 Commits 8 Branches 17 Tags 124 MiB
Commit Graph

41 Commits

Author SHA1 Message Date
Alexandre Flament a4dcfa025c [enh] engines: add about variable 
move meta information from comment to the about variable
so the preferences, the documentation can show these information
 2021-01-14 20:57:17 +01:00
Alexandre Flament d703119d3a [enh] add raise_for_httperror 
check HTTP response:
* detect some comme CAPTCHA challenge (no solving). In this case the engine is suspended for long a time.
* otherwise raise HTTPError as before

the check is done in poolrequests.py (was before in search.py).

update qwant, wikipedia, wikidata to use raise_for_httperror instead of raise_for_status
 2020-12-11 14:37:08 +01:00
Alexandre Flament bef2f2efa8 [fix] wikidata: fix crash when the item has no description at all and at least one URL. 2020-12-04 17:17:20 +01:00
Alexandre Flament 9ed3ee2beb [mod] wikidata: WDGeoAttribute class: doesn't change the method signature of get_str 2020-12-01 15:21:17 +01:00
Alexandre Flament 3038052c79 [mod] remove unused import 
use
from searx.engines.duckduckgo import _fetch_supported_languages, supported_languages_url  # NOQA
so it is possible to easily remove all unused import using autoflake:
autoflake --in-place --recursive --remove-all-unused-imports searx tests
 2020-11-14 14:11:02 +01:00
Alexandre Flament 95bd6033fa [mod] wikidata engine: use one SPARQL request instead of 2 HTTP requests. 2020-10-28 08:09:25 +01:00
Alexandre Flament 2006eb4680 [mod] move extract_text, extract_url to searx.utils 2020-10-02 18:13:56 +02:00
Dalf 1022228d95 Drop Python 2 (1/n): remove unicode string and url_utils 2020-09-10 10:39:04 +02:00
Marc Abonce Seguin 0d8970c8f2
only return one url per "type" in Wikidata (#2151) 
i.e. only one official website, one Twitter, etc.
 2020-08-27 21:44:48 +02:00
Adam Tauber 29960aa1d9 [enh] add official site link to the top of the infobox - closes #1644 2020-06-09 23:49:13 +02:00
Dalf 85b3723345 [mod] speed optimization 
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
 2019-11-15 09:33:15 +01:00
Dalf 6e0285b2db [fix] wikidata engine: faster processing, remove one HTTP redirection. 
* Search URL is https://www.wikidata.org/w/index.php?{query}&ns0=1 (with ns0=1 at the end to avoid an HTTP redirection)
* url_detail: remove the disabletidy=1 deprecated parameter
* Add eval_xpath function: compile once for all xpath.
* Add get_id_cache: retrieve all HTML with an id, avoid the slow to procress dynamic xpath '//div[@id="{propertyid}"]'.replace('{propertyid}')
* Create an etree.HTMLParser() instead of using the global one (see #1575)
 2019-07-29 07:39:39 +02:00
Noémi Ványi b63d645a52 Revert "remove 'all' option from search languages" 
This reverts commit 4d1770398a.
 2019-01-07 21:19:00 +01:00
Marc Abonce Seguin 5568f24d6c [fix] check language aliases when setting search language 2019-01-06 20:31:57 -06:00
Léo Bourrel 7a474db61b Fix formatting 2018-07-06 10:31:01 +02:00
Léo Bourrel acaef6600e Update path to wikidata image 2018-07-05 10:11:45 +02:00
Marc Abonce Seguin b12857a70d [fix] make search requests on wikidata more accurate 2018-04-08 21:17:00 -05:00
Marc Abonce Seguin 772c048d01 refactor engine's search language handling 
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
 2018-03-27 00:08:03 -06:00
marc 4d1770398a remove 'all' option from search languages 2017-12-06 01:20:15 -06:00
Adam Tauber 52e615dede [enh] py3 compatibility 2017-05-15 12:02:30 +02:00
marc af35eee10b tests for _fetch_supported_languages in engines 
and refactor method to make it testable without making requests
 2016-12-15 00:40:21 -06:00
marc f62ce21f50 [mod] fetch supported languages for several engines 
utils/fetch_languages.py gets languages supported by each engine and
generates engines_languages.json with each engine's supported language.
 2016-12-13 19:58:10 -06:00
marc 149802c569 [enh] add supported_languages on engines and auto-generate languages.py 2016-12-13 19:32:00 -06:00
marc ad58b14be7 [fix] merge infoboxes based on weight 
also minor changes in attributes and images from wikidata
 2016-08-05 23:51:04 -05:00
marc a0a1284998 wikidata refactor and more attributes (see issue #560) 2016-08-05 23:51:04 -05:00
a01200356 93ef11adc0 [enh] multilingual wikidata 
disambiguation and tags are in local language

TOFIX:
    needs to query the api every time to know each label's name
 2016-08-05 23:51:04 -05:00
a01200356 8d335dbdae [enh] wikipedia infobox 
creates simple multilingual infobox using wikipedia's api
 2016-04-17 16:22:19 -05:00
Adam Tauber bd22e9a336 [fix] pep8 compatibilty 2016-01-18 12:47:31 +01:00
Adam Tauber 362c849797 [fix][mod] wikidata date handling refactor - fixes #387 2015-09-07 22:39:33 +02:00
dalf d07cfd9089 [enh] use one single http connection pool : improve response time. close #100 2015-01-21 11:33:16 +01:00
Adam Tauber d7ea44ab8d [fix] dates before 1900 2015-01-11 13:26:42 +01:00
Adam Tauber cc4e17b668 [fix] pep8 2015-01-02 12:33:40 +01:00
Cqoicebordel 7937218be6 Use human readable date 
For DoB and DoD, wikipedia use a non standard ISO format, not easily readable.
Now the date is displayed in an human readable form, using the language setting as locale if available. If not, it uses the default locale.
 2014-12-09 02:36:53 +01:00
dalf ffcec383b7 [fix] pep8 : duckduckgo_definitions and wikidata engines 2014-12-07 16:36:20 +01:00
Adam Tauber b0bb94fd37 [fix] wikidata: using only the first url 2014-10-12 14:33:03 +02:00
dalf cac1761a54 [enh] infoboxes : if the result doesn't contain anything except one link, use the normal result template 2014-10-11 15:49:50 +02:00
dalf 295b1699ce [mod] return only one result from the wikidata engine 2014-10-11 12:47:30 +02:00
Adam Tauber 727c7226d9 [fix] code cleanup 2014-10-04 22:53:54 +02:00
dalf 63a0328c8b [enh] wikidata engine : add links to musicbrainz 2014-10-02 23:40:06 +02:00
Dalf 0a71525ab6 [enh] add infoboxes and answers (clean up) 2014-10-01 22:17:35 +02:00
Dalf 6bfd566353 [enh] add infoboxes and answers 2014-09-28 16:51:41 +02:00