use a sparql request on wikidata to get the list of currencies.
currencies.json contains the translation for all supported searx languages.
Supersede #993
Target pip-exe is a prerequisite of the targets:
- pyinstall
- pyuninstall
and was accidentally deleted in commit 9b48ae47.
HINT:
do not confuse pyinstall with penvinstall
pyinstall & pyuninstall
Installing into user's HOME using pip from OS,
therefore the message is needed.
pyenvinstall & pyenvuninstall
Installing into virtualenv (./local) using pip which is provided by
prerequisite 'pyenv' in the virtualenv.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
1. This patch fixes error:
rm -rf gh-pages/
make V=1 gh-pages
make[1]: Leaving directory '/800GBPCIex4/share/searx'
[ -d "gh-pages/.git" ] || git clone gh-pages
fatal: repository 'gh-pages' does not exist
2. The gh-page build has been moved to ./build/gh-pages this also affects
'travis-gh-pages'
3. The gh-pages commit messages now includes a ref to the repository and commit
4. Since a gh-pages history has only the drawback that the reposetory grows
fast, this patch also flattens the history:
cd build/gh-pages/; git log --oneline
bash: cd: build/gh-pages/: Datei oder Verzeichnis nicht gefunden
026126be (HEAD -> gh-pages, origin/gh-pages) make gh-pages: from https://github.com/return42/searx.git@71d66979c2935312e0aed7fc7c3cf6199fbe88a2
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
$ make test.sh
In utils/lxc.sh line 42:
ubu2010_boilerplate="$ubu1904_boilerplate"
^-----------------^ SC2034: ubu2010_boilerplate appears unused. Verify use (or export if used externally).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The fabfile.py has not been updated since 5 years. I also asked [1] if someone
still use Fabric wtihout any response. Lets drop outdated Fabric file.
[1] https://github.com/searx/searx/discussions/2400
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
pip install -e .
...
Obtaining file:///usr/local/searx/searx-src
ERROR: Command errored out with exit status 1:
command: /usr/local/searx/searx-pyenv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/usr/local/searx/searx-src/setup.py'"'"'; __file__='"'"'/usr/local/searx/searx-src/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-vzer91m2
cwd: /usr/local/searx/searx-src/
Complete output (9 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/searx/searx-src/setup.py", line 10, in <module>
from searx.version import VERSION_STRING
File "/usr/local/searx/searx-src/searx/__init__.py", line 19, in <module>
import searx.settings_loader
File "/usr/local/searx/searx-src/searx/settings_loader.py", line 8, in <module>
import yaml
ModuleNotFoundError: No module named 'yaml'
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Removes module searx/brand.py and creates a namespace at searx.brand.
This patch is a first 'proof of concept'. Later we can decide to remove the
brand namespace entirely or not.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Ubuntu 20.10 image [1] is avialable from https://images.linuxcontainers.org
BTW: replace builhost package 'ttf-dejavu' by more common meta package
'fonts-dejavu' ('ttf-dejavu' is no longer available in 20.10).
[1] d161de601b
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Official support for Ubuntu 19.10 'Eoan Ermine' ended on July 17, 2020.
The image has been dropped [1] from https://images.linuxcontainers.org
[1] d161de601b
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Without this commit the module searx checks the secret_key value.
With this commit, make docs, utils/standalone_searx.py,
utils/fetch_firefox_version.py works without SEARX_DEBUG=1
For reference see https://github.com/searx/searx/pull/2386
Create a (PY_ENV)/requirements.sha256 file with sha256 sums of the
./requirements*.txt files and reinstall only if the sha256 sums change.
Suggested-by: @dalf / https://github.com/searx/searx/pull/2418#pullrequestreview-558426579
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Install boilerplate packages (pip, wheel, setuptools and requirements.txt) only
once, when virtualenv is created.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In commit c2a6f145 the sphinx-doc and sphinx-live targets has been removed,
since Sphinx is already installed by the pyenvinstall target.
This commit removes the targets from the remaining prerequisites of the 'BOOK'
targets
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Makefile targets 'pyenv-install' and 'pyenv-uninstall' are unused duplicates of
'pyenvinstall' and 'pyenvuninstall'.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This patch keeps an eye on:
1. When installing searx, the branch to install needs to be a local branch.
2. The branch to install can be different from the current branch.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In commit a70b9b9f the SEARX_SETTINGS_TEMPLATE environment was added to the
.config file, but was not use in utils/searx.sh.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Installation and use of the command 'virtualenv' was only needed in py2 and py2
is no longer suported by searx. In py3 the command is replaced by 'python -m
venv'.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The path to the makefile.lxc is not always static and
'/share/searx/utils/makefile.lxc' can be wrong.
When target 'lxc.activate' is called within the LXC container the path to
makefile.lxc depends on LXC_SHARE_FOLDER and the basename of the folder where
searx was cloned inital on the HOST system.
This patch determines the path of 'makefile.lxc' next to 'makefile.include'
itself.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
see searx.search.processors.abstract.EngineProcessor
First the method searx call the get_params method.
If the return value is not None, then the searx call the method search.
Since #2291 is merged, it is recommend to use::
use_default_settings=True
1. Add a template file use_default_settings.yml::
SEARX_SETTINGS_TEMPLATE="${REPO_ROOT}/utils/templates/etc/searx/use_default_settings.yml"
2. In Chapter "Configuration" recommend to make use of
'use_default_settings=True' and describe it
3. Rewrite of docs/admin/settings.rst
- move chapter 'settings.yml location' to the top
- update and split chapter 'Global Settings'
4. Add environment SEARX_SETTINGS_TEMPLATE to .config.sh
5. Use environment $SEARX_SETTINGS_TEMPLATE in the utils/searx.sh script
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* Update call to sphinx-autobuild (remove --poll -B options, add --open-browser)
* Upgrade the Sphinx and sphinx-tabs versions.
* utils/makefile.sphinx: remove the SPHINXVERS variable.
* utils/makefile.sphinx: remove the sphinx-doc and sphinx-live targets, Sphinx is already installed by the pyenvinstall target.
Since there is no centos-7 documentation 'searx.sh doc' produce an error
message::
ERROR: centos-: uWSGI not yet implemented
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
## What does this PR do?
To test centos-7 on non centos systems add centos-7 image to the LXC suite.
## Why is this change important?
PR #2112 adds centos-7 support to the install scripts
## How to test this PR locally?
`sudo ./utils/lxc.sh build searx-centos7`
Python's default encoding depends on the platform, set (python) default encoding
UTF-8 in uwsgi ini files:
LANG=C.UTF-8
LANGUAGE=C.UTF-8
LC_ALL=C.UTF-8
Error pattern:
Traceback (most recent call last):
File "/usr/local/searx/searx-src/searx/webapp.py", line 74, in <module>
from searx.search import SearchWithPlugins, get_search_query_from_webapp
File "/usr/local/searx/searx-src/searx/search.py", line 32, in <module>
from searx.external_bang import get_bang_url
File "/usr/local/searx/searx-src/searx/external_bang.py", line 13, in <module>
for bang in json.load(json_file)['bang']:
File "/usr/lib/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/lib/python3.8/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 31341: ordinal not in range(128)
close: https://github.com/asciimoo/searx/issues/2041
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The sphinx-doc-prebuilds is the single point of definition for prerequisites
each doc build needs
BTW: remove $(LXC_ENV_FOLDER) offset from BOOKS_FOLDER and DOCS_FOLDER path.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In the past we did not really install searx into the virtualenv. This is a
problem, since entry-points and other python installation stuff is not trigger.
See discussion: https://github.com/asciimoo/searx/pull/1938#issuecomment-632056508
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Even it is a core component, some hoster do not have pre installed the
'inetutils' package. We do need this package for the 'hostname' command.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
A command to set new morty key in morty & searx service (settings.yml) and to
restart both services with new key::
utils/morty.sh option new-key
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This is an addittion to PR #1934:
The .0 change really counts on some engines like Bing which seems to allow
Windows NT 10.0 but not Windows NT 10.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- add installation method for nginx sites, morty and filtron
- clean up PUBLIC_URL environment in and outside of containers
- clean up comand lines
- handle uWSGI quirks on fedora (emperor mode)
- handle Python quirks on debian (there is no 'python' command anymore)
- lib.sh: add die and die_caller functions
- lxc_suite_install_info is now a function
- lint: shellcheck
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This is the revision of the documentation about the varous nginx installation
variants. It also implements the nginx installation scripts for morty and
filtron.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In utils/lib.sh there are two functions with the same name, but different tasks.
Rename one of them from lxc_init_container() into lxc_init_container_env().
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Commit 09a40625 adds virtualenv dependency. BTW remove deprecated
--no-site-packages. Not having access to global site-packages is now the
default behavior.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
revert "makefile.python: ... drop build support py2" to get back Py2 support.
TPy2 support need virtualenv installed by the OS.
BTW: log environment and python version in travis's install phase
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
From py3.3 on a subset of virtualenv is built-in, so you can run '-m venv' ot of
the box.
- replace: $(PY_ENV_BIN)/pip --> $(PY_ENV_BIN)/python -m pip
- remove obsolete virtualenv-exe target and adjust VTENV_OPTS
- remove obsolete msg-pip-exe target
- print list of py launchers available from $(PY_ENV_BIN) to the log
- fix hard coded ./local
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- no more need for a .config.mk
- docs: use searx.brands environment
- searx.sh, filtron.sh & morty.sh are sourcing utils/brand.env
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
To Makefile target brand, add creation of bash environment in::
utils/brand.env
In bash scripts (manage.sh) source env by::
. utils/brand.env
manage.sh help: show GIT_URL and more environment
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
checked with::
sudo ./utils/lxc.sh cmd -- make test.sh
on ubu1804, ubu1910, fedora31 and archlinux. All checks OK except:
ubu1604: uses shellcheck v0.3.7 (from 04/2015) which is no longer supported!
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
To get meaningfull diffs, the json file has to be sorted. Before applying any
further content patch, the json file needs a inital sort (without changing any
content).
Sorted by::
import sys, json
with open('engines_languages.json') as f:
j = json.load(f)
with open('engines_languages.json', 'w') as f:
json.dump(j, f, indent=2, sort_keys=True)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
By isolating the environment of makefiles and bash scripts into .config.mk and
.config.sh it is simple to maintain searx brands by setting some central
environments.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Add *Makefile* boilerplate useful for python projects. All python tasks are
using a virtualenv from ./local/py3
$ make help
run - run developer instance
install - developer install (./local)
uninstall - uninstall (./local)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Instead of a single line with 500000 characters use nicely formatted JSON.
Sort the lists in engine_languages.py so when updating it is possible to
more easily see the differences (search engines do change the order their
languages are listed in)
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.
Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
Alex Balgavy
Marc Abonce Seguin
Alexandre Flament
Markus Heiser
BBaoVanC
rachmadani haryono
renyhp
Adam Tauber
Eliesemoule
Ivan Skytte Jørgensen
cclauss