4 Commits

Author	SHA1	Message	Date
ArtikusHG	735e388cec	Merge branch 'master' into fasttext	2022-12-16 19:43:10 +00:00
ArtikusHG	1f8f8c1e91	Replace langdetect with fasttext	2022-12-16 21:07:39 +02:00
Markus Heiser	ed901ab18e	[mod] improve 'Autodetect search language' plugin ... - Add documentation to the plugin - Harmonize FastText language model with SearXNG's language model Reosurces:: import fasttext # --> +10 MB fasttext.load_model(str(data_dir / 'lid.176.ftz')) # --> +4MB Suggested-by: @dalf - To speed up and simplify the deployment use fasttext-wheel instead of fasttext - Building numpy on the Alpine Linux of docker-images takes ages --> install py3-numpy from Alpines package manager (apk) - Alpine Linux on docker-images (musl libc) do not support fasttext-wheel (gnu libc) --> patch Dockerfile and build from fastetxt: sed -i s/fasttext-wheel/fasttext/ requirements.txt Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>	2022-12-11 11:26:07 +01:00
ArtikusHG	9925a20950	[mod] new plugin: Autodetect search language	2022-12-10 13:11:47 +01:00