From c9833ded9f4518f258e6b5d7eb4cc8ec79ade56d Mon Sep 17 00:00:00 2001 From: Markus Heiser Date: Sun, 21 May 2023 18:12:39 +0200 Subject: [PATCH] [doc] update & fix documentation of the "SearXNG LXC suite" Signed-off-by: Markus Heiser --- docs/dev/lxcdev.rst | 351 +++++++++++++++++++++----------------- docs/utils/index.rst | 2 + docs/utils/lxc.sh.rst | 217 ++++++++++++++++------- docs/utils/searxng.sh.rst | 14 +- utils/lxc-searxng.env | 9 +- utils/searxng.sh | 12 ++ 6 files changed, 380 insertions(+), 225 deletions(-) diff --git a/docs/dev/lxcdev.rst b/docs/dev/lxcdev.rst index 6688c21f0..ef603e9fd 100644 --- a/docs/dev/lxcdev.rst +++ b/docs/dev/lxcdev.rst @@ -16,6 +16,12 @@ In this article we will show, how you can make use of Linux Containers (LXC_) in should have a serious meaning about the terms: *distributed*, *merge* and *linux container*. + **hint** + + If you have issues with the internet connectivity of your containers read + section :ref:`internet connectivity docker`. + + .. contents:: Contents :depth: 2 :local: @@ -25,38 +31,38 @@ In this article we will show, how you can make use of Linux Containers (LXC_) in Motivation ========== -Usually in our development cycle, we edit the sources and run some test and/or -builds by using ``make`` :ref:`[ref] ` before we commit. This cycle -is simple and perfect but might fail in some aspects we should not overlook. +Most often in our development cycle, we edit the sources and run some test +and/or builds by using ``make`` :ref:`[ref] ` before we commit. This +cycle is simple and perfect but might fail in some aspects we should not +overlook. **The environment in which we run all our development processes matters!** The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but -they do not have access to all prerequisites. For example, there may have -dependencies on packages that are installed on the developer's desktop, but +these tools do not have access to all prerequisites. For example, there may +have dependencies on packages that are installed on developer's desktop, but usually are not preinstalled on a server or client system. Another example is; settings have been made to the software on developer's desktop that would never be set on a *production* system. - **Linux Containers are isolate environments and not to mix up all the - prerequisites from various projects on developer's desktop is always a good - choice.** + **Linux Containers are isolate environments**, we use them to not mix up all + the prerequisites from various projects on developer's desktop. The scripts from :ref:`searx_utils` can divide in those to install and maintain -software: +software - :ref:`searxng.sh` -and the script :ref:`lxc.sh`, with we can scale our installation, maintenance or -even development tasks over a stack of isolated containers / what we call the: +and the script - **SearXNG LXC suite** +- :ref:`lxc.sh` -.. hint:: +with we can scale our installation, maintenance or even development tasks over a +stack of isolated containers / what we call the: - If you see any problems with the internet connectivity of your - containers read section :ref:`internet connectivity docker`. +- :ref:`searxng lxc suite` +.. _lxcdev install searxng: Gentlemen, start your engines! ============================== @@ -69,7 +75,7 @@ once: .. tabs:: - .. group-tab:: desktop + .. group-tab:: desktop (HOST) .. code:: bash @@ -81,7 +87,7 @@ fork: .. tabs:: - .. group-tab:: desktop + .. group-tab:: desktop (HOST) .. code:: bash @@ -89,76 +95,61 @@ fork: $ git clone https://github.com/searxng/searxng.git searxng $ cd searxng +.. sidebar:: The ``searxng-archlinux`` container + + is the base of all our exercises here. + The :ref:`lxc-searxng.env` consists of several images, see ``export -LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`. For this blog post -we exercise on a archlinux_ image. The container of this image is named -``searxng-archlinux``. Lets build the container, but be sure that this container -does not already exists, so first lets remove possible old one: +LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`. +For this blog post we exercise on a archlinux_ image. The container of this +image is named ``searxng-archlinux``. + +Lets build the container, but be sure that this container does not already +exists, so first lets remove possible old one: .. tabs:: - .. group-tab:: desktop + .. group-tab:: desktop (HOST) .. code:: bash $ sudo -H ./utils/lxc.sh remove searxng-archlinux $ sudo -H ./utils/lxc.sh build searxng-archlinux -.. sidebar:: The ``searxng-archlinux`` container - is the base of all our exercises here. +.. sidebar:: further read -In this container we install all services :ref:`including searx, morty & filtron -` in once: + - :ref:`lxc.sh install suite` + - :ref:`installation nginx` + +To install the complete :ref:`SearXNG suite ` and the HTTP +proxy :ref:`installation nginx` into the archlinux container run: .. tabs:: - .. group-tab:: desktop + .. group-tab:: desktop (HOST) .. code:: bash $ sudo -H ./utils/lxc.sh install suite searxng-archlinux - -To proxy HTTP from filtron and morty in the container to the outside of the -container, install nginx into the container. Once for the bot blocker filtron: - -.. tabs:: - - .. group-tab:: desktop - - .. code:: bash - - $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ - ./utils/filtron.sh nginx install + $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx + $ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL ... - INFO: got 429 from http://10.174.184.156/searx - -and once for the content sanitizer (content proxy morty): - -.. tabs:: - - .. group-tab:: desktop - - .. code:: bash - - $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ - ./utils/morty.sh nginx install - ... - INFO: got 200 from http://10.174.184.156/morty/ + [searxng-archlinux] SEARXNG_URL : http://n.n.n.140/searxng .. sidebar:: Fully functional SearXNG suite - From here on you have a fully functional SearXNG suite running with bot - blocker (filtron) and WEB content sanitizer (content proxy morty), both are - needed for a *privacy protecting* search engine. + From here on you have a fully functional SearXNG suite (including a + :ref:`redis db`). -On your system, the IP of your ``searxng-archlinux`` container differs from -http://10.174.184.156/searx, just open the URL reported in your installation -protocol in your WEB browser from the desktop to test the instance from outside -of the container. +In such a SearXNG suite admins can maintain and access the debug log of the +services quite easy. -In such a earXNG suite admins can maintain and access the debug log of the -different services quite easy. +In the example above the SearXNG instance in the container is wrapped to +``http://n.n.n.140/searxng`` to the HOST system. Note, on your HOST system, the +IP of your ``searxng-archlinux`` container is different to this example. To +test the instance in the conatiner from outside of the container, in your WEB +browser on your desktop just open the URL reported in your installation .. _working in containers: @@ -166,77 +157,76 @@ In containers, work as usual ============================ Usually you open a root-bash using ``sudo -H bash``. In case of LXC containers -open the root-bash in the container using ``./utils/lxc.sh cmd -searxng-archlinux``: +open the root-bash in the container is done by the ``./utils/lxc.sh cmd +searxng-archlinux`` command: .. tabs:: - .. group-tab:: desktop + .. group-tab:: desktop (HOST) .. code:: bash $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash INFO: [searxng-archlinux] bash - [root@searxng-archlinux searx]# pwd - /share/searxng + [root@searxng-archlinux SearXNG]$ -The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user in -the searxng-container. To debug the running SearXNG instance use: +The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user +in the container (GUEST). To debug the running SearXNG instance use: .. tabs:: - .. group-tab:: root@searxng-archlinux + .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST) .. code:: bash - $ ./utils/searx.sh inspect service + $ ./utils/searxng.sh instance inspect ... use [CTRL-C] to stop monitoring the log ... -Back in the browser on your desktop open the service http://10.174.184.156/searx + .. group-tab:: desktop (HOST) + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance inspect + ... + use [CTRL-C] to stop monitoring the log + ... + + +Back in the browser on your desktop open the service http://n.n.n.140/searxng and run your application tests while the debug log is shown in the terminal from above. You can stop monitoring using ``CTRL-C``, this also disables the *"debug option"* in SearXNG's settings file and restarts the SearXNG uwsgi application. -To debug services from filtron and morty analogous use: -Another point we have to notice is that the service (:ref:`SearXNG ` +Another point we have to notice is that the service :ref:`SearXNG ` runs under dedicated system user account with the same name (compare -:ref:`create searxng user`). To get a shell from these accounts, simply call: +:ref:`create searxng user`). To get a login shell from these accounts, simply +call: .. tabs:: - .. group-tab:: root@searxng-archlinux + .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST) .. code:: bash - $ ./utils/searxng.sh instance cmd bash + $ ./utils/searxng.sh instance cmd bash -l + (searx-pyenv) [searxng@searxng-archlinux ~]$ pwd + /usr/local/searxng -To get in touch, open a shell from the service user (searxng@searxng-archlinux): - -.. tabs:: - - .. group-tab:: desktop + .. group-tab:: desktop (HOST) .. code:: bash - $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash - INFO: [searxng-archlinux] ./utils/searxng.sh instance cmd bash - [searxng@searxng-archlinux ~]$ + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash -l + INFO: [searxng-archlinux] ./utils/searxng.sh instance cmd bash -l + (searx-pyenv) [searxng@searxng-archlinux ~]$ pwd + /usr/local/searxng The prompt ``[searxng@searxng-archlinux]`` signals that you are logged in as system -user ``searx`` in the ``searxng-archlinux`` container and the python *virtualenv* +user ``searxng`` in the ``searxng-archlinux`` container and the python *virtualenv* ``(searxng-pyenv)`` environment is activated. -.. tabs:: - - .. group-tab:: searxng@searxng-archlinux - - .. code:: bash - - (searxng-pyenv) [searxng@searxng-archlinux ~]$ pwd - /usr/local/searxng - Wrap production into developer suite ==================================== @@ -256,12 +246,11 @@ With the use of the :ref:`searxng.sh` the SearXNG service was installed as .. tabs:: - .. group-tab:: desktop + .. group-tab:: uwsgi@searxng .. code:: bash - $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ - systemctl stop uwsgi@searxng + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl stop uwsgi@searxng With the command above, we stopped the SearXNG uWSGI-App in the archlinux container. @@ -278,17 +267,25 @@ least you should attend the settings of ``uid``, ``chdir``, ``env`` and virtualenv = /usr/local/searxng/searxng-pyenv pythonpath = /usr/local/searxng/searxng-src -If you have read the :ref:`"Good to know section" ` you remember, that -each container shares the root folder of the repository and the command -``utils/lxc.sh cmd`` handles relative path names **transparent**. To wrap the -SearXNG installation into a developer one, we simple have to create a smylink to -the **transparent** reposetory from the desktop. Now lets replace the -repository at ``searxng-src`` in the container with the working tree from outside -of the container: +If you have read the :ref:`Good to know` you remember, that each container +shares the root folder of the repository and the command ``utils/lxc.sh cmd`` +handles relative path names **transparent**. + +To wrap the SearXNG installation in the container into a developer one, we +simple have to create a smylink to the **transparent** reposetory from the +desktop. Now lets replace the repository at ``searxng-src`` in the container +with the working tree from outside of the container: .. tabs:: - .. group-tab:: container becomes a developer suite + .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST) + + .. code:: bash + + $ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old + $ ln -s /share/SearXNG/ /usr/local/searxng/searxng-src + + .. group-tab:: desktop (HOST) .. code:: bash @@ -296,7 +293,7 @@ of the container: mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ - ln -s /share/searx/ /usr/local/searxng/searxng-src + ln -s /share/SearXNG/ /usr/local/searxng/searxng-src Now we can develop as usual in the working tree of our desktop system. Every time the software was changed, you have to restart the SearXNG service (in the @@ -304,48 +301,83 @@ container): .. tabs:: - .. group-tab:: desktop + .. group-tab:: uwsgi@searxng .. code:: bash - $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ - systemctl restart uwsgi@searx + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng Remember: :ref:`working in containers` .. here are just some examples from my daily usage: +To *inspect* the SearXNG instance (already described above): + .. tabs:: - .. group-tab:: desktop - - To *inspect* the SearXNG instance (already described above): + .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST) .. code:: bash - $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ - ./utils/searx.sh inspect service + $ ./utils/searx.sh inspect service - Run :ref:`makefile`, e.g. to test inside the container: + .. group-tab:: desktop (HOST) .. code:: bash - $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ - make test + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searx.sh inspect service - To install all prerequisites needed for a :ref:`buildhosts`: +Run :ref:`makefile`, e.g. to test inside the container: + +.. tabs:: + + .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST) .. code:: bash - $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ - ./utils/searxng.sh install buildhost + $ make test - To build the docs on a buildhost :ref:`buildhosts`: + .. group-tab:: desktop (HOST) .. code:: bash - $ sudo -H ./utils/lxc.sh cmd searxng-archlinux \ - make docs.html + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux make test + + + +To install all prerequisites needed for a :ref:`buildhosts`: + +.. tabs:: + + .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST) + + .. code:: bash + + $ ./utils/searxng.sh install buildhost + + .. group-tab:: desktop (HOST) + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost + + +To build the docs on a buildhost :ref:`buildhosts`: + +.. tabs:: + + .. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST) + + .. code:: bash + + $ make docs.html + + .. group-tab:: desktop (HOST) + + .. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux make docs.html + .. _lxcdev summary: @@ -356,48 +388,51 @@ We build up a fully functional SearXNG suite in a archlinux container: .. code:: bash + $ sudo -H ./utils/lxc.sh build searxng-archlinux $ sudo -H ./utils/lxc.sh install suite searxng-archlinux + ... + Developer install? (wraps source from HOST into the running instance) [YES/no] + +To wrap the suite into a developer one answer ``YES`` (or press Enter). + +.. code:: text + + link SearXNG's sources to: /share/SearXNG + ========================================= + + mv -f "/usr/local/searxng/searxng-src" "/usr/local/searxng/searxng-src.backup" + ln -s "/share/SearXNG" "/usr/local/searxng/searxng-src" + ls -ld /usr/local/searxng/searxng-src + |searxng| lrwxrwxrwx 1 searxng searxng ... /usr/local/searxng/searxng-src -> /share/SearXNG + +On code modification the instance has to be restarted (see :ref:`uWSGI +maintenance`): + +.. code:: bash + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng To access HTTP from the desktop we installed nginx for the services inside the container: -.. tabs:: +.. code:: bash - .. group-tab:: [root@searxng-archlinux] + $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx - .. code:: bash - - $ ./utils/filtron.sh nginx install - $ ./utils/morty.sh nginx install - -To wrap the suite into a developer one, we created a symbolic link to the -repository which is shared **transparent** from the desktop's file system into -the container : - -.. tabs:: - - .. group-tab:: [root@searxng-archlinux] - - .. code:: bash - - $ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old - $ ln -s /share/searx/ /usr/local/searxng/searxng-src - $ systemctl restart uwsgi@searx - -To get information about the searxNG suite in the archlinux container we can +To get information about the SearxNG suite in the archlinux container we can use: -.. tabs:: +.. code:: text - .. group-tab:: desktop - - .. code:: bash - - $ sudo -H ./utils/lxc.sh show suite searxng-archlinux - ... - [searxng-archlinux] INFO: (eth0) filtron: http://10.174.184.156:4004/ http://10.174.184.156/searx - [searxng-archlinux] INFO: (eth0) morty: http://10.174.184.156:3000/ - [searxng-archlinux] INFO: (eth0) docs.live: http://10.174.184.156:8080/ - [searxng-archlinux] INFO: (eth0) IPv6: http://[fd42:573b:e0b3:e97e:216:3eff:fea5:9b65] - ... + $ sudo -H ./utils/lxc.sh show suite searxng-archlinux + [searxng-archlinux] INFO: (eth0) docs-live: http:///n.n.n.140:8080/ + [searxng-archlinux] INFO: (eth0) IPv6: http://[fd42:555b:2af9:e121:216:3eff:fe5b:1744] + [searxng-archlinux] uWSGI: + [searxng-archlinux] SEARXNG_UWSGI_SOCKET : /usr/local/searxng/run/socket + [searxng-archlinux] environment /usr/local/searxng/searxng-src/utils/brand.env: + [searxng-archlinux] GIT_URL : https://github.com/searxng/searxng + [searxng-archlinux] GIT_BRANCH : master + [searxng-archlinux] SEARXNG_URL : http:///n.n.n.140/searxng + [searxng-archlinux] SEARXNG_PORT : 8888 + [searxng-archlinux] SEARXNG_BIND_ADDRESS : 127.0.0.1 diff --git a/docs/utils/index.rst b/docs/utils/index.rst index 2da26ed12..1ac77e285 100644 --- a/docs/utils/index.rst +++ b/docs/utils/index.rst @@ -20,6 +20,8 @@ Common command environments The scripts in our tooling box often dispose of common environments: +.. _FORCE_TIMEOUT: + ``FORCE_TIMEOUT`` : environment Sets timeout for interactive prompts. If you want to run a script in batch job, with defaults choices, set ``FORCE_TIMEOUT=0``. By example; to install a diff --git a/docs/utils/lxc.sh.rst b/docs/utils/lxc.sh.rst index 4308a12cc..cb0fe9361 100644 --- a/docs/utils/lxc.sh.rst +++ b/docs/utils/lxc.sh.rst @@ -14,6 +14,11 @@ ``utils/lxc.sh`` ================ +With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of +containers, what we call the: *lxc suite*. The :ref:`lxc-searxng.env` is +loaded by default, every time you start the ``lxc.sh`` script (*you do not need +to care about*). + .. sidebar:: further reading - snap_, `snapcraft LXD`_ @@ -21,10 +26,16 @@ - `LXC/LXD Image Server`_ - `LXD@github`_ -With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of -containers, what we call the: *lxc suite*. The *SearXNG suite* -(:origin:`lxc-searxng.env `) is loaded by default, every time -you start the ``lxc.sh`` script (*you do not need to care about*). +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + + +.. _lxd install: + +Install LXD +=========== Before you can start with containers, you need to install and initiate LXD_ once:: @@ -38,44 +49,19 @@ take some time**:: $ sudo -H ./utils/lxc.sh build +.. sidebar:: hint + + If you have issues with the internet connectivity of your containers read + section :ref:`internet connectivity docker`. + A cup of coffee later, your LXC suite is build up and you can run whatever task you want / in a selected or even in all :ref:`LXC suite containers `. -.. hint:: - - If you see any problems with the internet connectivity of your - containers read section :ref:`internet connectivity docker`. - -If you do not want to build all containers, **you can build just one**:: - - $ sudo -H ./utils/lxc.sh build searxng-archlinux - -*Good to know ...* - -Each container shares the root folder of the repository and the command -``utils/lxc.sh cmd`` **handles relative path names transparent**, compare output -of:: - - $ sudo -H ./utils/lxc.sh cmd -- ls -la Makefile - ... - -In the containers, you can run what ever you want, e.g. to start a bash use:: - - $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash - INFO: [searxng-archlinux] bash - [root@searxng-archlinux SearXNG]# - -If there comes the time you want to **get rid off all** the containers and -**clean up local images** just type:: - - $ sudo -H ./utils/lxc.sh remove - $ sudo -H ./utils/lxc.sh remove images - .. _internet connectivity docker: Internet Connectivity & Docker -============================== +------------------------------ .. sidebar:: further read @@ -115,21 +101,132 @@ Reboot your system and check the iptables rules:: :FORWARD ACCEPT [7048:7851230] +.. _searxng lxc suite: + +SearXNG LXC suite +================= + +The intention of the *SearXNG LXC suite* is to build up a suite of containers +for development tasks or :ref:`buildhosts ` with a very +small set of simple commands. At the end of the ``--help`` output the SearXNG +suite from the :ref:`lxc-searxng.env` is introduced:: + + $ sudo -H ./utils/lxc.sh --help + ... + LXC suite: searxng + Suite includes installation of SearXNG + images: ubu2004 ubu2204 fedora35 archlinux + containers: searxng-ubu2004 searxng-ubu2204 searxng-fedora35 searxng-archlinux + +As shown above there are images and containers build up on this images. To show +more info about the containers in the *SearXNG LXC suite* call ``show suite``. +If this is the first time you make use of the SearXNG LXC suite, no containers +are installed and the output is:: + + $ sudo -H ./utils/lxc.sh show suite + + LXC suite (searxng-*) + ===================== + + +------+-------+------+------+------+-----------+ + | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | + +------+-------+------+------+------+-----------+ + + WARN: container searxng-ubu2004 does not yet exists + WARN: container searxng-ubu2204 does not yet exists + WARN: container searxng-fedora35 does not yet exists + WARN: container searxng-archlinux does not yet exists + +If you do not want to run a command or a build in all containers, **you can +build just one**. Here by example in the container that is build upon the +*archlinux* image:: + + $ sudo -H ./utils/lxc.sh build searxng-archlinux + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd + +Otherwise, to apply a command to all containers you can use:: + + $ sudo -H ./utils/lxc.sh build + $ sudo -H ./utils/lxc.sh cmd -- ls -la . + +Running commands +---------------- + +**Inside containers, you can run scripts** from the :ref:`toolboxing` or run +what ever command you need. By example, to start a bash use:: + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash + INFO: [searxng-archlinux] bash + [root@searxng-archlinux SearXNG]# + +.. _Good to know: + +Good to know +------------ + +Each container shares the root folder of the repository and the command +``utils/lxc.sh cmd`` **handle relative path names transparent**:: + + $ pwd + /share/SearXNG + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd + INFO: [searxng-archlinux] pwd + /share/SearXNG + +The path ``/share/SearXNG`` will be different on your HOST system. The commands +in the conatiner are executed by the ``root`` inside of the container. Compare +output of:: + + $ ls -li Makefile + 47712402 -rw-rw-r-- 1 markus markus 2923 Apr 19 13:52 Makefile + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux ls -li Makefile + INFO: [searxng-archlinux] ls -li Makefile + 47712402 -rw-rw-r-- 1 root root 2923 Apr 19 11:52 Makefile + ... + +Since the path ``/share/SearXNG`` of the HOST system is wrapped into the +container under the same name, the shown ``Makefile`` (inode ``47712402``) in +the ouput is always the identical ``/share/SearXNG/Makefile`` from the HOST +system. In the example shown above the owner of the path in the container is +the ``root`` user of the conatiner (and the timezone in the container is +different to HOST system). + + .. _lxc.sh install suite: Install suite -============= +------------- -To install the complete :ref:`SearXNG suite (includes searx, morty & filtron) -` into all LXC_ use:: +.. sidebar:: further read + - :ref:`working in containers` + - :ref:`FORCE_TIMEOUT ` + +To install the complete :ref:`SearXNG suite ` into **all** LXC_ +containers leave the container argument empty and run:: + + $ sudo -H ./utils/lxc.sh build $ sudo -H ./utils/lxc.sh install suite +To *build & install* suite only in one container you can use by example:: + + $ sudo -H ./utils/lxc.sh build searxng-archlinux + $ sudo -H ./utils/lxc.sh install suite searxng-archlinux + The command above installs a SearXNG suite (see :ref:`installation scripts`). To :ref:`install a nginx ` reverse proxy (or alternatively use :ref:`apache `):: - sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx + $ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx + +Same operation just in one container of the suite:: + + $ sudo -H ./utils/lxc.sh cmd searxng-archlinux FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx + +The :ref:`FORCE_TIMEOUT ` environment is set to zero to run the +script without user interaction. To get the IP (URL) of the SearXNG service in the containers use ``show suite`` command. To test instances from containers just open the URLs in your @@ -137,22 +234,22 @@ WEB-Browser:: $ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL - [searxng-ubu2110] SEARXNG_URL : http://n.n.n.147/searxng - [searxng-ubu2004] SEARXNG_URL : http://n.n.n.246/searxng - [searxnggfedora35] SEARXNG_URL : http://n.n.n.140/searxng - [searxng-archlinux] SEARXNG_URL : http://n.n.n.165/searxng + [searxng-ubu2110] SEARXNG_URL : http://n.n.n.170/searxng + [searxng-ubu2004] SEARXNG_URL : http://n.n.n.160/searxng + [searxnggfedora35] SEARXNG_URL : http://n.n.n.150/searxng + [searxng-archlinux] SEARXNG_URL : http://n.n.n.140/searxng + +Clean up +-------- + +If there comes the time you want to **get rid off all** the containers and +**clean up local images** just type:: + + $ sudo -H ./utils/lxc.sh remove + $ sudo -H ./utils/lxc.sh remove images -Running commands -================ - -**Inside containers, you can use make or run scripts** from the -:ref:`toolboxing`. By example: to setup a :ref:`buildhosts` and run the -Makefile target ``test`` in the archlinux_ container:: - - sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost - sudo -H ./utils/lxc.sh cmd searxng-archlinux make test - +.. _Setup SearXNG buildhost: Setup SearXNG buildhost ======================= @@ -160,7 +257,7 @@ Setup SearXNG buildhost You can **install the SearXNG buildhost environment** into one or all containers. The installation procedure to set up a :ref:`build host` takes its time. Installation in all containers will take more time (time for another cup -of coffee).:: +of coffee). :: sudo -H ./utils/lxc.sh cmd -- ./utils/searxng.sh install buildhost @@ -174,13 +271,13 @@ To get IP of the container and the port number *live docs* is listening:: $ sudo ./utils/lxc.sh show suite | grep docs.live ... - [searxng-archlinux] INFO: (eth0) docs.live: http://n.n.n.12:8080/ + [searxng-archlinux] INFO: (eth0) docs.live: http://n.n.n.140:8080/ .. _lxc.sh help: -Overview -======== +Command Help +============ The ``--help`` output of the script is largely self-explanatory: @@ -189,8 +286,10 @@ The ``--help`` output of the script is largely self-explanatory: .. _lxc-searxng.env: -SearXNG suite -============= +SearXNG suite config +==================== + +The SearXNG suite is defined in the file :origin:`utils/lxc-searxng.env`: .. literalinclude:: ../../utils/lxc-searxng.env :language: bash diff --git a/docs/utils/searxng.sh.rst b/docs/utils/searxng.sh.rst index f6578f7b1..3a6ac6d43 100644 --- a/docs/utils/searxng.sh.rst +++ b/docs/utils/searxng.sh.rst @@ -5,6 +5,9 @@ ``utils/searxng.sh`` ==================== +To simplify the installation and maintenance of a SearXNG instance you can use the +script :origin:`utils/searxng.sh`. + .. sidebar:: further reading - :ref:`architecture` @@ -12,8 +15,11 @@ - :ref:`installation nginx` - :ref:`installation apache` -To simplify the installation and maintenance of a SearXNG instance you can use the -script :origin:`utils/searxng.sh`. +.. contents:: Contents + :depth: 2 + :local: + :backlinks: entry + Install ======= @@ -28,8 +34,8 @@ The installation is described in chapter :ref:`installation basic`. .. _searxng.sh overview: -Overview -======== +Command Help +============ The ``--help`` output of the script is largely self-explanatory: diff --git a/utils/lxc-searxng.env b/utils/lxc-searxng.env index 9d6efb42e..8cc0ce886 100644 --- a/utils/lxc-searxng.env +++ b/utils/lxc-searxng.env @@ -42,10 +42,11 @@ EOF lxc_suite_install() { ( lxc_set_suite_env - FORCE_TIMEOUT=0 - export FORCE_TIMEOUT - "${LXC_REPO_ROOT}/utils/searxng.sh" install all - rst_title "suite installation finished ($(hostname))" part + FORCE_TIMEOUT=0 "${LXC_REPO_ROOT}/utils/searxng.sh" install all + rst_title "Suite installation finished ($(hostname))" part + if ask_yn "Developer install? (wraps source from HOST into the running instance)" Yn; then + "${LXC_REPO_ROOT}/utils/searxng.sh" searxng.install.link_src "$(pwd)" + fi lxc_suite_info echo ) diff --git a/utils/searxng.sh b/utils/searxng.sh index dee90bbf4..e04f87e10 100755 --- a/utils/searxng.sh +++ b/utils/searxng.sh @@ -467,6 +467,18 @@ EOF popd > /dev/null } +searxng.install.link_src() { + rst_title "link SearXNG's sources to: $2" chapter + echo + tee_stderr 0.1 <&1 | prefix_stdout "$_service_prefix" +mv -f "${SEARXNG_SRC}" "${SEARXNG_SRC}.backup" +ln -s "${2}" "${SEARXNG_SRC}" +ls -ld /usr/local/searxng/searxng-src +EOF + echo + uWSGI_restart "$SEARXNG_UWSGI_APP" +} + searxng.install.pyenv() { rst_title "Create virtualenv (python)" section echo