QGIS-Server Monitoring Tool

[meyerlor]

Hello Dear Lizmap (and QGIS-server) users,

i worked on a little tool to help me get a better understanding for my qgis-server. It mainly helped me to find the culprits of slow rendering times etc, but i continuesly added more functionallity to it like some aggregated statistic functions etc. So now it feels ready to publish it, in case some of you have a use for it.

Some Key Points:

• How long does each MapRequest need on the QGIS-server
• which users/projects are the most active?
• did my change of hardware/serversetting bring the speed advance i hoped for?
• what layers should be optimized for speed?

And much more. Have a look at the github repo:

GitHub - meyerlor/qgis-server-monitoring: Real-time monitoring dashboard for QGIS Server —...

Cheers,

Lorenz

[wagner-it]

Hello Lorenz,

Thank you very much for this brilliant tool.

I am using it (with a few modifications) and it is very useful.
As described, in addition to the server data, it also displays user behavior and, above all, the speed at which projects and layers are loaded.
This is very valuable for optimizing projects.

Thank you and best regards,
Günter

[tgdbepe4]

Hello Lorenz

Great tool and very helpful. As a maybe too progressive guy I run my installation under Ubuntu 22.04 and it has the problem to install the python modules “pip install flask flask-socketio eventlet psutil”. With pipx I was able to install it and “python3 monitor.py” is working. But the installation of the service makes problems. I hope I can solve this problem.

Now two questions:

I use a SWAP file to prevent memory problems. Do you think I can add this to the monitor script as well?

Affect the NGINX installation a running py-qgisserver installation on the same machine?

Cheers

Peter

[meyerlor]

Thanks for the feedback!

I worked today a bit further on the tool, implementing your suggestion @tgdbepe4 to include a swap-file monitor (although I think your server should never need to fall back to it - this would probably cause a big hit on the rendering speed..).

I also added a ton of deeper insights about what users are actually doing - thanks to @wagner-it suggestions. Printing, Editing (with a small drawback: At least one field contrain needs to be configured via QGIS→Attribute Form, otherwise the Editing is not sent via QGIS server and therefore will never appear in the logs), opening PopUps is now also recorded.

About your installation issues: i think you are not progressive enough - I’m running on Ubunut 24.04 which is the latest LTR. 22.04 is outdated - which might be the cause for your pip install issue!

[tgdbepe4]

Hi Lorenz

Great that you extend your tool! Many thanks!

By the way I run

Distributor ID: Ubuntu
Description: Ubuntu 24.04.4 LTS
Release: 24.04
Codename: noble

python3 --version
Python 3.12.3

And on this system I have the problems with pip. How do you solved it? Or do you not have this problem?

Error when trying to install:

pip install flask flask-socketio eventlet psutil

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.

Cheers

Peter

[tgdbepe4]

Habe noch dies gefunden. So geht es soweit. Wie mache ich dies dann aber für den systemd service?

Ubuntu 24.04 blockiert systemweites pip install wegen PEP 668; du musst entweder ein venv benutzen oder gezielt per apt installieren.​

Empfohlene Lösung: virtuelles Environment

Für ein Projekt mit Flask, Flask‑SocketIO, eventlet und psutil willst du sowieso ein sauberes venv, sonst kollidierst du mit Systempaketen.

bash

# 1. Modul für venv sicherstellen

sudo apt install python3-venv python3-full -y # einmalig nötig[web:5][web:9]

# 2. Projektordner anlegen

mkdir myapp

cd myapp

# 3. Virtuelles Environment erstellen

python3 -m venv venv # Name ‘venv’ beliebig[web:3][web:5][web:9]

# 4. venv aktivieren

source venv/bin/activate # Prompt ändert sich zu (venv)[web:3][web:5][web:9]

# 5. Innerhalb des venv Pakete installieren

pip install --upgrade pip # optional, aber sinnvoll[web:5]

pip install flask flask-socketio eventlet psutil

Solange das venv aktiv ist, bekommst du keinen externally-managed-environment‑Fehler, weil pip dann in dieses isolierte Environment installiert und nicht ins System‑Python.

[wagner-it]

Hello Peter,
I have replied to you directly by email.
We should probably stick to English here. Otherwise, should we switch to the German list?

I made the installation very easy for myself.
I gave the AI the GitHub address with @meyerlor instructions and let myself be guided (copy & paste).
It worked well right away.

Best regards
Günter

[meyerlor]

Ok so no need have you solved it? pip packages should never be installed system wiede, creating a virtual environment for them is recommended. will update my installation guide!

[tgdbepe4]

Yes thanks KI all works now.

I had additional to create a file start.sh with content:

#!/bin/bash
cd /opt/monitoring
source venv/bin/activate
exec python monitor.py

Then adapt the file /etc/systemd/system/monitoring-dashboard.service

/etc/systemd/system/monitoring-dashboard.service

[Unit]
Description=QGIS Server Monitoring Dashboard
After=network.target

[Service]
WorkingDirectory=/opt/monitoring
ExecStart=/opt/monitoring/start.sh
Restart=always
RestartSec=5
Environment=“PATH=/opt/monitoring/venv/bin”
User=www-data

[Install]
WantedBy=multi-user.target

And then all in line with your guidelines:

sudo systemctl daemon-reload
sudo systemctl enable --now monitoring-dashboard.service

sudo systemctl stop monitoring-dashboard.service
sudo systemctl start monitoring-dashboard.service
sudo systemctl restart monitoring-dashboard.service

Now all is working as expected

Kind regards

Peter

[tgdbepe4]

I would like to activate the admin and anonymous users as default. Is this possible?

Peter

[tgdbepe4]

It would be very nice to see from which and where a user is coming ( ip adress ) but I know this is only available over the nginx logs.

[wagner-it]

I think it’s better that the tool remembers the last settings (for administrators and anonymous users). That’s how it is for me too. But I’ve also customized the tool extensively to my needs using AI.

Does the IP address really make sense? Most users probably don’t have a fixed IP anyway, right?
But yes, it works. I use it for the WMS services that are used by the local QGIS.

[Antoviscomi]

Very nice tool @meyerlor It would be nice to integrate it into the Lizmap admin dashboard as a plugin.

[meyerlor]

thanks a lot @Antoviscomi, i rellay appreciate it!

Integrating it into a single lwc admin interface would certainly be possible - but the strength / scope of that tool is to monitor the underlying QGIS Server (pools). So regardless how many instances of LWC you have running on your server, everything gets aggregated into one interface. It comes more out of a debugging direction ( i had severe problems from time to time with my QGIS server across all lwc instances) and had to find a way to find culprits!

[Antoviscomi]

@meyerlor

Integrating it into a single lwc admin interface would certainly be possible - but the strength / scope of that tool is to monitor the underlying QGIS Server (pools). So regardless how many instances of LWC you have running on your server, everything gets aggregated into one interface. It comes more out of a debugging direction ( i had severe problems from time to time with my QGIS server across all lwc instances) and had to find a way to find culprits!

Sure @meyerlor , It is clear that it works at a low level, not at client level, and that it can be customized by including the ability to also read LWC logs, in order to have a complete picture and understand if the problematic instances are caused by the server, the client, or both.