Monitoring Stack¶

Eight.ly includes an optional monitoring stack that gives you dashboards, alerting, and searchable logs for this machine and every container you run. Enable it from Settings > Monitoring in the web dashboard.

When to Deploy¶

The monitoring stack is optional and not enabled by default. Consider enabling it when:

• You're running 5 or more containers and want visibility into resource usage
• You want log search to troubleshoot container, Traefik, or Authelia issues without opening the web terminal
• You want alerting for disk space, high CPU, or container crashes
• You're running VMs or LXC containers and want unified metrics

For a server with 1-2 containers, the Eight.ly dashboard's built-in metrics are usually sufficient. The monitoring stack shines when you have enough services that you can't keep track of them by eye.

What Gets Deployed¶

One switch deploys all seven containers together — there's no separate step for logs:

Opening Grafana¶

Where to find it: click Open Grafana on the Monitoring settings page, which takes you to /grafana/ on your Eight.ly dashboard — Grafana is not published on its own port to your network. It runs behind Eight.ly's own login, and reaching it requires the Operator role or above.

Grafana does still have its own separate credentials underneath (an admin account, generated at install with a random password), because Eight.ly proxies into a real Grafana instance rather than replacing its UI. You won't normally need that admin login unless you're changing Grafana's own settings — signing in to Eight.ly with an Operator or Admin account is what gets you into /grafana/ day to day.

Pre-loaded Dashboards¶

Seven dashboards are provisioned automatically:

Find your dashboards from the Grafana home page or the dashboards icon in the left sidebar. There's no dedicated Logs dashboard — search logs from Grafana's Explore view instead (see below).

What each dashboard tells you¶

Eight.ly System Overview is your starting point. It shows aggregate CPU and memory usage, container count, running containers, and network throughput. If something looks off, drill into a specific dashboard.

Node Exporter shows host-level detail: per-core CPU usage, memory breakdown, disk I/O per device, and network traffic per interface. Use this to identify hardware bottlenecks.

Container Metrics breaks down resource usage per container. Sort by CPU or memory to find resource hogs.

The Traefik, Servarr, and Plex dashboards are useful only if you're running those services — they scrape metrics those apps expose.

Enabling Monitoring¶

1. Sign in to Eight.ly
2. Click your username > Settings
3. Go to the Monitoring tab
4. Toggle Enable Monitoring Stack
5. Wait about a minute for containers to start
6. Click Open Grafana

Disabling Monitoring¶

Same path: Settings > Monitoring > toggle off. All monitoring containers are stopped and removed. Your metrics and log data volumes are preserved — re-enabling restores your history.

Creating Custom Dashboards¶

The pre-loaded dashboards cover the common cases, but you can build your own:

1. In Grafana, click + > Dashboard > Add visualization
2. Select Prometheus as the data source for metrics or Loki for logs
3. Use the query builder or type PromQL/LogQL directly
4. Useful Prometheus metrics to start with:
   • container_cpu_usage_seconds_total — container CPU
   • container_memory_usage_bytes — container memory
   • node_filesystem_avail_bytes — free disk space
   • node_cpu_seconds_total — host CPU by core

Setting Up Alerts¶

Grafana can notify you when metrics cross thresholds:

1. Open any dashboard panel, click the panel title > Edit
2. Go to the Alert tab
3. Define a condition (e.g., avg(node_filesystem_avail_bytes) < 10737418240 for <10GB free)
4. Set the notification channel — Grafana supports email, Slack, Discord, webhooks, and more
5. Configure notification channels from Alerting > Contact points in the sidebar

Common alerts worth setting up:

• Disk space below 10% on any filesystem
• Container restart count increasing (crash loop detection)
• Host memory usage above 90% for 5+ minutes
• Container stopped unexpectedly

For alerting on Eight.ly's own events (a VM stopping, a backup failing, a drive that may be failing) rather than raw metrics, see Notifications — that's a separate, simpler path that doesn't require Grafana at all.

Searching Logs¶

With the monitoring stack enabled, container logs, the system journal, and Traefik/Authelia's own logs are all searchable from Grafana's Explore view (compass icon in the sidebar). Select Loki as the data source, then use LogQL. Exact label names depend on the source (container logs are labelled by container name; Traefik and Authelia logs come from their log files), so start by browsing labels in Explore rather than guessing them:

# All logs from a specific container
{container="plex"}

# Errors across all containers
{job="containers"} |= "error"

# The system journal
{job="system"}

# Traefik or Authelia's own log files
{job="traefik"}
{job="authelia"}