HAHA is a homelab project to build a fault-tolerant, high-performance smart home stack. Hosted on three Proxmox nodes, it ensures uninterrupted operation of key IoT services: Home Assistant, Mosquitto, Zigbee2MQTT, ESPHome, and Node-RED — all running in a high-availability cluster.

Overview

AspectDetails
Nodes3 Proxmox hosts
ClusteringPacemaker + Corosync
Storage3.6TB DRBD dual-primary + OCFS2
Network2x10Gbps LACP per node
ServicesHome Assistant, Mosquitto, Zigbee2MQTT, ESPHome, Node-RED

Technology Stack

Clustering

  • Pacemaker + Corosync with STONITH fencing (3-node cluster)
  • Floating virtual IP for seamless failover
  • Quorum-based decision making

Storage

  • 3.6TB DRBD dual-primary replication
  • OCFS2 cluster filesystem shared via NFS
  • KINGSTON SFYRD4000G NVMe drives with PCI passthrough

Networking

  • LACP etherchannel (2x10Gbps per node)
  • Cisco WS-C3850-12X48U switch
  • Network-attached IoT peripherals (no USB dependencies)

Dockerized Services

  • Home Assistant (core automation)
  • Mosquitto (MQTT broker)
  • Zigbee2MQTT (Zigbee gateway)
  • ESPHome (ESP device management)
  • Node-RED (flow-based automation)

IoT Peripherals

  • Ethernet Zigbee: TubesZB CC2652P7
  • Ethernet Bluetooth: Olimex ESP32-POE-ISO
  • USB-free design for clean failover

AI Integration

  • Local LLM (Ollama) on RTX 3090 Ti
  • Private voice control and inference
  • No cloud dependency

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Floating VIP (Active)                         │
└───────────────────────────┬─────────────────────────────────────┘
                            │
    ┌───────────────────────┼───────────────────────┐
    │                       │                       │
    ▼                       ▼                       ▼
┌─────────┐           ┌─────────┐           ┌─────────┐
│ pve01   │           │ pve02   │           │ pve03   │
│ Active  │◄─────────►│ Standby │◄─────────►│Arbitrator│
│         │  DRBD     │         │  DRBD     │         │
└────┬────┘  Sync     └────┬────┘  Sync     └────┬────┘
     │                     │                     │
     │    Corosync Ring    │                     │
     └─────────────────────┴─────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│                    DRBD + OCFS2 + NFS                           │
│              (3.6TB Replicated Storage)                         │
└─────────────────────────────────────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Docker Services                               │
│  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐            │
│  │Home Assistant│ │  Mosquitto   │ │ Zigbee2MQTT  │            │
│  └──────────────┘ └──────────────┘ └──────────────┘            │
│  ┌──────────────┐ ┌──────────────┐                              │
│  │   ESPHome    │ │   Node-RED   │                              │
│  └──────────────┘ └──────────────┘                              │
└─────────────────────────────────────────────────────────────────┘
                            │
         ┌──────────────────┼──────────────────┐
         ▼                  ▼                  ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Ethernet Zigbee │ │Ethernet Bluetooth│ │   Local LLM    │
│ TubesZB CC2652P7│ │ Olimex ESP32-POE │ │  RTX 3090 Ti   │
└─────────────────┘ └─────────────────┘ └─────────────────┘

Key Features

  • Seamless failover via Pacemaker/Corosync
  • High-speed, redundant storage with DRBD + NVMe passthrough
  • USB-free Zigbee/Bluetooth to support clean failover
  • Fully local voice assistant, no cloud dependency

Challenges Overcome

  • Fine-tuned Pacemaker quorum and timeouts to avoid split-brain
  • Stable dual-primary DRBD sync with OCFS2 lock handling
  • Migrated from USB to Ethernet peripherals for failover stability
  • Balanced low-latency storage and high throughput using LACP + passthrough NVMe

Performance Benchmarks

For IoT workloads, targets are: 50k IOPS, 200 MiB/s bandwidth, 5000 µs latency.

HAHA delivers:

  • Random Read: 58.2k IOPS, 227 MiB/s, 4392.34 µs
  • Sequential Write: 6.8k IOPS, 852 MiB/s, 9317.05 µs

Meeting or exceeding all targets despite OCFS2/DRBD/NFS network overhead.

Results

Achieved a resilient and scalable smart home infrastructure with zero single points of failure. The system gracefully tolerates node failures and restarts with no data loss or service interruption.

Skills Demonstrated

  • Virtualization (Proxmox)
  • High availability clustering (Pacemaker, Corosync)
  • DRBD replication
  • OCFS2 cluster filesystem
  • LACP network bonding
  • Docker containerization
  • Home Assistant automation
  • IoT integration
  • Linux system administration

Part of the Nuclear Lighters homelab infrastructure