Ask "what’s the best self-hosted home automation platform" and most answers stop at a single name: Home Assistant. That’s not wrong, exactly. It’s the most popular starting point for good reason. But it undersells how the self-hosted home automation space actually works. A real self-hosted smart home is rarely one application. It’s a stack: a hub for dashboards and automation, protocol bridges that translate hardware you own into something the hub understands, a message backbone that lets the pieces talk to each other, and, for some households, an entirely different hub instead of Home Assistant altogether.
This guide walks through that stack layer by layer, and where each of the tools we cover on this site — Home Assistant, openHAB, Zigbee2MQTT, Mosquitto, ESPHome, Node-RED, and Frigate — actually fits.
The hub layer: Home Assistant and its real alternatives
The hub is the piece with a dashboard, a mobile app, a list of integrations, and an automation engine. This is the one layer where "alternative" is the right word, because these tools compete for the same job.
Home Assistant is the default recommendation for most people starting out: Python-based, fast to install, backed by the nonprofit Open Home Foundation, with an integration count that dwarfs everything else in this space. openHAB is the most serious alternative: a Java/OSGi framework with a longer history, deep protocol-level bindings, and a steeper learning curve, licensed under the Eclipse Public License 2.0. Beyond these two, Domoticz offers a lighter-weight option for people who want less of everything: fewer integrations, less configuration, less resource use. Outside the tools we’ve written full guides for, Homebridge deserves a mention for households anchored in Apple’s ecosystem (it’s less a hub than a translator that exposes non-HomeKit devices to Siri and the Home app), and ioBroker is a popular JavaScript-based hub in European markets that plays a role similar to openHAB’s.
The honest decision point: pick Home Assistant unless you have a specific reason not to. Pick openHAB if you’re already comfortable with Java tooling, need a binding that only openHAB has, or have hit the edges of Home Assistant’s fast-moving release cadence and want an ecosystem with a stronger backward-compatibility track record.
Protocol bridges: Zigbee2MQTT and ESPHome
This is the layer where the "alternatives to Home Assistant" framing actively misleads people. Zigbee2MQTT and ESPHome are not hubs and not competitors to Home Assistant. They’re infrastructure that makes hardware usable, and in most self-hosted stacks they run alongside Home Assistant (or openHAB), not instead of it.
Zigbee2MQTT takes the Zigbee radio traffic from your bulbs, sensors, and plugs — hardware from Ikea, Aqara, Philips Hue, and dozens of other vendors — and republishes it as MQTT messages any platform can consume, replacing the vendor’s proprietary cloud-connected hub with something you run locally. Home Assistant has its own built-in Zigbee integration (ZHA) that skips MQTT entirely, so Zigbee2MQTT only really earns its keep once you want your Zigbee devices visible to more than one platform, or want to keep them portable in case you migrate hubs later.
ESPHome plays a similar translator role for a different hardware category: cheap ESP32, ESP8266, and RP2040 microcontrollers, compiled from a YAML configuration file into real firmware, exposed to Home Assistant either through a native API or, notably, through MQTT — the same broker Zigbee2MQTT uses. That shared MQTT option is exactly why the next layer matters.
The message backbone: why MQTT (and Mosquitto) matter
Mosquitto doesn’t automate anything and has no dashboard. It’s a message broker: the piece that receives a published message on one topic and routes it to everyone subscribed to that topic, decoupling every publisher from every subscriber. Zigbee2MQTT publishes to it. ESPHome devices can optionally publish to it instead of using Home Assistant’s native API. Home Assistant’s own MQTT integration subscribes to it, with auto-discovery that turns a well-formed discovery message into a fully configured entity with no manual YAML.
You don’t strictly need Mosquitto if your entire stack talks directly to Home Assistant through native integrations. You do need it, or another MQTT broker, the moment more than one piece of your stack needs to see the same data. In practice that means most Zigbee2MQTT setups, and any ESPHome device you’ve chosen to run over MQTT instead of the native API.
Automation logic: where Node-RED fits
Both Home Assistant and openHAB ship their own automation engines, and for most households that’s enough. Node-RED exists for the automations that outgrow YAML or a simple trigger-condition-action UI: a browser-based flow editor, licensed Apache-2.0 under the OpenJS Foundation, that lets you wire hardware, APIs, and services together visually. It’s not a hub replacement. It typically connects to Home Assistant (or openHAB) as a client, consuming the same entities and MQTT topics as everything else in the stack, and it’s worth adding once your automation logic gets complex enough that a visual flow genuinely beats reading nested YAML conditionals.
Monitoring and security: Frigate’s place in the stack
Frigate is the odd one out in this guide, because it isn’t really a home automation tool at its core. It’s a local NVR that runs real-time AI object detection on your camera feeds, MIT-licensed with a separate trademark carve-out on the Frigate name and logo. It earns its place here through how tightly it integrates with the rest of the stack: Frigate communicates over MQTT for camera events, so a person-detected event from Frigate can trigger a Home Assistant automation, get logged by Node-RED, or feed into any other MQTT-connected piece of the stack, the same way a Zigbee2MQTT motion sensor would.
Putting the stack together
A representative self-hosted setup might look like this. A Zigbee coordinator feeds Zigbee2MQTT, which publishes device states to Mosquitto. A handful of DIY sensors built with ESPHome publish to the same broker. Home Assistant subscribes to Mosquitto for both, auto-discovering entities. Frigate watches the cameras and publishes detection events to Mosquitto too. Home Assistant handles the straightforward automations natively, and anything more elaborate gets built in Node-RED, which talks back to Home Assistant’s API. One broker, one hub, several bridges, all replaceable independently of each other. That’s the actual advantage of building a stack instead of buying into a single closed platform.
Who governs what
The tools in this stack come from genuinely different corners of the open-source world, and it shows in their licenses. Home Assistant is Apache-2.0, backed by the nonprofit Open Home Foundation. Mosquitto and openHAB are both Eclipse Foundation-licensed under EPL-2.0 (Mosquitto also offers an EDL-1.0 option, functionally equivalent to BSD-3-Clause). Node-RED is Apache-2.0 under the OpenJS Foundation. Zigbee2MQTT is GPL-3.0, maintained independently rather than through a foundation. ESPHome splits its license by component: MIT for the Python tooling, GPLv3 for the C++ runtime that ships on the microcontroller. Frigate is MIT with a trademark carve-out. None of this stops you from running any of it freely in a homelab (licensing mostly matters if you’re modifying source code or redistributing it), but it’s a useful reminder that "self-hosted home automation" isn’t one project’s ecosystem. It’s a loose federation of independently governed tools that happen to speak compatible protocols.
You don’t need the whole stack on day one
None of this is a checklist you need to complete before your smart home "counts." A single Home Assistant install talking directly to a handful of Wi-Fi devices and one or two Zigbee gadgets over ZHA is a complete, legitimate self-hosted setup, and plenty of homelabs never touch Mosquitto, Node-RED, or Frigate at all. The stack described above is what tends to accumulate over time, as specific problems show up: vendor lock-in on a Zigbee hub pushes people toward Zigbee2MQTT, automations that turn into unreadable YAML push people toward Node-RED, and motion detection false positives push people toward Frigate. Treat this guide as a map of where to look when you hit one of those walls, not a starting shopping list.
Choosing your own stack
Start with Home Assistant unless you already know why you wouldn’t. Add Zigbee2MQTT and Mosquitto together the moment you own Zigbee hardware from more than one vendor, or want it decoupled from any single hub. Add ESPHome when you’re ready to build your own sensors instead of buying them. Add Node-RED when your automations stop fitting comfortably in Home Assistant’s UI. Add Frigate when motion detection isn’t precise enough and you want to know it was a person, not a shadow. And consider openHAB, seriously, if you’ve outgrown Home Assistant’s assumptions rather than its feature set. It’s the one tool here built for people who already know what they’re looking for.
Further reading: Open Home Foundation and MQTT.org for background on the protocol itself.