devLap

Connecting 3000+ Smart Home Devices to Edge AI: Building a Home Assistant Bridge with NeoMind

rust dev.to

Connecting 3000+ Smart Home Devices to Edge AI: Building a Home Assistant Bridge with NeoMind

Home Assistant is the undisputed king of open-source home automation. It supports over 3000 integrations — from Philips Hue bulbs to Tesla Powerwalls to Daikin heat pumps. But there's a gap that no integration can fill: intelligence.

Home Assistant can tell you "the living room temperature is 24°C." It can't tell you "based on the last 3 weeks of occupancy patterns, your HVAC is running 40% longer than needed, and here's a schedule adjustment that would save you $18/month."

That's where NeoMind comes in. Its Home Assistant bridge imports every entity from your HA instance, feeds them to an LLM-powered AI agent running on your local hardware, and lets you interact with your entire smart home through natural language — no cloud, no subscription, no data leaving your network.

Here's how it works, what you can build, and the technical details under the hood.

The Architecture: HA as the Device Layer, NeoMind as the Brain 

┌─────────────────────────────────────────────────────────┐
│                    Your Local Network                     │
│                                                          │
│  ┌──────────────┐        ┌──────────────────────────┐   │
│  │ Home Assistant│◄──────►│  NeoMind (Edge AI)       │   │
│  │  (3000+      │  REST  │  ┌──────────────────┐    │   │
│  │  integrations)│  API   │  │ HA Bridge Ext.   │    │   │
│  └──────┬───────┘        │  │ Entity Import    │    │   │
│         │                │  │ State Sync       │    │   │
│  ┌──────┴───────┐        │  │ Command Dispatch │    │   │
│  │ Your Devices │        │  └────────┬─────────┘    │   │
│  │ Lights, HVAC,│        │           │               │   │
│  │ Sensors, ... │        │  ┌────────▼─────────┐    │   │
│  └──────────────┘        │  │ AI Agent Runtime │    │   │
│                          │  │ LLM + Memory     │    │   │
│                          │  │ + Automation     │    │   │
│                          │  └──────────────────┘    │   │
│                          └──────────────────────────┘   │
└─────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

The bridge is bidirectional:

  • HA → NeoMind: All entities (sensors, switches, lights, climate, cameras, etc.) are imported with their current states, attributes, and history. NeoMind sees your entire home as a structured data model.
  • NeoMind → HA: AI-generated commands flow back through the HA REST API. "Turn off everything on the ground floor after midnight" becomes a series of precise service calls.

Setup: 5 Minutes to a Smarter Home

1. Install NeoMind 

curl -fsSL https://raw.githubusercontent.com/camthink-ai/NeoMind/main/scripts/install.sh | sh
Enter fullscreen mode Exit fullscreen mode

Or if you already have NeoMind running, skip to step 2.

2. Enable the Home Assistant Extension

In NeoMind's web UI (http://localhost:9375), navigate to Extensions → Marketplace and enable "Home Assistant Bridge." Or via the config file:

[extensions.home_assistant]
enabled = true
url = "http://homeassistant.local:8123"
token = "your-long-lived-access-token"
sync_interval = 30  # seconds between state refreshes
Enter fullscreen mode Exit fullscreen mode

Generate a long-lived access token in Home Assistant: Profile → Security → Long-Lived Access Tokens.

3. Configure Your LLM Backend

For fully local AI (recommended for privacy):

[ai]
backend = "ollama"
model = "llama3.1:8b"
api_url = "http://localhost:11434"
Enter fullscreen mode Exit fullscreen mode

Or connect to a cloud model for more capable reasoning:

[ai]
backend = "openai"
model = "gpt-4o-mini"
api_key = "sk-..."
Enter fullscreen mode Exit fullscreen mode

4. Start Talking to Your Home

Open the NeoMind chat interface and start asking questions:

You: "What's the energy consumption looking like today?"

NeoMind: "Your home has used 14.2 kWh so far today, which is 12% above your daily average. The main contributors are the HVAC (6.8 kWh, running since 6 AM due to the cold snap) and the EV charger (4.1 kWh, charging the Tesla from 32% to 78%)."

You: "Can you optimize the HVAC schedule for this week?"

NeoMind: "Based on your occupancy patterns over the last 14 days, nobody is home between 8:30 AM and 5:15 PM on weekdays. I can create an automation rule that sets the thermostat to eco mode (18°C) during those hours and pre-heats to 22°C at 4:45 PM. Estimated savings: ~$22/week. Shall I create this rule?"

Entity Import: How 3000+ Devices Become AI-Readable

When the bridge connects to Home Assistant, it imports the full entity registry:

// Simplified: what NeoMind sees for each HA entity
pub struct HomeAssistantEntity {
    pub entity_id: String,           // "light.living_room_main"
    pub domain: String,              // "light"
    pub friendly_name: String,       // "Living Room Main Light"
    pub state: String,               // "on" / "off" / "24.5"
    pub attributes: HashMap<String, Value>,  // brightness, color_temp, etc.
    pub area: Option<String>,        // "Living Room"
    pub device_class: Option<String>, // "temperature" / "humidity" / etc.
}
Enter fullscreen mode Exit fullscreen mode

The AI agent uses this structured model to:

  1. Answer questions — "Which lights are currently on?" queries the state of all light.* entities
  2. Analyze patterns — "How has the bedroom humidity changed over the past week?" reads time-series data from sensor.bedroom_humidity
  3. Create automations — "Turn off all lights when the last person leaves" combines person.* presence with light.* control
  4. Suggest optimizations — "Your bathroom fan runs 6 hours daily but humidity rarely exceeds 55%" identifies waste

Natural Language → HA Service Calls

The most powerful feature is turning conversational intent into precise Home Assistant service calls. Here's what happens when you say "Dim the living room to 30% and turn on the reading lamp":

User Input: "Dim the living room to 30% and turn on the reading lamp"
    │
    ▼
AI Agent parses intent into structured commands:
    │
    ├── Command 1: light.turn_on
    │   entity_id: light.living_room_main
    │   brightness_pct: 30
    │
    └── Command 2: light.turn_on
        entity_id: light.reading_lamp
    │
    ▼
HA Bridge dispatches via REST API:
    POST http://ha:8123/api/services/light/turn_on
    {"entity_id": "light.living_room_main", "brightness_pct": 30}
    POST http://ha:8123/api/services/light/turn_on
    {"entity_id": "light.reading_lamp"}
Enter fullscreen mode Exit fullscreen mode

This isn't regex matching or keyword spotting. The LLM understands context:

  • "Turn off the kitchen" → resolves to all entities in the "Kitchen" area
  • "Set the thermostat to what it was yesterday morning" → looks up historical state
  • "If the garage door is open after 10 PM, close it and notify me" → creates an automation rule with condition + action + notification

Automation Rules: Beyond HA's Native Automations

Home Assistant's YAML automations are powerful but verbose. NeoMind lets you create them conversationally:

You: "Every weekday at 7 AM, if it's raining, turn on the entryway light and send me a reminder to bring an umbrella"

NeoMind generates:

{"name":"Rainy Morning Routine","condition":{"operator":"and","conditions":[{"source":"time","operator":"equals","value":"07:00"},{"source":"time","operator":"day_of_week","value":["mon","tue","wed","thu","fri"]},{"source":"device:weather.home:condition","operator":"equals","value":"rainy"}]},"actions":[{"type":"ha_service","service":"light.turn_on","entity_id":"light.entryway"},{"type":"notify","channel":"push","message":"Rainy morning — don't forget your umbrella!"}],"cooldown":86400}
Enter fullscreen mode Exit fullscreen mode

This rule runs in NeoMind's engine — not in HA's automation system — which means it can incorporate AI analysis:

{"name":"Smart Energy Saver","trigger":{"source":"schedule","interval":"1h"},"actions":[{"type":"ai_agent","prompt":"Analyze the last hour of energy consumption. If any device used more than 2x its typical hourly average, explain why and suggest whether it should be turned off or scheduled differently."}]}
Enter fullscreen mode Exit fullscreen mode

Multi-Room Voice Control Without the Cloud

Combine NeoMind's Home Assistant bridge with the voice extensions (CosyVoice TTS + multilingual ASR) for a fully local voice assistant:

"Hey NeoMind, what's the air quality in the bedroom?"
    │
    ▼ (local ASR → text)
    │
    ▼ (AI Agent → query sensor.bedroom_air_quality)
    │
    ▼ (LLM generates natural response)
    │
    ▼ (local TTS → audio)
    │
"The bedroom AQI is 42 — good air quality. The PM2.5 level dropped significantly 
after you turned on the purifier at 3 PM."
Enter fullscreen mode Exit fullscreen mode

No Alexa. No Google. No audio sent to the cloud. Everything runs on your hardware.

Scaling: From a Studio Apartment to a Smart Campus

The bridge handles anything Home Assistant supports:

Scale Entities NeoMind Performance
Studio ~20 entities < 50ms response, ~80MB RAM
3-bedroom house ~150 entities < 100ms response, ~150MB RAM
Large villa ~500 entities < 200ms response, ~300MB RAM
Small office ~1500 entities < 500ms response, ~600MB RAM
Smart campus (multi-HA) ~5000+ entities Run multiple NeoMind instances, one per HA

For multi-site deployments, each NeoMind instance connects to its own Home Assistant, and you coordinate across instances through shared dashboards and cross-instance queries.

Privacy: Your Home Data Stays Home

This is the killer feature for privacy-conscious users:

  • No cloud processing — LLM runs locally via Ollama (Llama 3, Mistral, Qwen, etc.)
  • No telemetry — NeoMind doesn't phone home. Ever.
  • No third-party access — Your HA entities never leave your LAN
  • Offline capable — Works even when your internet is down

Compare this to cloud-based smart home AI (Google Home, Alexa, etc.), where every voice command and device state is transmitted to external servers.

Get Started

Already running Home Assistant? Install NeoMind, enable the bridge, and ask your home a question. The AI is local, the integration is instant, and your data never leaves your network.

Source: dev.to

arrow_back Back to Tutorials