Home Assistant with Local Ollama on an RTX 5070 Ti

In this article

The system I actually tested

The model runs on my desktop workstation. Home Assistant runs on a separate always-on machine and reaches Ollama over the local network. No prompt needs to leave the house, but Home Assistant also has to accept that the workstation may be rebooting or switched off.

Those are observations from this machine, not estimates from a spec sheet or a controlled comparison. The local and Home Assistant requests had different prompt and tool context; the numbers do not isolate network overhead. They are the reason I kept the 14B model for this setup, not a prediction for another installation.

The 24B model was slower for the wrong reason

I also tried mistral-small3.2:24b. Its 15 GB of weights left too little VRAM for a useful context window, so part of the model spilled to the CPU at every context size I tested. It generated 14–16 tokens/second at 16k context and about 27 at 4k. Qwen stayed fully on the GPU and ran roughly five times faster.

For this setup, I needed to leave room beyond the model weights for context and runtime allocations. Roughly 13 GB of weights was a working budget, not a fixed limit of every 16 GB card. Model architecture, context length, cache format, concurrent requests, and other GPU workloads can change what fits. ollama ps settles the question: if the processor column does not say 100% GPU at the context I need, I am paying for that spill on every response.

Home Assistant needs to expect the model to disappear

The workstation does not have the same uptime contract as Home Assistant. I added a slow availability probe and a binary sensor, then made every consumer check that sensor before asking the model for anything important.

condition:
  - condition: state
    entity_id: binary_sensor.ollama_available
    state: "on"

That condition is intentionally boring. Without it, an automation can silently do nothing while the workstation is off. A five-minute probe interval is enough for this job and avoids filling the log when the endpoint is expected to be unavailable.

Ollama listens only on localhost by default, so I changed its bind address to make it reachable from Home Assistant. Ollama does not add authentication to that local API. I allow it on the trusted LAN and do not expose or port-forward the service to the internet. The Ollama FAQ documents the host setting and network behavior; the Home Assistant Ollama integration handles the conversation agent and AI Task entities.

A probe you can adapt

This example checks whether Ollama answers and lists the required model. Add it to configuration.yaml; if you already have a binary_sensor: list, append the item instead of adding a second key. For a separate included binary-sensor file, omit the outer binary_sensor: line. Replace the documentation-only IP address with your Ollama host and use the exact model name returned by /api/tags.

# configuration.yaml: merge into an existing binary_sensor list if you have one.
# Replace the documentation-only address and exact model tag before use.
binary_sensor:
  - platform: rest
    name: Ollama available
    unique_id: ollama_model_available
    resource: http://192.0.2.10:11434/api/tags
    method: GET
    scan_interval: 300
    timeout: 5
    value_template: >-
      {% set found = namespace(model=false) %}
      {% if value_json is mapping and value_json.models is sequence
            and value_json.models is not string and value_json.models is not mapping %}
        {% for model in value_json.models %}
          {% if model is mapping and model.name | default('') == 'qwen3:14b' %}
            {% set found.model = true %}
          {% endif %}
        {% endfor %}
      {% endif %}
      {{ found.model }}

Check the Home Assistant configuration before restarting to load the new sensor. Confirm its actual entity ID under Developer Tools → States; a name collision can add a suffix. Use that ID in the condition above and the dashboard button below. Only on should pass the condition: off, unknown, and unavailable should not.

A five-minute poll can be stale for nearly five minutes, plus the request timeout. The workstation can disappear after an on result. This is a preliminary check, not a substitute for handling failure in the action that calls the model. Listing a model does not load it into VRAM, test generation, or prove that a tool call will succeed.

Before relying on it, inspect the sensor with the intended model installed, with a deliberately unmatched model name in a test configuration, and with the server unreachable. The YAML and response template were checked offline against 12 fixtures; this example has not been deployed as a new test on my household system.

See the RESTful binary sensor configuration and Ollama model-list endpoint for the underlying options and response format.

The setup step that produced no entities

Creating the Ollama integration entry was not enough. The parent entry stores the server URL, but the useful entities came from two subentries: one conversation agent and one AI Task. Until I added those, Home Assistant had a configured integration with nothing to call.

I ended up with these useful pieces:

• conversation.local_ai_qwen3 for Assist and exposed-device control.
• ai_task.local_ai_task for structured or free-text generation inside automations.
• binary_sensor.ollama_available as the availability gate.
• A small REST-backed script for prompts that need explicit model and context parameters.

I put the entry point on the dashboard

I added an Ask Ollama button directly below the weather card. The button shows binary_sensor.ollama_available, so an offline workstation is visible before I open a conversation. A tap opens text Assist with the dedicated local pipeline; a hold opens the sensor details.

type: button
entity: binary_sensor.ollama_available
name: Ask Ollama
icon: mdi:robot
show_state: true
tap_action:
  action: assist
  pipeline_id: REPLACE_WITH_YOUR_PIPELINE_ID
  start_listening: false
hold_action:
  action: more-info

I use start_listening: false because this profile has no speech engines; it opens as text chat. Home Assistant also accepts preferred or last_used instead of a specific pipeline ID. The official dashboard Assist guide covers the visual editor, while the dashboard action reference documents those pipeline choices.

I added memory, but not by pretending the model remembers

A fresh Assist chat starts fresh. I wanted the assistant to remember stable details about the house—things like the Bond/Lutron ceiling-fan fix—without dumping old transcripts into every prompt or sending them to a cloud service.

I split that job in two. A short, reviewed house brief is part of the conversation agent’s instructions. For facts that change, I use a native Home Assistant Local to-do list called todo.home_memory. The model can call one exposed script to read it, add a note, or forget a note.

I tested the whole path instead of trusting the assistant’s reply. In one chat I asked it to save a fact, saw the script report success, and independently found the note in Home Assistant’s to-do list. After a full dashboard reload and a new chat, the assistant read the list and recalled the note without access to the earlier conversation.

The script rejects exact duplicates, limits active notes to 40, caps new notes at 500 characters, and reports an error instead of silently discarding old context. I can review or edit everything under Home Assistant’s To-do lists. Completing a note archives it; unchecking it restores it.

That boundary matters. Household context is visible to people who can use this Home Assistant instance, so it is not a private per-user memory store. The prompt also remains manually maintained. Repository changes and my other Codex conversations do not flow into it automatically.

The model can control the house; that does not mean it should improvise

An explicit command worked well: dim the kitchen lights to 30 percent and set the TV mood light to blue. The model made both calls in one turn, and the entity states confirmed both changes.

A vague goal was much less reliable. I asked it to make the kitchen dim and warm for a late-night snack three times. Once it changed the brightness correctly. Twice it changed nothing. In both failures, it still replied as if it had done the job.

That is the failure mode I care about: not an error, but a confident false confirmation. The response payload also showed only the last successful tool call, so I could not treat the assistant’s summary as an audit log. The state of the entities is the evidence.

The fix is a scene, not a more poetic prompt

At the time of these tests, the house had 57 scenes. Exposing a short list of named scenes gives the model fewer settings to invent, although it can still choose the wrong scene or fail to call it. I did not run a controlled comparison of scene selection versus individual device calls. The benefit is that the desired brightness and color settings live in Home Assistant, where I can inspect and test them independently.

That changes the model’s job:

1. Interpret what I said.
2. Choose from a narrow set of known end states.
3. Call one scene.
4. Let Home Assistant—not the model—own the actual device settings.

I also keep locks, security-sensitive controls, and anything with meaningful physical consequences out of the initial exposure set. A language model should earn a wider control surface.

Exposing every entity wastes context and hurts accuracy

The September 12 inventory contained 758 entities, including 40 lights, 63 switches, 24 media players, 57 scenes, and 55 scripts. With too much exposed, the entity list and tool definitions consumed 6,526 prompt tokens before I had said a word. That was 40 percent of a 16,384-token context window.

A curated Assist exposure list improves three things at once: the prompt is shorter, the model has fewer wrong tools to choose from, and more context remains for the conversation. Hiding irrelevant entities is not cosmetic cleanup. It is part of the model configuration.

I let the model extract numbers, then calculate in code

The local model is useful for turning messy text into fields. With reasoning disabled, Qwen read “12,400 sends” correctly, wrote the correct division beside it, and still returned 1.45 percent instead of 0.145 percent in five out of five runs.

The refusal was the best third answer. It noticed that “conversion rate” did not specify whether signups should be divided by sends, opens, or clicks. Reasoning cost about 9.5 times as many output tokens and roughly three seconds on this short test. That is a fair trade for document work, but it still did not produce a usable answer three times out of three.

Reasoning improved the result; it did not make the model safe for unattended arithmetic. I use --think for numeric or multi-step document work, state the denominator explicitly, and still prefer structured extraction followed by deterministic Home Assistant math:

Validate the extracted fields before dividing. A schema can request numbers without proving they are the right numbers. A missing denominator, a zero, or a count copied from a different campaign should not become a plausible-looking percentage.

This is an action-list fragment for an automation’s actions: or a script’s sequence:, not a complete automation. Define report_text before it and replace ai_task.local_ai_task with your own AI Task entity. It sends that report to the configured provider; check that the entity uses your local Ollama server before supplying private text. The fragment creates signup_rate for later actions; it does not send a notification or change a device.

The example assumes whole-number sends and unique signups from the same cohort, with no more signups than sends. It also imposes a one-trillion-count ceiling to reject extreme values. Those are validation rules for this example, not universal campaign metrics. If the source cannot support that interpretation, stop and clarify the metric.

# Action-list fragment: place inside an automation's actions or a script's sequence.
# Define report_text first and replace ai_task.local_ai_task with your AI Task entity.
- action: ai_task.generate_data
  data:
    task_name: extract_campaign
    entity_id: ai_task.local_ai_task
    instructions: >-
      Extract the explicitly stated sends and unique signups for the same campaign
      and time period. Do not calculate a rate or invent a missing count.
      Report text: {{ report_text }}
    structure:
      sends:
        required: true
        selector:
          number: {}
      signups:
        required: true
        selector:
          number: {}
  response_variable: result

- if:
    - condition: template
      value_template: >-
        {{ not (
          result is mapping
          and result.data is mapping
          and result.data.sends is number
          and result.data.sends is not boolean
          and result.data.signups is number
          and result.data.signups is not boolean
          and 0 < result.data.sends <= 1000000000000
          and 0 <= result.data.signups <= result.data.sends
          and result.data.sends == (result.data.sends | int)
          and result.data.signups == (result.data.signups | int)
        ) }}
  then:
    - stop: "Invalid or missing campaign counts; no rate calculated."
      error: true

- variables:
    signup_rate: >-
      {{ (result.data.signups / result.data.sends * 100) | round(3) }}
# signup_rate is a percentage, available to subsequent actions in this sequence.

An AI Task failure stops the sequence by default. If a response arrives with invalid or missing counts, the explicit error stop prevents the calculation; inspect the script or automation trace rather than treating the previous result as a fresh one. Required fields and the validation step do not detect a model that invents two believable integers. Check important results against the source report.

For a quick arithmetic check, 18 signups divided by 12,400 sends should produce 0.145 percent after rounding. Zero signups with a positive send count is valid; zero sends is not a defined rate. Offline YAML and Jinja tests cover 24 valid and invalid input cases and two arithmetic results. These checks do not call Ollama or run an automation in Home Assistant.

The AI Task documentation describes structured responses; the script documentation covers variables, conditional steps, and stopping with an error.

Measure generation separately from waiting for an answer

For a comparison on your own machine, keep the model digest, prompt, context length, thinking setting, output limit, and concurrency fixed. Record the Ollama version and GPU placement from ollama ps. Run repeated requests rather than selecting the fastest response, and keep cold loads separate from requests made while the model is resident.

The Ollama response fields separate model loading, prompt processing, and output generation. Compute output tokens per second as eval_count / eval_duration * 1000000000, because durations are in nanoseconds. That rate excludes loading and prompt processing. Measure client wall-clock time separately for the wait the user experiences; a streamed first token is not a completed answer. My timing table does not report percentiles, first-token latency, or enough repetitions to establish a general performance range.

Pinning the model trades VRAM for responsiveness

A cold request took about seven seconds. Once the model and its 6,500-token tool prompt were cached, the same path took about two. I set keep_alive: -1 to keep it resident, which holds 13.8 GB of VRAM even while the GPU is almost idle.

That is a good trade while I am testing voice control and a bad one before launching a game that wants the same memory. A five-minute keep-alive would return roughly 11.5 GB after an idle spell, at the cost of a seven-second first response next time.

The setting also has to match on both the conversation and AI Task subentries. When mine differed, switching between them forced a reload and even changed the active context size. They now use the same model, 16k context, history limit, thinking setting, and keep-alive value.

What I use it for now

The Home Assistant side is for short language tasks and carefully bounded control: choosing a scene, generating a notification, answering a question about exposed state, or extracting structured fields for an automation.

Long documents do not go through Home Assistant. I use a small command-line client that sends local files directly to Ollama. A 3,500-token document is read in about two seconds on this hardware. Keeping that workflow separate avoids turning Home Assistant into a batch-processing dependency for no reason.

Would I build it this way again?

Yes, with the same boundaries—but I would not buy an RTX 5070 Ti solely to run Home Assistant. The card makes sense here because it already lives in a desktop that can use it for other work. It can run this particular 14B model fast enough for an assistant, and local processing makes private household context easier to justify. Someone buying hardware only for a house assistant should compare the full machine cost, idle power, and cold-start tolerance first. I keep that decision separate in what I would buy again for Home Assistant.

The useful version is not an all-knowing house brain. It is a fast language layer in front of a smaller set of deterministic Home Assistant actions—with availability checks, curated exposure, scenes for important end states, and ordinary code doing the arithmetic.

Disclosure: I bought and run the hardware measured here.

Clarification, October 10, 2026: the timing observations do not isolate LAN overhead, the model-weight budget is setup-specific, and scene selection was not benchmarked against individual device calls.