PVOutput Uploader for Home Assistant — missed uploads don’t have to mean missing data

Auto Uploader
#1

I’ve built a new Home Assistant integration for uploading sensors to PVOutput. My main requirement was simple: if an upload cannot go through, keep the data and send it later.

A basic scheduled upload can leave holes when the internet drops, PVOutput is unavailable or the hourly limit is reached. This integration keeps its own local five-minute history log and uploads oldest first. Only statuses confirmed by PVOutput count as sent.

You can switch uploading off and leave recording running. When you switch it back on, it catches up. Rate-limit pauses survive restarts, and if the integration itself was stopped while Home Assistant continued recording, it rebuilds the missed history from the recorder.

Everything is configured through the Home Assistant UI: choose your sensors, map them to PVOutput fields, and select how their readings should be combined. Energy counters are converted to Wh since local midnight, with unit conversion handled automatically.

It supports standard fields, donor extended fields and battery data. It can also work alongside an existing uploader such as IoTaWatt, adding battery or other measurements to the statuses it creates.

There are UI options to reupload from a chosen date and time, or import earlier Home Assistant history. Fields can be recorded without uploading them, then enabled and their history sent later.

There are practical limits: recovery needs recorded data, and PVOutput accepts detailed statuses only within its historical window. Older days can be uploaded as daily totals. Hourly Home Assistant statistics can recover older history, although they cannot recreate the original five-minute detail.

The integration is still in beta. I’d welcome people trying it with different solar and battery setups, particularly alongside existing uploaders. Reports of anything that leaves data unsent would be especially useful.

Installation, documentation and source:

2 Likes
#2

Update: it now works in both directions.

Each field can be set to Send, Record only or Download, from a selector on the device page or all at once in the settings. Switching never loses anything: a field keeps its sensor and settings whatever its mode, and a field switched to Download keeps recording its own sensor, so switching back later leaves no gap.
How the modes work: https://github.com/darek-margas/ha-pvoutput-uploader/wiki/Field-modes

Download reads the latest status from PVOutput into Home Assistant sensors, any of v1-v12 (v7-v12 for donors), plus normalised output. That is handy next to another uploader: let IoTaWatt or the inverter send generation and consumption, read them back into Home Assistant, and have this integration add temperature or battery data to the same statuses.

The bigger part is history. PVOutput keeps statuses for a long time (more than a year in my tests), and the integration can now bring that history into Home Assistant as hourly long-term statistics, every column PVOutput has, including what other uploaders sent:

  • A daily download adds yesterday automatically and fetches the two days before again, so late statuses (an outage being caught up, a reupload) are picked up.
  • A one-off download fetches any range from the past. It checks first how far back PVOutput has detail and tells you if part of the range has none.
  • Energy is stored as hourly totals for the Energy dashboard; power, temperature and voltage as hourly mean, min and max, so peaks still show.
  • A field switched to Download gets a live sensor whose statistics are filled from that history, so a sensor created today shows last year.

Home Assistant only takes past data as hourly statistics, so the downloaded history is hourly; new readings keep full detail from the moment a field downloads.

Everything the integration does with history, in both directions: https://github.com/darek-margas/ha-pvoutput-uploader/wiki/History-operations
Details of the PVOutput history download: https://github.com/darek-margas/ha-pvoutput-uploader/wiki/PVOutput-history-in-Home-Assistant

0.1.8 is the current stable release: https://github.com/darek-margas/ha-pvoutput-uploader/releases

1 Like
#3

Another update, now in beta as 0.1.9.

A fix first, thanks to a detailed report on GitHub (https://github.com/darek-margas/ha-pvoutput-uploader/issues/3): daily energy could end up short of the sensor when the energy counter went briefly unavailable, or when Home Assistant wrote a reading to its recorder late. The local log is filled from the recorder in small steps, and an increment falling exactly between two steps was counted by neither. Each counter’s last value is now carried from one step to the next and kept across restarts, so nothing is lost there. Days already affected can be rebuilt from the recorder and sent again with the Import history option: https://github.com/darek-margas/ha-pvoutput-uploader/wiki/Reupload-and-import

New: export and import of the configuration. A site’s field mappings and settings can be exported as YAML and imported on another site or installation. The API key and system ID are never part of an export, so it can be shared as it is. On import you choose whether to add only the missing fields or replace them all, and settings come across only if you ask. Any sensors with different names on your installation are listed on one screen to pick your own.

That makes ready-made setups possible: mappings for a particular inverter or battery integration (Sigenergy, Huawei, Sungrow and so on) that others can import instead of mapping every sensor by hand. If yours works well, please export it and add it:

Betas show up in HACS with “Show beta versions” turned on for the integration. Feedback is welcome, especially from anyone who saw daily totals in PVOutput lower than their inverter’s.

Source and documentation: https://github.com/darek-margas/ha-pvoutput-uploader

#4

Update: the integration is now in HACS’s default list, so installing no longer needs a custom repository.

In Home Assistant, open HACS, search for “PVOutput Uploader”, download it and restart. Or open it straight in your HACS here:

If you added it earlier as a custom repository, there’s nothing to do: it keeps updating the same way.

The current stable version is 0.1.9: no-gap uploads from a local history log, sending any period again, PVOutput’s history back into Home Assistant statistics, and export/import of the configuration. There’s now a first shared setup too, for Sigenergy SigenStor with the Local Modbus integration, contributed by a user. If you have a working mapping for your inverter or battery, please share it:

#5

Thank you for creating this. I installed it this morning using your instructions. I am not interested in uploading data 24/7 to PVoutput, I would prefer to only upload between dawn and dusk. Do you have a suggested script for that?

I seem unable to upload my Sigenergy config to your GitHub using the Pull request. Is there a problem or am I doing something wrong?

#6

Thanks, glad it installed fine.

Dawn to dusk: the Uploading switch only pauses sending. Recording carries on, and when it’s switched back on it catches up with everything it missed (that’s the “no gaps” part), so turning it off overnight alone would still send the night in the morning. To skip the night, clear the backlog at sunrise before switching uploading back on. “Clear queued history” takes everything older than the last couple of upload intervals off the queue; it stays in the local log, just isn’t sent:

automation:
  - alias: "PVOutput: upload only in daylight"
    mode: single
    triggers:
      - trigger: sun
        event: sunrise
        id: sunrise
      - trigger: sun
        event: sunset
        id: sunset
    actions:
      - choose:
          - conditions:
              - condition: trigger
                id: sunrise
            sequence:
              - action: button.press
                target:
                  entity_id: button.YOUR_SITE_clear_queued_history
              - action: switch.turn_on
                target:
                  entity_id: switch.YOUR_SITE_uploading
          - conditions:
              - condition: trigger
                id: sunset
            sequence:
              - action: switch.turn_off
                target:
                  entity_id: switch.YOUR_SITE_uploading

Replace YOUR_SITE with your system’s name as it appears in the entity IDs (both entities are on the PVOutput device page). The sun trigger takes an offset (e.g. offset: “-00:30:00”) if you want a margin. One thing to know: PVOutput then has nothing from you overnight, so night-time consumption and battery discharge won’t show there.

Sigenergy setup: no problem, pull requests aren’t the easiest route. Simplest is to post the YAML from Configure → Export configuration here or in a GitHub issue, and I’ll add it. There’s already one for the TypQxQ Sigenergy Local Modbus integration (on the wiki’s Shared setups page); if yours is for a different integration (e.g. MQTT) or adds fields, say so and I’ll add it alongside.

#7

Thanks for your information. I have decided to leave it running 24/7 as it does contain information that could be a useful reference in the future. If I change my mind, I will use your suggested automation.

Here is the config that I use, I use the HACS Sigenergy config, my system is only 6 months old. If you consider it of any use, by all means put it on your site as a sample, else disregard. I have one value duplicated, inverter temperature as V5 and again as V12 so that it shows on the front page as well as in the extended data.

You can see what it looks like at https://www.pvoutput.org/intraday.jsp?sid=9778
No information is hidden so you can see extended data as well. You can see that my previous implementation did not upload extended data well at all, yours works much better.

pvoutput_uploader: 1
fields:

  • field: v1
    sensor: sensor.sigen_plant_daily_pv_energy_wh
    name: Energy Generation
    mode: send
    value_per_interval: energy_today
  • field: v2
    sensor: sensor.sigen_plant_pv_power
    name: Power Generation
    mode: send
    value_per_interval: average
  • field: v3
    sensor: sensor.sigen_plant_daily_load_consumption
    name: Energy Consumption
    mode: send
    value_per_interval: energy_today
  • field: v4
    sensor: sensor.sigen_plant_consumed_power
    name: Power Consumption
    mode: send
    value_per_interval: average
  • field: v5
    sensor: sensor.sigen_inverter_pcs_internal_temperature
    name: Temperature
    mode: send
    value_per_interval: average
  • field: v7
    sensor: sensor.sigen_plant_battery_state_of_charge
    name: Extended 7
    mode: send
    value_per_interval: average
  • field: v8
    sensor: sensor.sigen_plant_battery_power
    name: Extended 8
    mode: send
    value_per_interval: average
  • field: v9
    sensor: sensor.sigen_plant_grid_import_power
    name: Extended 9
    mode: send
    value_per_interval: average
  • field: v10
    sensor: sensor.sigen_plant_grid_export_power
    name: Extended 10
    mode: send
    value_per_interval: average
  • field: v11
    sensor: sensor.sigen_inverter_battery_average_cell_temperature
    name: Extended 11
    mode: send
    value_per_interval: average
  • field: v12
    sensor: sensor.sigen_inverter_pcs_internal_temperature
    name: Extended 12
    mode: send
    value_per_interval: average
    settings:
    account_type: auto
    upload_interval: auto
    keep_history_days: 91
    daily_output_fallback: false
#8

Thanks, and good to hear the extended data comes through properly now.

Leaving it on 24/7 makes sense: night-time consumption and battery discharge are part of the picture, and the data is there if you want it later.

Your config is now in the repository as a second Sigenergy setup, credited to you with a link to your system, next to the one another user contributed for the same integration:

The two differ in approach: yours uses the daily energy counters and puts battery and grid data in the extended fields; the other uses lifetime counters and PVOutput’s battery fields (b1-b5). Both are listed here:

I only renamed the extended fields to say what they hold (e.g. “Battery state of charge” instead of “Extended 7”). The names are labels for your own reference and don’t change what is sent. The v5/v12 duplicate of the inverter temperature is kept, with a note on why.

Since your system is only six months old, one thing you might like: PVOutput keeps all of that history, and Configure → Download history from PVOutput brings it into Home Assistant’s statistics, so it’s there even if you ever rebuild Home Assistant.

#9

I discovered a minor error in field: v1 which should read “sensor.sigen_plant_daily_pv_energy” with no _wh. When I checked earlier today, the previous entry was flagged as being in error, however it still worked. No idea how or when that error had snuck in. Corrected now. Cheers