Flukso To Pvoutput - (bypassing the flukso server) - Script Discussion

Bookworm 12
I ran the commands as requested.
This is what it gave
https://imgur.com/a/vV1VVNu

Is it ok?

It works now, will activate #V3 script in my test enviroment -Thanks

@ciarly
It won’t upload if there’s nothing to send…
What are you trying to do?

Doesnt look installed?

Does the script work?

I’m not sure this is past my knowledge.
Probably can be installed in an virtual environment but I don’t know much about that.

Try
sudo apt install python3-pip python3-dev libffi-dev

sudo pip3 install paho-mqtt

python3 -m pip show paho-mqtt

Let me know If that doesn’t work maybe we need to install python 3.9.5 in a virtual environment

OK test script (V3) is now running in my test enviroment. - Thanks

1 Like

@ B-Man
I will buy you brekky and a coffee :slight_smile:
Getting 2 green traces on PV output .
I may of sensor put code in wrong V areas.
For the Flukso what V do you put each sensor in please?
Baz

1 Like

@bazzle

im not sure what you mean does this help?

SENSORS = [
    # Sensor 1
    {
        "id": "sensor-id", 		             # Sensor ID - Flukso Sensor id # replace the "sensor-id" section (retaining the quotation marks)
        "type": "gauge",                            # Sensor type: counter or gauge (I recomend gauge for power and counter for water)
        "pvoutput_v": "v2"                        # PVOutput Power Value (v1 - v12) # adjust to what suits see list below for "v" values.
    },									
    # Sensor 2							
    {									# https://pvoutput.org/help/api_specification.html
        "id": "sensor-id",						
        "type": "gauge",						# v1 - Energy Generation - kWh
        "pvoutput_v": "v4"						# v2 - Power Generation (live) - W
    },									      # v3 - Energy Consumption - kWh
    # Sensor 3							        # v4 - Power Consumption (live) - W		
    {									      # v5 - Temperature
        "id": "sensor-id",						# v6 - Voltage
        "type": "gauge",						# v7 - v12 - Extended Data 
        "pvoutput_v": "v7"						  
    },									 
    # Sensor 4							        # Counter for water/gas may require a rule pending pulse output.. 
    {									        # 1 counter is 1 pulse per _ L
        "id": "sensor-id",						# my meter is 0.5L per pulse so I have a rule "v7": "v7 / 2", # To display actual L used
        "type": "gauge",
        "pvoutput_v": "v8"
    },
    # Sensor 5
    {
        "id": "sensor-id",
        "type": "counter",
        "pvoutput_v": "v10"
    },
]

this is an example

# Sensor 1
{
    "id": " 32 character sensor id goes inside these quote marks", 		             # Sensor ID - Flukso Sensor id # replace the "sensor-id" section (retaining the quotation marks)
    "type": "gauge or counter goes in here gauge for power, counter for water",                            # Sensor type: counter or gauge (I recomend gauge for power and counter for water)
    "pvoutput_v": "this is a value of v1 to v12 you can see the comments of what you want dont use capitals"                        # PVOutput Power Value (v1 - v12) # adjust to what suits see list below for "v" values.
},							

Use v2 for solar and v4 for consumption. v1 and v3 get automatically worked out by pvoutput

1 Like

@B-Man Thank you. I was using 1 and 3 and getting no where fast. I must of swapped sensors and other V’s around 20 times with strange results but must of missed that combination and then having to wait ~10 mins to check readings each time.
Solar gone off here now so will need to wait til tomorow but power in use OK :wink:
Your help is very much appreciated :slight_smile:
Flukso site gone again too…
Baz

Great news.

I should make a note somewhere about what pvoutput v values to use… I set it up with 2 and 4 so they were already setup but didnt mention to use those values. i just assumed people would know what values done what but if you haven’t worked with any scripts before i guess you wouldn’t know.

If its not too much trouble can you let me know what problems you had/where you realised you went wrong/ how you fixed it/ so I can update the readme to make it easier to understand?

did the last lot of commands get paho mqtt working or did you just test the script and it worked?
or was it fine from the start and you just had the sensors id’s wrong?

and yes it must have gone down again some time today as it was there this morning but not at lunch.
that why I chose this forum to discuss it. and its pvoutput related aswell…

I found an issue with the backlog data not being sent. :roll_eyes:
I did have it working but something I have changed from v1 has broken it and I didnt test it.
I built some data up overnight and have noticed it wont send it… hopefully a fix for it soon…

how did V3 go with the rules?

yes, that seemed to be the fix :slight_smile:

@B-Man

Well it makes a BIG difference when we run the right script. I finnished testing it today and I can now confirm that all custom scripts now work as expected without using the extended attributes 11 and 12.

So the rules we test are:

CUSTOM_RULES = {
“v4”: [“v4 * 0.89”,"(v4 + v2) if v2 < 50 else v4"],
“v2”: [“v2 * 0.89”,“0 if v2 < 50 else v2”],
}

Here is the debug file showing the rule activating as the inverter shutdown this evening.

2024-06-14 17:13:50 - Initial payload: {‘d’: ‘20240614’, ‘t’: ‘17:13’, ‘v1’: 0, ‘v2’: 65, ‘v3’: 0, ‘v4’: 683, ‘v5’: 0, ‘v6’: 0, ‘v7’: 0, ‘v8’: 0, ‘v9’: 0, ‘v10’: 0, ‘v11’: 0, ‘v12’: 0}
2024-06-14 17:13:50 - Adjusted payload: {‘d’: ‘20240614’, ‘t’: ‘17:13’, ‘v1’: 0, ‘v2’: 57.85, ‘v3’: 0, ‘v4’: 607.87, ‘v5’: 0, ‘v6’: 0, ‘v7’: 0, ‘v8’: 0, ‘v9’: 0, ‘v10’: 0, ‘v11’: 0, ‘v12’: 0}
2024-06-14 17:13:51 - Sent data to PVOutput: 200 OK 200: Added Status
2024-06-14 17:13:51 - Payload: {‘d’: ‘20240614’, ‘t’: ‘17:13’, ‘v2’: 57.85, ‘v4’: 607.87}
2024-06-14 17:16:09 - Initial payload: {‘d’: ‘20240614’, ‘t’: ‘17:16’, ‘v1’: 0, ‘v2’: 45, ‘v3’: 0, ‘v4’: 1325, ‘v5’: 0, ‘v6’: 0, ‘v7’: 0, ‘v8’: 0, ‘v9’: 0, ‘v10’: 0, ‘v11’: 0, ‘v12’: 0}
2024-06-14 17:16:09 - Adjusted payload: {‘d’: ‘20240614’, ‘t’: ‘17:16’, ‘v1’: 0, ‘v2’: 0, ‘v3’: 0, ‘v4’: 1224.25, ‘v5’: 0, ‘v6’: 0, ‘v7’: 0, ‘v8’: 0, ‘v9’: 0, ‘v10’: 0, ‘v11’: 0, ‘v12’: 0}
2024-06-14 17:16:10 - Sent data to PVOutput: 200 OK 200: Added Status
2024-06-14 17:16:10 - Payload: {‘d’: ‘20240614’, ‘t’: ‘17:16’, ‘v2’: 0, ‘v4’: 1224.25}

You can see the system on pvoutput here:
https://pvoutput.org/intraday.jsp?id=3880&sid=3051&dt=20240614&gs=0&da=1&dxa=1&m=0

I notice the script behaves a little different to the previous version. It now uploads to pvoutput after 5 min from starting the script rather than on each 5 min mark.

I set up my test system on pvouput with a very low sysrem PV capacity and as my genneration increased, pvoutput rejected my upload as it said that amount of genneration exceeded what would be normally expected. Your script in response sent the data to the log file. When I corrected the system size the script sent the current upload the sent some of the backlog updates. However pvoutput did not display them, I think because the data was time stamped in the past ?? I can send the debug entries if you would like to see the actual data.

I have started some documentation of using the Flukso Version 2 and 3 with your custom rules.
This is only a draft at this stage and I need to add the Flukso version 3 into the document. It will give other Flukso owners why we have spent do much time on the Custom Rules for both V2 & V3 Flukso, so its worth looking at before making embarking on this project.

Draft only remmember, I hope to finnish it over the weekend before my cateract operation next week.

I am still very impressed with your script, for someone who has not used python before its VERY impressive. Makes me wonder what you do for a living.

It is also my first time seeing a python script. Half my working life was with electonics as a communication technician and the other half was in IT. The scipting I did included, Unix shell scripts, Windows batch files, ADSI scripting, Visual Basic, Gawk, Awk, Perl scripts. I think I am showing my age with some of those.

Bottom line I am impressed with you Python script seeing it was your first script.

Thanks For your efforts
Chris

1 Like

i did create a small guide to the rules section. that also located in the patch-1 link.

yes im having alot of trouble sending backlog data. its been doing my head in!. im not sure when it broke…

im about to go back to my old code and see what I done as im sure it worked to start with!

at least the code is working and the data from the backlog can manually be added if required in the mean time.
im just glad the rules worked!

yes user error… have alot of that going on…

I have updated the script.
same link as before. its v3.1 at the top.
you can keep all the same sensor data, rules etc.

To apply these changes, copy everything from the end of the “Debug mode” code and below.

just need to copy from the end of the “Debug mode” code and below…
I sectioned debug off as it is sort of user configurable…
I have updated the script to version 3.1. The same link as before still applies. You can keep all the same sensor data, rules, and configuration.

Key Changes:

  • Debug mode sectioned off for clarity.
  • Added notes about the debug mode. (handy for creating a 10 second loop)
  • Provided additional notes for clarity.
  • Fixed the issue of not aligning to the 5-minute interval, which was due to commenting it out (for debugging) . Now, the script checks the debug time: if it equals 0, it aligns to the 5-minute interval; otherwise, it runs at the debug time.
  • Corrected the backlog issue. Initially, I tried to make the backlog file more readable by using new lines, but that broke the functionality. This has been redone correctly. sadly cant seem to get it to run with a new line.
  • Fixed HTML code formatting that was displayed on failure to send data to PVOutput and the backlog.
  • Made several other improvements, though I can’t recall all the specifics.

I have tested this and seems to be working as expected… although that’s what I thought about the other version.

in the debug log if you get an Error in main loop: 'v1' ignore it… I know what is causing it but it doesn’t affect anything… its when I remove all values that don’t have a sensor and are = 0. it just stops sending 0 values for all “v” values. (work around for another issue I had).

Hi guys.
This is the error reported in the debug, but I don’t know what to do.
" 2024-06-14 23:50:00 - Initial payload: {‘d’: ‘20240614’, ‘t’: ‘23:50’, ‘v1’: 0, ‘v2’: 22, ‘v3’: 0, ‘v4’: 392, ‘v5’: 0, ‘v6’: 0, ‘v7’: 0, ‘v8’: 0, ‘v9’: 0, ‘v10’: 0, ‘v11’: 0, ‘v12’: 0}

2024-06-14 23:50:00 - Error in main loop: eval() arg 1 must be a string, bytes or code object"

thanks

@ciarly I could take a wild guess, that error is the same as I had when my custom rule commands were not compatable with the script version. Double check which script you are running.

@B-Man Noted on the updates to the script will download it (Takeing care I get the right one) might have time to test it my test enviroment today.

@ciarly
Quite possibly is a rule issue.

You can try the patch-1 script (v3.1 at the top) but will need to redo the rules in the correct format.

Link above

@cwjames
Cheers. I hope it’s all good

@B-Man
Is it ok to keep asking q’s in this thread?

PVoutput graph is showing my solar input ok but graph label says Energy used (red)
I dont see a 2nd graph trace.
I have sensors configured in pv—mqtt.py file and they also show as working in Terminal.


Any ideas? Can you give me screen dump of your settings in PVoutput for the flukso please?

Your second sensor is labelled “v8”, which is an extended user-defined parameter. PVOutput expects power (not energy) generated as “v2”. Energy is v1, but the Flukso emits power values in its Mosquitto events.

You can graph v8 separately by tinkering with the settings of your PVOutput account. But that is not normally used, unless you want to graph some other source of data that PVOutput doesn’t cater for.

So, you should have the Flukso sensor as v4 (which you already have). That should show you consumption.

For generation, your sensor should specify v2 if your solar system sends power data, or v1 if it sends energy data.

@B-Man A quick update, testing of V3.1 scripts show all is working as expected in my test enviroment. Custom Rules only using v2 & v4. The script now uploads to pvoutput on each 5 min time slot as before. The only thing I did not confirm in real world was the backlog function. So all OK here.

@bazzle

Looking at your sensors code you enclosed, you have not assigned V2 (Power Generation) to any sensor-id and then you have assigned sensor 4 to v8 (Which is an Extended attribute). Not knowing your system I guess if your remove the sensor ID assigned in Sensor 4 and assigne it to Sensor 1.

image

I think michi has seen this too…

I hope I have not confused you.

Thanks Chris