Help Required Uploading Growatt Inverter To PVOutput

Inverters Growatt
#12

No problem, thanks.

Bob.

#13

Hi Bob,

I´ve been in my mom´s home and done some work to the script. It ended up in a spaghetti code so I decided to rewrite it organizing in a way I can, in future, break it into modules. It is not perfect but still works and little better now. There are bugs and corner cases I did not account - sure! But now I have VPN connection to the RPi and deal with issues will no longer take months.

Timezone issue ended up being docker “issue”. I did not realize that docker image has default UTC timezone so I made python code timezone aware and added proper configuration to the pvoutput.txt.

The code now saves energy reported between two reports (as I described early) what decreased occurrences of “0W average followed by 1200W or 2400W average”. However, sometimes it still occurs, I will debug overtime to understand why. Anyway, I still have some tests to do since this version of script was not running (my mistake) and entered production today.

Also, I added some code to sleep over night not trying to query inverter between 21:00 and 5:00 (I chose these hours because on daylight saving period, on winter it will waste some hours polling inverter). This is cosmetic but helps to save CPU cycles on the PI while inverter is turned off. One idea is to get sunrise and sunset times from Openweathermap instead of fixed hours like I did. Warning: this was not extensively tested simple to remove.

“Well, shut up and show me the code!” You might be thinking now.

Here you have it.. Hope it serves as inspiration. Please, post any comments here but open issues on github if you find bugs (I know they are there).

BR,
Benito

1 Like
#14

Thank you very much. I’m away from home at the moment but will have a good look at this soon. It may take me a while to understand and get it up and running but I’ll report back and let you know how i get on.

Bob.

#15

Fantastic. If you need any help drop me a message.

One thing I did not mention, although I use docker (for portability), script itself is not docker dependent so don´t worry about docker stuff mentioned in readme if you are not interested in it.

#16

One quick question, I need to get one of these, is that correct? https://smile.amazon.co.uk/UGREEN-20210-Converter-Connecting-Controller-grey/dp/B00QUZY4UG/ref=sr_1_1?ie=UTF8&qid=1533134028&sr=8-1&keywords=RS232%2FUSB+converter+cable

Also does it matter which USB port I use on the Pi?

Bob.

#17

Yes, you need a USB to RS232 converter like that one (expensive!!! You may find cheaper yet good options). The USB port does not matter, probably your Pi will recognize it on /dev/ttyUSB0 and script has it hardcoded, change if necessary. (to find out what device was associated with cable, after insert it, run dmesg command on Pi, it will show you the new device information - 99% chance it will be ttyUSB0)

Sidenote: other option for USB/RS232 is a TTL/RS232 converter direct attached to PI serial (pin header on PI), the device would be /dev/ttyAM0 (if I remember). This option is, for sure, less practical requiring wiring and connectors but could be a good option for people with USB power issues or using USB for other things.

Live update Growatt
#18

I’ve got this working (system name Sunny Idle 1.34) so in case anyone else is looking to set it up from scratch I’ve documented the steps I took below:

  1. On my Pi I created the folder canadianSolar-pvoutput-master under /home/pi/
  2. Downloaded canadianSolar-pvoutput-master.zip from GitHub, unzipped and copied contents to /home/pi/canadianSolar-pvoutput-master
  3. From canadianSolar-pvoutput-master folder ran pip install -r requirements.txt
  4. Renamed pvoutput.txt.rename to pvoutput.txt
  5. Edited pvoutput.txt with System Id, API Key, OWM variables and TimeZone=Europe/London
  6. Connected Pi to inverter with USB to RS232 interface lead and rebooted Pi.
  7. $ chmod +x pvoutput.sh
  8. $ ./pvoutput.sh

Data then started being uploaded to my system :slight_smile: I added two extended data variables to my PVOutput system settings - AC Volts and Inverter Temperature (I didn’t include ‘v10’: inv.wh_total).

I’d been doing all this from a ssh session on my laptop and I wanted to get the script to start automatically when the Pi restarts so I created a cron job for user pi with the entry “@reboot /home/pi/canadianSolar-pvoutput-master/pvoutput.sh >> /home/pi/canadianSolar-pvoutput-master/canadianSolar.log 2>&1”. To get this to work I also had to modify pvoutput.sh as follows:

#!/bin/bash
cd /home/pi/canadianSolar-pvoutput-master/ # Needs to be run from the directory where code is #
while true; do
python /home/pi/canadianSolar-pvoutput-master/canadian_reads.py
echo “python script erro, sleeping few seconds and call it again”
sleep 60s
done

So all working at the moment but will monitor over the next few days.

2 Likes
#20

Hi Bob,

Glad to see you mastered it. Congratulations.

#21

Hi All,

I would like to register that there is a way to intercept traffic from Growatt Wifi adapter and send it o pvoutput. I did not tested it myself since I don´t have Growatt Wifi dongle but worth to mention for those who have to keep both, growatt and pvouput, working together.

Here: https://www.photovoltaikforum.com/download/file.php?id=42091

BRs

#22

I’d seen this but it seemed a very complicated solution :wink:

Bob.

1 Like
#23

@jrbenito, I’ve made a small change to your code to stop all the r.status_code = 200 messages being written to my log file. I’m not sure if you’re interested in any changes I make though?

Bob

if r.status_code != 200:
print localnow().strftime(’%Y-%m-%d %H:%M’), r.status_code

#24

I would like to make a remark to the shine wifi adapter to pvoutput solution https://www.photovoltaikforum.com/download/file.php?id=42091 mentioned by jrbenito.
The latest editions of the shine wifi adapter serial nr starting with IUB is not working with this solution due to changes in the way data is transmitted to Growatt. I spent the last two weeks in trying to get this to work and the first trick i had to find out is how to reroute the data to my Pi. After reading this file https://www.pvo-int.com/wp-content/uploads/2018/06/Troubleshooting-Guide-for-Growatt-Monitoring-devices.doc i managed to reroute the data. WARNING if you make a mistake in changing the IP the connection is lost and it is saved in to the firmware of the logger so a factory reset will not recover the connection.
Finaly i managed to use wireshark to view the data and with help from someone using this solution with an older version of the wifi adapter i compared the data sent by the different versions. The data sent by the new versions of the logger is totaly different.

#25

Hi @jrbenito, I’ve been running your script for 12 days now and everything looks ok with the exception that I’m occasionally getting this error:

Traceback (most recent call last):
File “/home/pi/canadianSolar-pvoutput-master/canadian_reads.py”, line 262, in
main_loop()
File “/home/pi/canadianSolar-pvoutput-master/canadian_reads.py”, line 238, in main_loop
pvoutput(inv, owm)
File “/home/pi/canadianSolar-pvoutput-master/canadian_reads.py”, line 195, in pvoutput
r = requests.post(url_status, headers=headers, data=payload)
File “/home/pi/.local/lib/python2.7/site-packages/requests/api.py”, line 112, in post
return request(‘post’, url, data=data, json=json, **kwargs)
File “/home/pi/.local/lib/python2.7/site-packages/requests/api.py”, line 58, in request
return session.request(method=method, url=url, **kwargs)
File “/home/pi/.local/lib/python2.7/site-packages/requests/sessions.py”, line 512, in request
resp = self.send(prep, **send_kwargs)
File “/home/pi/.local/lib/python2.7/site-packages/requests/sessions.py”, line 622, in send
r = adapter.send(request, **kwargs)
File “/home/pi/.local/lib/python2.7/site-packages/requests/adapters.py”, line 513, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host=‘pvoutput.org’, port=80): Max retries exceeded with url: /service/r2/addstatus.jsp (Caused by NewConnectionError(’<urllib3.connection.HTTPConnection object at 0x75e0ccd0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,))
python script erro, sleeping few seconds and call it again

Any idea what the problem might be?

Bob.

#26

I totally agree!

#27

Sure! Any improvements are welcome.

I let the logs behind, this is one thing I would like to change.

#28

Too bad!

A friend bought a Growatt recently (sorry, I don´t have the model) and his unit has RS232 where the Wifi dongle is plugged-in and an RS485. This interface can be used to monitor the unit and/or chain it with other inverters and monitor all with just one cable (RS485 is default industrial interface for these applications). If your inverter has the RS485 available, you can keep the wifi dongle on RS232 and use the RS485 to monitor and send data to pvoutput as proposed on this thread. The only difference is that you would need as USB/RS485 converter instead of USB/RS232. Protocol is the same (ModBus RTU), just physical interface is different but with USB converter, both appears to the Pi as a Serial port. In other words: change RS232 to RS485 with my code shall be a matter of change the USB dongle only!

If further assistance is necessary on this path I would gladly help.

PS: my unit is a Canadian Solar re-branded Growatt and although it mentions RS485 in its manual, there is no port exposed to user outside inverter; I did not open it to see if it is there or is optional (manual does not talk about this). However my friend´s Growatt unit has both ports exposed under the unit case. If yours is like that you are in luck and can have both wifi and pvouput independently.

#29

Hi Bob,

The problem is in fact two issues:

  1. This exception should be caught by my code… shame on me
    In fact, I see this happening very rarely and never gave the attention needed.

  2. The main issue not being treated here is that for some reason program cannot contact pvoutput server. This may occur due poor internet connection like DNS not answering in time, HTTP timeout due momentarily slow connection or lost internet connectivity. Any issue that may happen between your RPi and pvoutput server that lasts long enough to timeout python´s HTTP library.
    While this is perfect normal on the internet, on a good internet connection it shall happen seldom. For instance, my mom´s place used to have slow internet on copper (DSL like but poor quality provider), she switched to a fiber provider with faster and more reliable connection and this kind of issue became more rare. However, this is only your side of the issue, something may happen on pvoutput server like a spike in income traffic or in the path between your provider and pvoutput servers. What I am trying to say is “no matter what, soon or later, a connection error will happen and that shall be caught and retried on software”.

I will try…catch this error as soon as possible.

1 Like
#30

I had thought that there might be problems with Internet outages. I have a robust and fast Internet connection but there is always the chance of an outage or maybe PVOutput is taken down for maintenance.

I also have a SMA inverter and use SBFspot (https://github.com/SBFspot/SBFspot/wiki) to upload the data to PVOutput. SBFSpot writes the inverter data to a SQLite database and then a second program uploads the data from the database to PVOutput. If the upload isn’t successful then it keeps retrying until it succeeds and this works very well. I had thought that I might try something similar with your code although I’d have to learn quite a lot of new stuff so it would take some time.

To test this, do you know if I could have two programs querying the inverter at the same time without messing things up? If this is possible I’d leave your original script running and develop the part that writes the data to the database first. I could then test that was running ok before developing the database to PVOutput upload part.

Any thoughts?

Bob.

#31

The SQL Lite idea is not bad and not too difficult to implement; I thought of it a bit different by buffering requests on memory like a queue (a FIFO). But SQL Lite could be a good alternative that brings backup as bonus.

To test this, do you know if I could have two programs querying the inverter at the same time without messing things up?

No and yes!
No, serial ports are not meant to have concurrent access.

Yes, it will work but sometimes it may conflict and fail readings on both sides. I do this to test sometimes but you can try reading at same time the other program is doing its stuff. In this case both programs will result in invalid readings. For sporadic test it “ok” but for long running it is not good idea.

When you have one datasource like the inverter and want many consumers of that data, the best approach is to use a proxy agent in the middle. Most IoT home projects today uses MQTT protocol to buffer between datasources and data consumers. It would be simple to break the code into “inverter reader” that reads inverter and post data to somewhere (MQTT or SQLite or Redis) and another program that subscribe to that data, process and send it to another place. In fact, this architecture (using MQTT or Redis) would allow for fun things like real time dashboard, alarms, etc…

#32

In fact it was not your internet but the DNS server used by your network: “Temporary failure in name resolution”