I’d seen this but it seemed a very complicated solution 
Bob.
1 Like
@jrbenito, I’ve made a small change to your code to stop all the r.status_code = 200 messages being written to my log file. I’m not sure if you’re interested in any changes I make though?
Bob
if r.status_code != 200:
print localnow().strftime(’%Y-%m-%d %H:%M’), r.status_code
I would like to make a remark to the shine wifi adapter to pvoutput solution https://www.photovoltaikforum.com/download/file.php?id=42091 mentioned by jrbenito.
The latest editions of the shine wifi adapter serial nr starting with IUB is not working with this solution due to changes in the way data is transmitted to Growatt. I spent the last two weeks in trying to get this to work and the first trick i had to find out is how to reroute the data to my Pi. After reading this file https://www.pvo-int.com/wp-content/uploads/2018/06/Troubleshooting-Guide-for-Growatt-Monitoring-devices.doc i managed to reroute the data. WARNING if you make a mistake in changing the IP the connection is lost and it is saved in to the firmware of the logger so a factory reset will not recover the connection.
Finaly i managed to use wireshark to view the data and with help from someone using this solution with an older version of the wifi adapter i compared the data sent by the different versions. The data sent by the new versions of the logger is totaly different.
Hi @jrbenito, I’ve been running your script for 12 days now and everything looks ok with the exception that I’m occasionally getting this error:
Traceback (most recent call last):
File “/home/pi/canadianSolar-pvoutput-master/canadian_reads.py”, line 262, in
main_loop()
File “/home/pi/canadianSolar-pvoutput-master/canadian_reads.py”, line 238, in main_loop
pvoutput(inv, owm)
File “/home/pi/canadianSolar-pvoutput-master/canadian_reads.py”, line 195, in pvoutput
r = requests.post(url_status, headers=headers, data=payload)
File “/home/pi/.local/lib/python2.7/site-packages/requests/api.py”, line 112, in post
return request(‘post’, url, data=data, json=json, **kwargs)
File “/home/pi/.local/lib/python2.7/site-packages/requests/api.py”, line 58, in request
return session.request(method=method, url=url, **kwargs)
File “/home/pi/.local/lib/python2.7/site-packages/requests/sessions.py”, line 512, in request
resp = self.send(prep, **send_kwargs)
File “/home/pi/.local/lib/python2.7/site-packages/requests/sessions.py”, line 622, in send
r = adapter.send(request, **kwargs)
File “/home/pi/.local/lib/python2.7/site-packages/requests/adapters.py”, line 513, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host=‘pvoutput.org’, port=80): Max retries exceeded with url: /service/r2/addstatus.jsp (Caused by NewConnectionError(’<urllib3.connection.HTTPConnection object at 0x75e0ccd0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,))
python script erro, sleeping few seconds and call it again
Any idea what the problem might be?
Bob.
I totally agree!
Sure! Any improvements are welcome.
I let the logs behind, this is one thing I would like to change.
Too bad!
A friend bought a Growatt recently (sorry, I don´t have the model) and his unit has RS232 where the Wifi dongle is plugged-in and an RS485. This interface can be used to monitor the unit and/or chain it with other inverters and monitor all with just one cable (RS485 is default industrial interface for these applications). If your inverter has the RS485 available, you can keep the wifi dongle on RS232 and use the RS485 to monitor and send data to pvoutput as proposed on this thread. The only difference is that you would need as USB/RS485 converter instead of USB/RS232. Protocol is the same (ModBus RTU), just physical interface is different but with USB converter, both appears to the Pi as a Serial port. In other words: change RS232 to RS485 with my code shall be a matter of change the USB dongle only!
If further assistance is necessary on this path I would gladly help.
PS: my unit is a Canadian Solar re-branded Growatt and although it mentions RS485 in its manual, there is no port exposed to user outside inverter; I did not open it to see if it is there or is optional (manual does not talk about this). However my friend´s Growatt unit has both ports exposed under the unit case. If yours is like that you are in luck and can have both wifi and pvouput independently.
Hi Bob,
The problem is in fact two issues:
-
This exception should be caught by my code… shame on me
In fact, I see this happening very rarely and never gave the attention needed. -
The main issue not being treated here is that for some reason program cannot contact pvoutput server. This may occur due poor internet connection like DNS not answering in time, HTTP timeout due momentarily slow connection or lost internet connectivity. Any issue that may happen between your RPi and pvoutput server that lasts long enough to timeout python´s HTTP library.
While this is perfect normal on the internet, on a good internet connection it shall happen seldom. For instance, my mom´s place used to have slow internet on copper (DSL like but poor quality provider), she switched to a fiber provider with faster and more reliable connection and this kind of issue became more rare. However, this is only your side of the issue, something may happen on pvoutput server like a spike in income traffic or in the path between your provider and pvoutput servers. What I am trying to say is “no matter what, soon or later, a connection error will happen and that shall be caught and retried on software”.
I will try…catch this error as soon as possible.
1 Like
I had thought that there might be problems with Internet outages. I have a robust and fast Internet connection but there is always the chance of an outage or maybe PVOutput is taken down for maintenance.
I also have a SMA inverter and use SBFspot (https://github.com/SBFspot/SBFspot/wiki) to upload the data to PVOutput. SBFSpot writes the inverter data to a SQLite database and then a second program uploads the data from the database to PVOutput. If the upload isn’t successful then it keeps retrying until it succeeds and this works very well. I had thought that I might try something similar with your code although I’d have to learn quite a lot of new stuff so it would take some time.
To test this, do you know if I could have two programs querying the inverter at the same time without messing things up? If this is possible I’d leave your original script running and develop the part that writes the data to the database first. I could then test that was running ok before developing the database to PVOutput upload part.
Any thoughts?
Bob.
The SQL Lite idea is not bad and not too difficult to implement; I thought of it a bit different by buffering requests on memory like a queue (a FIFO). But SQL Lite could be a good alternative that brings backup as bonus.
To test this, do you know if I could have two programs querying the inverter at the same time without messing things up?
No and yes!
No, serial ports are not meant to have concurrent access.
Yes, it will work but sometimes it may conflict and fail readings on both sides. I do this to test sometimes but you can try reading at same time the other program is doing its stuff. In this case both programs will result in invalid readings. For sporadic test it “ok” but for long running it is not good idea.
When you have one datasource like the inverter and want many consumers of that data, the best approach is to use a proxy agent in the middle. Most IoT home projects today uses MQTT protocol to buffer between datasources and data consumers. It would be simple to break the code into “inverter reader” that reads inverter and post data to somewhere (MQTT or SQLite or Redis) and another program that subscribe to that data, process and send it to another place. In fact, this architecture (using MQTT or Redis) would allow for fun things like real time dashboard, alarms, etc…
In fact it was not your internet but the DNS server used by your network: “Temporary failure in name resolution”
@bobboulby, I just pushed correction code, could you please give it a try? I could not push it to production yet, will do later today. I also use your suggestion of log only errors and caught most commons exceptions printing informative errors instead of crash 
Check tag v0.6.8
Running it now. All ok so far but will monitor over the next few days.
Bob.
Hi again @jrbenito
Just to report that the latest version of your code has been working fine for the last 12 days. I’ve had a couple of DNS errors reported in my log file and these have been trapped correctly by your code.
One thing I have noticed however is that each day I loose one 5 minute value (for an example see the attached image for 2:25PM today).
There is nothing in the log file to indicate any kind of error when this happens. It happens pretty much every day, usually in the afternoon.
Any ideas what might be happening?
Bob.
Hi @bobboulby,
Does gaps match time and date of DNS errors? Notice that the way code is right now it does not retry send data to pvoutput; it just don´t crash in case of an error. Hence, if an error occurred while sending 2:25PM status, it will sleep 300 seconds and send new data. I do have a new version with a for loop doing 3 times retry before giving up. I will upload it as soon as possible.
I too have one gap, usually arround 13 localtime but sometimes around 17, almost every day. Actually 7 days with one gap in the last 12 days (08/24 to 09/05). I will look into logs later as arrive home.
Another issue:
I have some ideas to buffer data not sent (due any problem with internet) and sending than after. I am working on another project and as soon I have sometime I will try to implement it.
Benito.
No there is nothing in the log file at the same time as the gaps.
Thanks, will try this, probably at the weekend.
Bob.
Hi @bobboulby,
Yesterday I went to check logs and see if my gaps leaved any trace behind but I noticed I did not update code to the latest version on my mom´s inverter. Guess what, I did not. Anyway, I put version v0.6.9 to run yesterday and tomorrow will be holiday here in Brazil so I will be at home while inverter is working. Good opportunity to do some debug.
BRs.
1 Like
Back again 
I’ve just implemented v0.6.9 so will monitor over the next few days. Yesterday (while still running 0.6.8) I had two missed 5 minute values one at 12:40PM
and one at 3:35PM
My log file for yesterday was:
2018-09-06 20:00 - Next shift starts in 660 minutes
Error: %s
Error: %s
Error: %s
(‘2018-09-07 12:42’, ‘Error Connecting:’, ConnectionError(MaxRetryError(“HTTPConnectionPool(host=‘pvoutput.org’, port=80): Max retries exceeded with url: /service/r2/addstatus.jsp (Caused by NewConnectionError(’<urllib3.connection.HTTPConnection object at 0x75ea21b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution’,))”,),))
Error: %s
2018-09-07 20:00 - Next shift starts in 660 minutes
so there was an entry in the log for 12:40PM but not for 3:35PM. Not sure if this helps with debugging?
By the way I’ve not mentioned the “Error: %s” in the log file before - I get a few of these each day. Looking at the code it seems to be an error while getting the temperature from OWM so I’ve not really worried about it.
Thanks for your ongoing support by the way.
Bob
I’ve rolled back to 0.6.8 this morning. Yesterday running 0.6.9 I had five missing 5 minute values plus the else condition in your new code is being activated every 5 minutes. This is a sample of my log file:
(‘2018-09-08 19:25’, ‘Could not send data after some attempts’)
(‘2018-09-08 19:30’, ‘Could not send data after some attempts’)
(‘2018-09-08 19:36’, ‘Could not send data after some attempts’)
(‘2018-09-08 19:41’, ‘Could not send data after some attempts’)
There were no DNS errors yesterday.
Bob.