Tesla Powerwall2 data to PVoutput?

Energy Monitors Powerwall
#113

OK, thanks. Found the logs now.
Here are the extracts for the latest attempt to start the service.

From the pvoutput log file.

2018-08-30 20:58:19,937 INFO [WrapperListener_start_runner] (Controller.java:109) - *** Starting PVOutput Integration Service v1.5.3.1
2018-08-30 20:58:19,937 INFO [Thread-1] (Controller.java:130) - Install Directory: C:\Program Files (x86)\PVOutput
2018-08-30 20:58:19,937 ERROR [Thread-1] (Controller.java:714) - Startup Error: Missing configuration file: C:\Program Files (x86)\PVOutput\bin…\conf\pvoutput.ini
2018-08-30 20:58:19,938 INFO [Thread-1] (Controller.java:2214) - *** Stopped PVOutput Integration Service v1.5.3.1
2018-08-30 20:58:20,938 INFO [Wrapper-Shutdown-Hook] (Controller.java:2214) - *** Stopped PVOutput Integration Service v1.5.3.1

and from the service log

INFO | jvm 1 | 2018/08/30 20:58:19 | *** Starting PVOutput Integration Service v1.5.3.1
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.runtime.name=Java™ SE Runtime Environment
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.version=25.172-b11
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.vendor=Oracle Corporation
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vendor.url=http://java.oracle.com/
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.name=Java HotSpot™ Client VM
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.specification.name=Java Virtual Machine Specification
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.runtime.version=1.8.0_172-b11
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.awt.graphicsenv=sun.awt.Win32GraphicsEnvironment
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.endorsed.dirs=C:\Program Files (x86)\Java\jre1.8.0_172\lib\endorsed
INFO | jvm 1 | 2018/08/30 20:58:19 | >> os.arch=x86
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.io.tmpdir=C:\WINDOWS\system32\config\systemprofile\AppData\Local\Temp
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.specification.vendor=Oracle Corporation
INFO | jvm 1 | 2018/08/30 20:58:19 | >> os.name=Windows 10
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.library.path=…/lib
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.specification.name=Java Platform API Specification
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.class.version=52.0
INFO | jvm 1 | 2018/08/30 20:58:19 | >> os.version=10.0
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.awt.printerjob=sun.awt.windows.WPrinterJob
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.specification.version=1.8
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.class.path=…/lib/wrapper.jar;…/lib/org.pvoutput.integration.jar;…/lib/commons-logging-1.1.1.jar;…/lib/httpcore-4.1.2.jar;…/lib/httpclient-4.1.2.jar;…/lib/jetty-http-7.5.1.v20110908.jar;…/lib/jetty-util-7.5.1.v20110908.jar;…/lib/jetty-io-7.5.1.v20110908.jar;…/lib/jetty-server-7.5.1.v20110908.jar;…/lib/jetty-continuation-7.5.1.v20110908.jar;…/lib/servlet-api-2.5.jar;…/lib/json_simple-1.1.jar;…/lib/jxl.jar;…/lib/log4j-1.2.15.jar;…/lib/RXTXcomm.jar;…/lib/jackcess-1.2.2.jar;…/lib/bsh-core-2.0b4.jar;…/lib/sqlitejdbc-v056.jar;…/lib/commons-net-3.1.jar
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.specification.version=1.8
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.home=C:\Program Files (x86)\Java\jre1.8.0_172
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.specification.vendor=Oracle Corporation
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.info=mixed mode, sharing
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.version=1.8.0_172
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.ext.dirs=C:\Program Files (x86)\Java\jre1.8.0_172\lib\ext;C:\WINDOWS\Sun\Java\lib\ext
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vendor=Oracle Corporation
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vendor.url.bug=http://bugreport.sun.com/bugreport/
INFO | jvm 1 | 2018/08/30 20:58:20 | *** Stopping PVOutput Integration Service v1.5.3.1 (0)
STATUS | wrapper | 2018/08/30 20:58:22 | <-- Wrapper Stopped

From what I can see, on my installation the directories where the wrapper is looking for the .ini file does not exist.

Garry

#114

The file C:\Program Files (x86)\PVOutput\conf\pvoutput.ini could not be found.

This should not be the case if it was extracted properly. You would have had to edited pvoutput.ini to enter your API key and system Id.

The contents of C:\Program Files (x86)\PVOutput\ should look like the following.

image

#115

OK, well yes that is exactly how it looks, the edited files are in their respective sub-directories but the service will still not keep running and gives the same error messages each time it starts and stops.

ps. I was mis-interpreting the

C:\Program Files (x86)\PVOutput\bin…\conf\pvoutput.ini

…\conf\pvoutput.ini to mean it was a subdirectory of \bin

so yes, my directory structure is exactly as you show above.

#116

Please continue this in the thread below -

Since this problem is unrelated to Tesla Powerwall.

#117

Hi. This is my first day reporting live data from my PowerWall. I have some questions. First, this is what I did:

I took my existing graphing database that I use to create my own graphs at http://ulmo.solar/ and set them up to output to the extended (annual donation only) PVOutput attributes.

I did the update back 90 days, then set it to live update every 5 minutes or so. Plus, daily it goes back and re-uploads any stragler data my sweep up cleaners grab. For now, I’ll have to manually trigger adding missing data that has been found that is longer than around a day back.

You can look at my graphs at https://pvoutput.org/list.jsp?userid=60344

Here’s some questions:

  1. When I am not logged in, I cannot see my battery graphing from the extended data regardless of everything I have clicked on; Live does not show battery data. I set extended data to Public, but it doesn’t seem public.

  2. When I am logged in, I ONLY see my battery graphing data for live, and cannot see the regular solar data for live.

My question for #1 and #2 is how to make both solar and battery extended data both available to the public and to myself regardless of whether or not I am logged in.

  1. When I add some of the figures calculated by PVOutput.Org for my system, there are some descrepancies. I believe this is due to the PVOutput.Org not exactly defining what all the fields mean and what data they need, so when a device such as a battery is introduced, we don’t know what data to submit. I already noticed that what I sent it was a little off, so I re-sent the 90 days of data with a better definition, but it still seems wrong.

At first, I thought Consumption is what I use. Unfortunately, that gave really wacky results on PVOUTPUT, so I don’t think Consumption actually means consumption, but means consumption of solar. That needs to be clarified. Anyway, so I added my actual consumption to the storage that my batteries store, and output THAT addition as “Consumption” of what is taken by the solar panels, even though it doesn’t include what I export to the grid (from anywhere) and it also includes what the batteries take from the grid. In my case, the settings on my system are such that charging from grid or discharging to grid is not yet commonplace, but will be in a few years. Obviously, PVOUTPUT needs to have their calculation system totally revamped to understand each item in the electrical arrays.

Here are the discrepancies I noticed. I’ll use September 8, 2018 for an example.

Date: 2018-09-08
Generated: PVOUTPUT: 30.556kWh; MY RECORDS: 30,557.1658333503Wh: AGREEMENT.
Efficiency: PVOUTPUT: 5.197kWh/kW: cool. I don’t know what that means, but I’m guessing it’s energy per insolation. I’ll have to play with that some time.
Exported: PVOUTPUT: 1.761kWh; MY RECORDS: Exported to Grid: 1,941.64833457023Wh; Imported from Grid: 255.303056792356Wh; Net Exported to Grid: 1,686Wh. I can’t tell if these are kind of close to what PVOUTPUT got or not. I am suspicious that this is wrong. I don’t know what PVOUTPUT means by Exported, or how it gets it.
Consumption: PVOUTPUT: 44.664kWh; MY RECORDS: House use + Battery use: Home 28,037.1697222255Wh + Battery Charging 16,740Wh = 44,777Wh. Since I didn’t have much grid<->battery action, these numbers essentially look correct, but when I start charging from grid and discharging to grid, this will rapidly need updating.
imported: PVOUTPUT: 15.868kWh; MY RECORDS: So, what does Imported mean? I’ll assume it means Imported from Grid to satisfy Home use. That also ignores grid <-> battery; that could add some errors. Here’s my Grid importation: 255.303056792356Wh. That’s right, PVTOUTPUT shows importation of 15.868kWh, and my records show importation of only 0.255kWh, over 62 times as much at PVOUTPUT in meaningful amounts; their data is obviously not what I think this is.

What does their “Importation” mean? Where is it coming from?

I want to understand what the PVOUTPUT “Generated”, “Exported”, “Consumption”, and “Imported” columns mean, how I upload to them, what I upload to them, and how they related to battery use.

Once I know that, I can hack around any bugs, ignore the wrong things that can’t be fixed, and use all the extended attributes as best I can to map that.

On to more bugs:

I hate dates that are a mishmash of numbers like 39/29/10/29/39/43 that mean nothing to me, so I do all dates Big Endian like this: 2018-09-08. But, when I select that date format in the Settings, the graphs all show “2018-” for every date. How do I select a sensible date format that doesn’t just get cut off in the graphs? For now, I have one of the confusing a/b/c/d/e/f types of date fields where everything is jumbled, backwards, and spagetti twisted like some sort of mystical secret coding. It is annoying as hell.

Weather Underground doesn’t allow registering of new data streams. I would like to input data from my closest weather station at KCAAPTOS60; should I scrape that data myself using my own web scraper software and upload those temperatures myself? I suppose that’s the easiest solution.

There is alternate forecast data including cloudiness, insolation, temperature, expected PV output, etc. that I get from https://solcast.com.au/ and I would like to integrate that for PVOUTPUT display; is there an effort underway to do such a thing? Since I already have a profile at SolCast, I think the best way would be a site-wide PVOUTPUT link to SolCast, probably with a special deal between the two for a proper fee for SolCast, to use their API to input the data into PVOUTPUT. Since SolCast has more accurate panel array pointing and stuff like that, I’d like PVOUTPUT not to skimp on integration of the panel parameters, and if anything, to add more parameters to the PVOUTPUT data, or simply use the settings the user has set at SolCast. Then PVOUTPUT could efficiently download and display the data from SolCast. I have difficulty displaying all the data on my website (see for instance http://ulmo.solar/powerflow/center.1.png ), but I managed to figure out that gray can be clouds (yay!), purple is my temperature forecast, and for now I have trouble setting up the right color for me to use for the PV output forecast but went with a march of the number “0” for now. It’s ugly, but it’s there. It’s usually wrong until the last half hour, too, but I show their historical forecasts so we can see how it improves over time. The Australian SolCast doesn’t seem to do much in the way of predicting our microclimate hilly beachside fog very well, almost always getting it way wrong in both directions. But I like to see the PVOUTPUT forecasts as an indication of what is possible from the position of the sun.

A feature missing from SolCast is an ability to give a 3D rendering of the foliage, buildings, and other obstructions around my solar panels, so that they can figure out the shading, as well as how the strings respond to the shading given my particular setup. It would not be too hard given some data to figure that out, but for now, they don’t seem enamored of that quality of output, either. This ends up leaving a few kWh extra in their forecasts every day that I will never get (and don’t want considering I love those trees more).

Any assistance provided would be appreciated!

Solving every problem above seems like a full time job, so I’m not asking anybody to do anything, but any improvement or ideas for improvement you get from my novice questions would of course be great.

#118

Update:

I found the “Views” bar with the color squares. Here’s where I read about it in help:

==================
Views
Below the graph is the View Navigator, clicking on one of the squares will switch to that view. There are five views available -

■ ■ ■ ■ ■   All - This view shows generation, consumption, temperature and voltage.
■ ■ ■ ■ ■   Generation - This view shows generation data only.
■ ■ ■ ■ ■   Consumption - This view shows consumption data only.
■ ■ ■ ■ ■   Net Data - This view shows Net Energy and Power.
■ ■ ■ ■ ■   Extended Data - This view is available when the system has Extended Data.

==================

Once I found that, I was able to see some error message for Extended Data, and then see what the other squares displayed, and realized some were summaries; this allowed me to realize that the “Summary” column for my Extended data was not set up (it was all empty), so when I chose values for that (I chose average for everything), it started putting data into Extended Data for Summary.

I also am able to see the non-extended data when I look at Live now.

What I still need to find out, though, is what all the columns mean, so that PVOutput can properly account for my battteries.

#119

Followup 2:

I can now clearly see that Imported in the main view is wrong. It should not be so high. If you look at my “Extended” view, you should readily see that I do not use any grid on many sunny days when I do not use electric heaters, especially more so in summer, so the high “Imported” is wrong.

I am not sending that Imported to PVOutput; where is it coming from? How is it calculated?

#120

Followup 3:

I found another “Net Data” output section; do I really have to make two reports every time I submit data? That’s twice as many API calls. Botheration! I guess I’ll have to figure out how to make that happen. I’ll have to see if it even works.

#121

Followup 4: as I’m loading the new utility grid import & export data to see if that fixes it, I will also note that I went ahead and programmed my own scraper to grab the temperature as well. This is coming together one piece at a time. I look forward to seeing if the import export data works correctly.

#122

Followup 5: Things are going horribly. PVOutput uses inconsistent terms everywhere:

For one single example, of which there are many: In one spot, there is “Energy Import”. In another spot, the same number being accumulated is “Energy Used”. WHAT THE HELL ARE THEY TALKING ABOUT? IMPORT FROM WHERE? USED WHERE?!

UGGGH!! GRR!!! I can’t make heads or tails of this.

First of all, the fundamentals of these electrical networks is that each object has some number of wires going between it and some number of other items. The wires have properties, the objects have properties, and energy flows between the objects over those wires according to those properties. That’s it. That’s the fundamental basis of this whole thing. But, PVOUTPUT seems to want to make a rigid, incorrect, imprecise, hidden, and convoluted specification of the objects and wires in the network without explaining any of their choices.

If they could treat each object and wire appropriately, then it would be very simple: an object of type solar panel with these attributes, and the wires to and from it of these parameters. Object of type inverter, object of type battery, object of type utility grid, object of type car charger, object of type house. Templates could present simple networks like grid,inverter,PV,house, or allow more complicated networks as needed. Here, the abstraction is everything is an object or wires, each with properties, each linked to other objects or wires. You could even abstract wires into objects, so that wires are objects, then every object would just be a object with properties and connections, and make it even simpler.

But if they try to abstract it in the other direction to everything being a photovoltaic panel, then call the outputs all of these weird spagetti names, then the result is a programming mess with lots of different people interpreting a lot of different things in different ways, and nothing works. That’s what I’m seeing now.

#123

This has been fixed, extended data will now show when not logged in.

For other issues, please raise separate topics as they are unrelated to Telsa Powerwall uploads.

#124

let me know if you get net metering working… I love what you have done with all your graphs and prediction stuff - I have two PV’s at https://pvoutput.org/intraday.jsp?id=61344&sid=55208 both extended data with powerwall2.

#125

Hi,

I would appreciate some help with getting this set up as I have been going round in circles all night with the error below. I managed to get the Window installer setup and working fine for a little over a day and then decided I would follow the additional steps to get the solar generation showing on the default page, what a mistake that turned out to be!

As soon as I got the second (solar generation) service running the timings on the Standard View were one hour ahead of the actual time (the timings on the original / extended view were correct), this is despite the values in the PVOutput.log and Log file showing the correct time. Because of this the original service stopped working as it was recognising on the status check that there was already an entry at a later time than it was attempting to upload.

Steps I have taken so far:

  • Reinstalled second service & followed config steps again to make sure I hadn’t missed anything
  • Reinstalled original service
  • Uninstalled second service
  • Waited for the hour to go by such that the original service could attempt to upload results

I am now in a situation whereby I get the following repeating error on the the original service:

2018-09-30 02:03:05,502 INFO [Thread-1] (Controller.java:731) - Startup Complete: Waiting for data…
2018-09-30 02:03:05,502 INFO [Thread-1] (Controller.java:1806) - >>> 20180930,02:05,-1,-1,-1.0,-42,-1000.0,248.8,-100,-42.849,43.005,49.989,5.213
2018-09-30 02:03:05,502 INFO [Thread-1] (WebClient.java:133) - >>> http://pvoutput.org:80/service/r2/addbatchstatus.jsp?data=20180930,02:05,-1,-1,-1.0,-42,-1000.0,248.8,-100,-42.849,,43.005,49.989,5.213
2018-09-30 02:03:05,764 WARN [Thread-1] (Controller.java:1838) - <<< [400] Bad request 400: Missing energy and power values [20180930,02:05,-1,-1,-1.0,-42,-1000.0,248.8,-100,-42.849,43.005,49.989,5.213]
2018-09-30 02:03:05,764 INFO [Thread-1] (Controller.java:1960) - <<< [400] 20180930,02:05,-1,-1,0

I have uninstalled and reinstalled the original service, I have deleted all live outputs from my system, I have even created a new system and gone through the entire config again from step 1, but no matter what I now do I always get this same error.

Can anyone shed any light on where I am going wrong?

Thanks!

#126

The system has the Daylight Savings setting enabled and will adjust for DST when data is uploaded. This setting should only be enabled if the uploader does not support DST. PVOutput Integration Service supports DST so this setting should be disabled.

image

The 400 errors are due to trying to upload extended data without generation/consumption data.

  • The first service should be uploading consumption and extended data.
  • The second service should be uploading solar only with extended data disabled.
#127

Thank you for your reply - I have not yet installed the second service again but have changed the DST setting for when I do.

As for the original service I have had it working for approximately one hour before it started to throw the 400 error repeatedly, e.g. the uploads for 19:30 BST through to 20:20 BST worked fine but the upload for 20:25 and subsequent entries failed:

2018-09-30 20:24:04,865 INFO [Thread-1] (Controller.java:1806) - >>> 20180930,20:25,-1,-1,-1.0,-5,-1000.0,246.7,-10,-5.376,-13.42,63.773,4.126
2018-09-30 20:24:04,866 INFO [Thread-1] (WebClient.java:133) - >>> http://pvoutput.org:80/service/r2/addbatchstatus.jsp?data=20180930,20:25,-1,-1,-1.0,-5,-1000.0,246.7,-10,-5.376,,-13.42,63.773,4.126
2018-09-30 20:24:05,148 WARN [Thread-1] (Controller.java:1838) - <<< [400] Bad request 400: Missing energy and power values [20180930,20:25,-1,-1,-1.0,-5,-1000.0,246.7,-10,-5.376,-13.42,63.773,4.126]
2018-09-30 20:24:05,149 INFO [Thread-1] (Controller.java:1960) - <<< [400] 20180930,20:25,-1,-1,0

If I am reading the Output log file correctly field 5 is Power (varies) and 6 is energy (always -1.000), although the log entry above states it is for 20:25 BST it appears to be using the reading from 20:23 BST:

20180930;20:23:43;-1000.0;246.7;-5;-1.000;-1.000;-10.000;-5.376;NaN;-13.420;63.773;4.126

Am I right in thinking that it is complaining about the Power field showing a negative value (as if I were exporting to the grid)? It seems the Powerwall sensors are a little sensitive as there are a small number of occasions in the output log file when the power value goes negative. It probably doesn’t help that at times there is very little load from the house as I have noticed on the PW app that it sometimes just shows blank icons for House, Grid, Solar & PW as if it cannot read such low W load values.

Is there a way that I can prevent this from ‘crashing’ the upload? Something like the rule that you suggested a few months back for “nswfugitive” - e.g.

rule=if (power < 0 && out) power = 0;

Also, is there a way of editing the data it is attempting to upload so that I don’t lose this information or get gaps in the graphs?

Thanks

#128

Well in the time it took me to write that reply and attempt to figure out what was going on it has skipped past the 20:25 entry and begun uploading again - I assume it only holds the last 4 or so of read history from the PW while uploading?

I have also setup the second service, changed the direction to out on powerwall.ini and commented out the V7-V12 entries, the generation upload appears to be working albeit it is registering <5W generation despite it being pitch black outside - is that another rule entry in the powerwall.ini file?

rule=if (power < 20 && out) power = 0;

1 Like
#129

Yes, the rule is placed in powerwall.ini

1 Like
#131

How to Display V7 to V12 on PVoutput.org, do I need to setup something else up, I’ve fill-in the parameters for all of the Extended Data in my Account, plus I’ve donation support?
I wish someone would write property help me files.

#132

I run 2 separate services in Windows:

IN
login-url=https://192.168.2.215/api/login/Basic
login-pass=xxxxxxxxx

url=https://192.168.2.215/api/meters/aggregates

poll=30

direction=in

voltage=instant_average_voltage

v7=battery.instant_power

v8=load.instant_power

v10=site.instant_power

v12=solar.instant_power

soc-url=https://192.168.2.215/api/system_status/soe

soc-parameter=v11

OUT
login-url=https://192.168.2.215/api/login/Basic
login-pass=xxxxxxxxx

url=https://192.168.2.215/api/meters/aggregates

poll=30

direction=out

voltage=instant_average_voltage

v7=battery.instant_power

v8=load.instant_power

v10=site.instant_power

v12=solar.instant_power

soc-url=https://192.168.2.215/api/system_status/soe

soc-parameter=v11

#133

If interested: