Tesla Powerwall2 data to PVoutput?


#103

Hi All, new here but been trying to get PVOutput to work from my PW2 for a few weeks and just finished another attempt to get it working. Got the editing done correctly and in the right directories this time, I think. In a folder under Program Files (x86) …… PVOutput

When I run Start-Service, I get a couple of error messages, and I am way out of my depth with what they mean.
In the cmd shell, first error is along the lines of “access denied” and the second one “need to run as an elevated service”. “Hit any key to continue”, and it just exits.

All and any help appreciated.


#104

The service install needs to be run as an administrator -


#105

OK, thanks for that, got the service to run/start without an error message but after more than
24 hours still no data showing up. Both test calls to the PW produce the expected results.

Must be another error somewhere now as well, I thought I had done the editing correctly and put the files in the right directories etc.

Just had a thought, do I have to create a seed file in c:|logs ? I was not quite sure on that bit.


#106

The file pvoutput.log under the logs directory will show any errors / activity.


#107

I thought so, the directory remains empty.


#108

If you’re unfamiliar with windows services it is best to install using the setup instead.

This contains a wizard that will install the service and start it.


#109

Ran that, but still no go.

When I look at Services, PVOutput service shows as stopped, right click and select “start” it starts and shuts down in a few seconds.


#110

The log file service.log inside the logs folder will tell you why the service failed to start.


#111

Unfortunately, I can repeat the start/stop routine and no files are found in c:|logs, the folder is empty.

I guess I am just finding that in my mid 60s having been a technical person all my life, something that appears to be as good as this system can be is eluding me. I am just not a computer scripting person, hardware etc. is no issue, this bit of it is just beyond me it seems. Thanks for your help.


#112

Its actually logs under the PVOutput installation location i.e. c:\pvoutput_install\logs rather than c:\logs


#113

OK, thanks. Found the logs now.
Here are the extracts for the latest attempt to start the service.

From the pvoutput log file.

2018-08-30 20:58:19,937 INFO [WrapperListener_start_runner] (Controller.java:109) - *** Starting PVOutput Integration Service v1.5.3.1
2018-08-30 20:58:19,937 INFO [Thread-1] (Controller.java:130) - Install Directory: C:\Program Files (x86)\PVOutput
2018-08-30 20:58:19,937 ERROR [Thread-1] (Controller.java:714) - Startup Error: Missing configuration file: C:\Program Files (x86)\PVOutput\bin…\conf\pvoutput.ini
2018-08-30 20:58:19,938 INFO [Thread-1] (Controller.java:2214) - *** Stopped PVOutput Integration Service v1.5.3.1
2018-08-30 20:58:20,938 INFO [Wrapper-Shutdown-Hook] (Controller.java:2214) - *** Stopped PVOutput Integration Service v1.5.3.1

and from the service log

INFO | jvm 1 | 2018/08/30 20:58:19 | *** Starting PVOutput Integration Service v1.5.3.1
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.runtime.name=Java™ SE Runtime Environment
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.version=25.172-b11
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.vendor=Oracle Corporation
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vendor.url=http://java.oracle.com/
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.name=Java HotSpot™ Client VM
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.specification.name=Java Virtual Machine Specification
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.runtime.version=1.8.0_172-b11
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.awt.graphicsenv=sun.awt.Win32GraphicsEnvironment
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.endorsed.dirs=C:\Program Files (x86)\Java\jre1.8.0_172\lib\endorsed
INFO | jvm 1 | 2018/08/30 20:58:19 | >> os.arch=x86
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.io.tmpdir=C:\WINDOWS\system32\config\systemprofile\AppData\Local\Temp
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.specification.vendor=Oracle Corporation
INFO | jvm 1 | 2018/08/30 20:58:19 | >> os.name=Windows 10
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.library.path=…/lib
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.specification.name=Java Platform API Specification
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.class.version=52.0
INFO | jvm 1 | 2018/08/30 20:58:19 | >> os.version=10.0
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.awt.printerjob=sun.awt.windows.WPrinterJob
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.specification.version=1.8
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.class.path=…/lib/wrapper.jar;…/lib/org.pvoutput.integration.jar;…/lib/commons-logging-1.1.1.jar;…/lib/httpcore-4.1.2.jar;…/lib/httpclient-4.1.2.jar;…/lib/jetty-http-7.5.1.v20110908.jar;…/lib/jetty-util-7.5.1.v20110908.jar;…/lib/jetty-io-7.5.1.v20110908.jar;…/lib/jetty-server-7.5.1.v20110908.jar;…/lib/jetty-continuation-7.5.1.v20110908.jar;…/lib/servlet-api-2.5.jar;…/lib/json_simple-1.1.jar;…/lib/jxl.jar;…/lib/log4j-1.2.15.jar;…/lib/RXTXcomm.jar;…/lib/jackcess-1.2.2.jar;…/lib/bsh-core-2.0b4.jar;…/lib/sqlitejdbc-v056.jar;…/lib/commons-net-3.1.jar
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.specification.version=1.8
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.home=C:\Program Files (x86)\Java\jre1.8.0_172
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.specification.vendor=Oracle Corporation
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vm.info=mixed mode, sharing
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.version=1.8.0_172
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.ext.dirs=C:\Program Files (x86)\Java\jre1.8.0_172\lib\ext;C:\WINDOWS\Sun\Java\lib\ext
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vendor=Oracle Corporation
INFO | jvm 1 | 2018/08/30 20:58:19 | >> java.vendor.url.bug=http://bugreport.sun.com/bugreport/
INFO | jvm 1 | 2018/08/30 20:58:20 | *** Stopping PVOutput Integration Service v1.5.3.1 (0)
STATUS | wrapper | 2018/08/30 20:58:22 | <-- Wrapper Stopped

From what I can see, on my installation the directories where the wrapper is looking for the .ini file does not exist.

Garry


#114

The file C:\Program Files (x86)\PVOutput\conf\pvoutput.ini could not be found.

This should not be the case if it was extracted properly. You would have had to edited pvoutput.ini to enter your API key and system Id.

The contents of C:\Program Files (x86)\PVOutput\ should look like the following.

image


#115

OK, well yes that is exactly how it looks, the edited files are in their respective sub-directories but the service will still not keep running and gives the same error messages each time it starts and stops.

ps. I was mis-interpreting the

C:\Program Files (x86)\PVOutput\bin…\conf\pvoutput.ini

…\conf\pvoutput.ini to mean it was a subdirectory of \bin

so yes, my directory structure is exactly as you show above.


#116

Please continue this in the thread below -

Since this problem is unrelated to Tesla Powerwall.


#117

Hi. This is my first day reporting live data from my PowerWall. I have some questions. First, this is what I did:

I took my existing graphing database that I use to create my own graphs at http://ulmo.solar/ and set them up to output to the extended (annual donation only) PVOutput attributes.

I did the update back 90 days, then set it to live update every 5 minutes or so. Plus, daily it goes back and re-uploads any stragler data my sweep up cleaners grab. For now, I’ll have to manually trigger adding missing data that has been found that is longer than around a day back.

You can look at my graphs at https://pvoutput.org/list.jsp?userid=60344

Here’s some questions:

  1. When I am not logged in, I cannot see my battery graphing from the extended data regardless of everything I have clicked on; Live does not show battery data. I set extended data to Public, but it doesn’t seem public.

  2. When I am logged in, I ONLY see my battery graphing data for live, and cannot see the regular solar data for live.

My question for #1 and #2 is how to make both solar and battery extended data both available to the public and to myself regardless of whether or not I am logged in.

  1. When I add some of the figures calculated by PVOutput.Org for my system, there are some descrepancies. I believe this is due to the PVOutput.Org not exactly defining what all the fields mean and what data they need, so when a device such as a battery is introduced, we don’t know what data to submit. I already noticed that what I sent it was a little off, so I re-sent the 90 days of data with a better definition, but it still seems wrong.

At first, I thought Consumption is what I use. Unfortunately, that gave really wacky results on PVOUTPUT, so I don’t think Consumption actually means consumption, but means consumption of solar. That needs to be clarified. Anyway, so I added my actual consumption to the storage that my batteries store, and output THAT addition as “Consumption” of what is taken by the solar panels, even though it doesn’t include what I export to the grid (from anywhere) and it also includes what the batteries take from the grid. In my case, the settings on my system are such that charging from grid or discharging to grid is not yet commonplace, but will be in a few years. Obviously, PVOUTPUT needs to have their calculation system totally revamped to understand each item in the electrical arrays.

Here are the discrepancies I noticed. I’ll use September 8, 2018 for an example.

Date: 2018-09-08
Generated: PVOUTPUT: 30.556kWh; MY RECORDS: 30,557.1658333503Wh: AGREEMENT.
Efficiency: PVOUTPUT: 5.197kWh/kW: cool. I don’t know what that means, but I’m guessing it’s energy per insolation. I’ll have to play with that some time.
Exported: PVOUTPUT: 1.761kWh; MY RECORDS: Exported to Grid: 1,941.64833457023Wh; Imported from Grid: 255.303056792356Wh; Net Exported to Grid: 1,686Wh. I can’t tell if these are kind of close to what PVOUTPUT got or not. I am suspicious that this is wrong. I don’t know what PVOUTPUT means by Exported, or how it gets it.
Consumption: PVOUTPUT: 44.664kWh; MY RECORDS: House use + Battery use: Home 28,037.1697222255Wh + Battery Charging 16,740Wh = 44,777Wh. Since I didn’t have much grid<->battery action, these numbers essentially look correct, but when I start charging from grid and discharging to grid, this will rapidly need updating.
imported: PVOUTPUT: 15.868kWh; MY RECORDS: So, what does Imported mean? I’ll assume it means Imported from Grid to satisfy Home use. That also ignores grid <-> battery; that could add some errors. Here’s my Grid importation: 255.303056792356Wh. That’s right, PVTOUTPUT shows importation of 15.868kWh, and my records show importation of only 0.255kWh, over 62 times as much at PVOUTPUT in meaningful amounts; their data is obviously not what I think this is.

What does their “Importation” mean? Where is it coming from?

I want to understand what the PVOUTPUT “Generated”, “Exported”, “Consumption”, and “Imported” columns mean, how I upload to them, what I upload to them, and how they related to battery use.

Once I know that, I can hack around any bugs, ignore the wrong things that can’t be fixed, and use all the extended attributes as best I can to map that.

On to more bugs:

I hate dates that are a mishmash of numbers like 39/29/10/29/39/43 that mean nothing to me, so I do all dates Big Endian like this: 2018-09-08. But, when I select that date format in the Settings, the graphs all show “2018-” for every date. How do I select a sensible date format that doesn’t just get cut off in the graphs? For now, I have one of the confusing a/b/c/d/e/f types of date fields where everything is jumbled, backwards, and spagetti twisted like some sort of mystical secret coding. It is annoying as hell.

Weather Underground doesn’t allow registering of new data streams. I would like to input data from my closest weather station at KCAAPTOS60; should I scrape that data myself using my own web scraper software and upload those temperatures myself? I suppose that’s the easiest solution.

There is alternate forecast data including cloudiness, insolation, temperature, expected PV output, etc. that I get from https://solcast.com.au/ and I would like to integrate that for PVOUTPUT display; is there an effort underway to do such a thing? Since I already have a profile at SolCast, I think the best way would be a site-wide PVOUTPUT link to SolCast, probably with a special deal between the two for a proper fee for SolCast, to use their API to input the data into PVOUTPUT. Since SolCast has more accurate panel array pointing and stuff like that, I’d like PVOUTPUT not to skimp on integration of the panel parameters, and if anything, to add more parameters to the PVOUTPUT data, or simply use the settings the user has set at SolCast. Then PVOUTPUT could efficiently download and display the data from SolCast. I have difficulty displaying all the data on my website (see for instance http://ulmo.solar/powerflow/center.1.png ), but I managed to figure out that gray can be clouds (yay!), purple is my temperature forecast, and for now I have trouble setting up the right color for me to use for the PV output forecast but went with a march of the number “0” for now. It’s ugly, but it’s there. It’s usually wrong until the last half hour, too, but I show their historical forecasts so we can see how it improves over time. The Australian SolCast doesn’t seem to do much in the way of predicting our microclimate hilly beachside fog very well, almost always getting it way wrong in both directions. But I like to see the PVOUTPUT forecasts as an indication of what is possible from the position of the sun.

A feature missing from SolCast is an ability to give a 3D rendering of the foliage, buildings, and other obstructions around my solar panels, so that they can figure out the shading, as well as how the strings respond to the shading given my particular setup. It would not be too hard given some data to figure that out, but for now, they don’t seem enamored of that quality of output, either. This ends up leaving a few kWh extra in their forecasts every day that I will never get (and don’t want considering I love those trees more).

Any assistance provided would be appreciated!

Solving every problem above seems like a full time job, so I’m not asking anybody to do anything, but any improvement or ideas for improvement you get from my novice questions would of course be great.


#118

Update:

I found the “Views” bar with the color squares. Here’s where I read about it in help:

==================
Views
Below the graph is the View Navigator, clicking on one of the squares will switch to that view. There are five views available -

■ ■ ■ ■ ■   All - This view shows generation, consumption, temperature and voltage.
■ ■ ■ ■ ■   Generation - This view shows generation data only.
■ ■ ■ ■ ■   Consumption - This view shows consumption data only.
■ ■ ■ ■ ■   Net Data - This view shows Net Energy and Power.
■ ■ ■ ■ ■   Extended Data - This view is available when the system has Extended Data.

==================

Once I found that, I was able to see some error message for Extended Data, and then see what the other squares displayed, and realized some were summaries; this allowed me to realize that the “Summary” column for my Extended data was not set up (it was all empty), so when I chose values for that (I chose average for everything), it started putting data into Extended Data for Summary.

I also am able to see the non-extended data when I look at Live now.

What I still need to find out, though, is what all the columns mean, so that PVOutput can properly account for my battteries.


#119

Followup 2:

I can now clearly see that Imported in the main view is wrong. It should not be so high. If you look at my “Extended” view, you should readily see that I do not use any grid on many sunny days when I do not use electric heaters, especially more so in summer, so the high “Imported” is wrong.

I am not sending that Imported to PVOutput; where is it coming from? How is it calculated?


#120

Followup 3:

I found another “Net Data” output section; do I really have to make two reports every time I submit data? That’s twice as many API calls. Botheration! I guess I’ll have to figure out how to make that happen. I’ll have to see if it even works.


#121

Followup 4: as I’m loading the new utility grid import & export data to see if that fixes it, I will also note that I went ahead and programmed my own scraper to grab the temperature as well. This is coming together one piece at a time. I look forward to seeing if the import export data works correctly.


#122

Followup 5: Things are going horribly. PVOutput uses inconsistent terms everywhere:

For one single example, of which there are many: In one spot, there is “Energy Import”. In another spot, the same number being accumulated is “Energy Used”. WHAT THE HELL ARE THEY TALKING ABOUT? IMPORT FROM WHERE? USED WHERE?!

UGGGH!! GRR!!! I can’t make heads or tails of this.

First of all, the fundamentals of these electrical networks is that each object has some number of wires going between it and some number of other items. The wires have properties, the objects have properties, and energy flows between the objects over those wires according to those properties. That’s it. That’s the fundamental basis of this whole thing. But, PVOUTPUT seems to want to make a rigid, incorrect, imprecise, hidden, and convoluted specification of the objects and wires in the network without explaining any of their choices.

If they could treat each object and wire appropriately, then it would be very simple: an object of type solar panel with these attributes, and the wires to and from it of these parameters. Object of type inverter, object of type battery, object of type utility grid, object of type car charger, object of type house. Templates could present simple networks like grid,inverter,PV,house, or allow more complicated networks as needed. Here, the abstraction is everything is an object or wires, each with properties, each linked to other objects or wires. You could even abstract wires into objects, so that wires are objects, then every object would just be a object with properties and connections, and make it even simpler.

But if they try to abstract it in the other direction to everything being a photovoltaic panel, then call the outputs all of these weird spagetti names, then the result is a programming mess with lots of different people interpreting a lot of different things in different ways, and nothing works. That’s what I’m seeing now.