OpenWeatherMap data gaps - fill in missing values

[mikemillar]

Thanks mainly to BB and willisave I finally got PVOutput recording and displaying my Powerwall 2 data. Now I’m trying to put on a sensible air temperature line on the live graph. I registered with OpenWeatherMap as basic data is free and got data at mostly 10 minute intervals in the table but nothing on the graph, except 2 data points which I realised were at 5 minute intervals. Finally got a line showing by changing Status Interval to 10 minutes but it’s broken where the data interval is more than 10 minutes - and I’d prefer a 5 minute interval anyway. Is there a solution to this? The OpenWeatherMap instructions suggest a minimum interval of 10 minutes.

Alternatively is there a better alternative free source? Wunderground appears to be a fully paid service, and I think you have to have a weather station.

[bankstownbloke]

You could try another city id close by to see if the updates are more frequent and/or reliable.

Unfortunately there isn’t an alternative free solution at the moment - we may look at revisiting the following weather station idea again soon -

PVOutput Weather Service

Wunderground will be discontinuing API access to weather data which we have used for temperature data. We are proposing to add a simple weather upload service to PVOutput that will accept 5-minute temperature data which can then be shared across all PVOutput systems. e.g. https://pvoutput.org/service/r2/addtemperature.jsp?d=20180525&t=10:00&temp=37.5&station=SYD_AIRPORT Then anyone can use SYD_AIRPORT as their weather station. Having our own temperature service ensures that we don’t have to…

[mikemillar]

Thats a good idea. Thanks, BB, I’ll give that a go. It’s not really a major problem, just a slight annoyance! I assume from what you say it’s the actual output rate from OpenWeatherMap that causes the problem. Maybe why they recommend a minimum 10 minute interval.

But couldn’t the PWO graph just join the dots between the missing data on the 5 minute status interval?

[bankstownbloke]

But couldn’t the PWO graph just join the dots between the missing data on the 5 minute status interval?

It a possible future enhancement for openweathermap, however the result will be ‘staircase’ graph where the missing data points are filled in using the last known value.

Change this topic to an idea to track.

[lwsmiser]

I have suggested showing the individual isolated dots rather than connecting them. You can scan them with the mouse and they will show up or they appear briefly when one of the channels is turned off or on.

[mikemillar]

I changed the city id to London and status interval back to 5 minutes. No joy, London data output interval was similar, mainly 10 minute but variable. But worse - large parts the last 2 day’s 10 minute extended data somehow got 5 minute intervals inserted with no data in them. Now the extended data graph won’t plot anything in these sections for the same reason that 10 minute weather data won’t plot with 5 minute status interval selected - aaagh!

I had hoped I could download the affected data, remove the null lines and use Add Output to re-insert it, but it seems this doesn’t work for extended data,

I can live with gaps in the weather data, but I’d prefer to be able to use 5 minute status interval and a continuous line ignoring missing data. Excel seems to manage it with no problem. However, you guys have done a great job providing the service so I’m not complaining.

PS - ‘change this topic to an idea to track’ - was that a suggestion for me to implement? I’m not that hot on forum techniques so how does one do that?

[bankstownbloke]

PS - ‘change this topic to an idea to track’ - was that a suggestion for me to implement? I’m not that hot on forum techniques so how does one do that?

Nothing for you to do. The topic was edited and categorised as ‘Idea’.

Ideas are often revisited to see if they can be implemented, or not.

[mikemillar]

OK, thanks BB. I feel honoured!

[lennybig]

Just wanted to keep this idea alive.

Having the same issue of random intervals of temperature being recorded, but I believe for a different reason. With an Enphase system, polling is only every 15 or 20 minutes to stay under the monthly request limit, so I believe this is why I am seeing the holes. (If someone out there has an Enphase system but the OpenWeatherMap is still showing regular 5 minute increments in extended data, let me know, it must be a different problem then.) I have tried different city codes and all show the same issue.

If that is indeed the issue, and barring any fixes on the PVOutput side to log the last known value (which would be great), anybody have any (fairly simple) ideas for a workaround?

[bankstownbloke]

ideas for a workaround?

If the concern is the gaps in the graph not looking pretty then try enabling the No Graph Gaps option under ‘Customise’ -

This will join the gaps in the graph to display a continuous line.

[lennybig]

That’s cool and does make it look pretty.

My question was in regards to comparing against another day to see how temperature affects production. It’s all over the board where there’s temperature gaps in either day.

[kemyers95963]

You could just bite the cost and get your own PWS and upload your data directly or use the API that is provided by the PWS vendor. I have currently 4 choices of weather data from APIs. Wunderground, Openweathermap, Aerisweather and Ambient Weather. All are free API’s including Wunderground because I am grandfathered in. All based on my station data. Ambient Weather is the maker of my PWS. Cost is around $100 for a full featured station like my Ambient Weather WS-1001-WiFi model.