Wrong device history shape for obs_sky

Developers
1

Hi,

From the API documentation I saw that for this API endpoint GET /observations/device/{device_id}

type obs_sky should look like:

Sky (type="obs_sky")
Observation Layout
0 - Epoch (seconds UTC)
1 - Illuminance (lux)
2 - UV (index)
3 - Rain Accumulation (mm)
4 - Wind Lull (m/s)
5 - Wind Avg (m/s)
6 - Wind Gust (m/s)
7 - Wind Direction (degrees)
8 - Battery (volts)
9 - Report Interval (minutes)
10 - Solar Radiation (W/m^2)
11 - Local Day Rain Accumulation (mm)
12 - Precipitation Type (0 = none, 1 = rain, 2 = hail, 3 = rain + hail (experimental))
13 - Wind Sample Interval (seconds)
14 - Rain Accumulation Final (Rain Check) (mm)
15 - Local Day Rain Accumulation Final (Rain Check) (mm)
16 - Precipitation Analysis Type (0 = none, 1 = Rain Check with user display on, 2 = Rain Check with user display off)

But I am getting this

{
bucket_step_minutes: 5, 
device_id: 9747, 
obs: [
["1559053200","8138","0.48","0","1.6","3.8","6.4","61","3.4","5"], 
["1559053500","10280","0.62","0","2","4","10.1","61","3.4","5"], 
["1559053800","7516","0.43","0","1.6","3.6","7","59","3.4","5"], 
["1559054100","10475","0.64","0","0.7","3.4","5.6","64","3.41","5"], 
["1559054400","8052","0.48","0","0","2.9","5.3","55","3.41","5"], 
["1559054700","10888","0.66","0","0.4","2.6","5.5","75","3.41","5"], 
["1559055000","8026","0.46","0.054941","0","2.9","4.4","55","3.41","5"], 
["1559055300","11181","0.7","0.012095","0","1.8","4.1","65","3.41","5"], 
["1559055600","12381","0.77","0","0.9","2.3","5.2","32","3.41","5"], 
["1559055900","11504","0.7","0","0.6","2.2","4.3","57","3.41","5"]
], 
source: 'db', 
status: {
status_code: 0, 
status_message: 'SUCCESS'
}, 
type: 'obs_sky'
}

Could you please explain why i am getting a list of 10 elements while a list of 17 is expected?

2

I just tried it online, and my sky reports all the values. I use the “try this” options you get when clicking on the api on the top of this community page.

3

It works fine for most devices. But this issue should not happen at all. I would expect to have null values if sensors don’t report anything, but still the same shape.

4

These records are from May, 2019! After a while, WF’s storage system summarizes the obs_* observations into 5-minute averages “buckets” for archiving. Hence, “bucket_step_minutes: 5” at the beginning of the JSON.

FWIW, I do not recall seeing this packet format for “bucket” data documented, but I have never used the websocket/REST API interfaces…

5

Thanks for your reply.

The bucket size was not the problem (BTW, bucket size also adjusts depending on the start to end span of your request even for recent dates).

If the data was an object (dictionary), I would not mind having missing keys. But for an array (list), having different dimensions as the documentation is cumbersome. You cannot know which value is missing. Are those only the last items that are missing.

10 - Solar Radiation (W/m^2)
11 - Local Day Rain Accumulation (mm)
12 - Precipitation Type (0 = none, 1 = rain, 2 = hail, 3 = rain + hail (experimental))
13 - Wind Sample Interval (seconds)
14 - Rain Accumulation Final (Rain Check) (mm)
15 - Local Day Rain Accumulation Final (Rain Check) (mm)
16 - Precipitation Analysis Type (0 = none, 1 = Rain Check with user display on, 2 = Rain Check with user display off)

Could anyone from the Weatherflow staff help me figure things out?

6

Can you post the URL of the API request you are making?

7

Here it is: GET https://swd.weatherflow.com/swd/rest/observations/device/9747?time_start=1559053122&time_end=1559485122

8

OK this appears to happen when data is requested from some time ago. I don’t have a Sky running at the moment, but if I request the latest available data I have from January 2021 then the individual obs in obs array have the correct shape with 17 elements. This is true if the bucket_step is 1 minute or 5 minutes

However, if I request data from 2019 (which correspond to the timestamps you provided), then I also see a shorter obs array with only 12 elements (ending with Local daily rain accumulation).

I suspect what has happened here is that the extra fields (13 to 16) have been added more recently, however WF have not gone back through the entire archive of users past observations to add in a bunch of nulls.