# Data archive buckets explained **URL:** \ **Category:** Tempest Owners\ **Created:** [May 22, 2018, 8:07pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063 "2018-05-22T20:07:02Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![WFmarketing](https://sea2.discourse-cdn.com/flex020/user_avatar/community.tempest.earth/wfmarketing/32/62_2.png) [@WFmarketing](https://community.tempest.earth/u/WFmarketing)\ **Post date:** [May 22, 2018, 8:07pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/1 "2018-05-22T20:07:02Z")
So how does WeatherFlow store data? Explained: your AIR & SKY sensor devices are constantly collecting and sending observations to the HUB. The HUB then uses a set of sophisticated algorithms to inspect the data before publishing to the secure WeatherFlow servers. HUB also makes the data available locally via UDP and BLE broadcast for users who wish to collect and store the data at home. The secure WeatherFlow servers not only store all the 1min data, but the data is also processed and stored as 5min | 30min | 3hour | 1day . Each temporal resolution is it’s own data set, ready to be queried quickly by the app for your display. Here’s a more detailed summary of stored data by sensor device and parameter: ![AIR_dataarchive](https://cdck-file-uploads-us1.s3.dualstack.us-west-2.amazonaws.com/flex020/uploads/sws/original/2X/6/69706ace817876741969a1fd1491900f41b815f5.png) ![SKY_dataarchive](https://cdck-file-uploads-us1.s3.dualstack.us-west-2.amazonaws.com/flex020/uploads/sws/original/2X/7/7777f48d0e06de0d5fc2c94e8bcf51a1dfd00301.png) ---
**Author:** ![WFmarketing](https://sea2.discourse-cdn.com/flex020/user_avatar/community.tempest.earth/wfmarketing/32/62_2.png) [@WFmarketing](https://community.tempest.earth/u/WFmarketing)\ **Post date:** [October 24, 2018, 5:48pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/2 "2018-10-24T17:48:04Z")
---
**Author:** ![peter](https://sea2.discourse-cdn.com/flex020/user_avatar/community.tempest.earth/peter/32/1365_2.png) [@peter](https://community.tempest.earth/u/peter)\ **Post date:** [November 15, 2018, 7:37pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/3 "2018-11-15T19:37:33Z")
@dsj I know this is slightly off-topic, but regarding the data buckets is there any chance of replacing “Closest Ob” in many of the data buckets with “Avg” over the period of that data bucket instead? “Closest Ob” can give a very skewed impression of how parameters have varied throughout the day, if it happens to be anomalous compared to the rest of the day. I don’t have any particular examples to share, but say, for example, during a cloudy day there might be single break in the cloud that creates in peak in solar radiation for 5 minutes at noon. This peak is far greater than anything else observed that day, however when you zoom out to say the 3 hour bucket, it complete dominates the graph, giving the impression the day has been significantly more sunny than it actually has. Averaging the data over that 3 hour bucket would be a significantly better overview of how the radiation as evolved. ---
**Author:** ![dsj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.tempest.earth/dsj/32/3_2.png) [@dsj](https://community.tempest.earth/u/dsj)\ **Post date:** [November 15, 2018, 11:31pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/4 "2018-11-15T23:31:26Z")
I moved your post so it’s no longer off-topic! > [@Change in graphing rain rate](https://community.tempest.earth/t/change-in-graphing-rain-rate/2319/42): > > is there any chance of replacing “Closest Ob” in many of the data buckets with “Avg” over the period of that data bucket instead? That’s a really good question. Reducing data can be tricky and inevitably there is some loss of fidelity. Many parameters don’t change much over a short time period, and in that case “snapping” to the closest observation creates the “truest” line. But you’re right that this is not necessarily true for the solar parameters (UV, Lux, Radiation). Perhaps, like wind, we should display three values (high, low, average)? ---
**Author:** ![sunny](https://sea2.discourse-cdn.com/flex020/user_avatar/community.tempest.earth/sunny/32/16368_2.png) [@sunny](https://community.tempest.earth/u/sunny)\ **Post date:** [November 15, 2018, 8:30pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/5 "2018-11-15T20:30:43Z")
btw, rain rate is missing from that table with the data buckets, or isn’t it stored, but calculated from the accumulated rain? ---
**Author:** ![dsj](https://sea2.discourse-cdn.com/flex020/user_avatar/community.tempest.earth/dsj/32/3_2.png) [@dsj](https://community.tempest.earth/u/dsj)\ **Post date:** [November 15, 2018, 11:33pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/6 "2018-11-15T23:33:36Z")
Good catch - that’s an oversight. We’re missing solar radiation too. We’ll get that updated shortly! ---
**Author:** ![peter](https://sea2.discourse-cdn.com/flex020/user_avatar/community.tempest.earth/peter/32/1365_2.png) [@peter](https://community.tempest.earth/u/peter)\ **Post date:** [November 19, 2018, 9:06am UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/7 "2018-11-19T09:06:49Z")
> [@dsj](#): > > Perhaps, like wind, we should display three values (high, low, average)? Personally I would just display the average. Max/min can be left to the Z4 (1 day) level (which as an aside, I’ve always thought would look better if a line connected the daily averages). > [@dsj](#): > > Many parameters don’t change much over a short time period I think this is true for the Z1 and probably Z2 level, but at the Z3 level, I think all parameters (not just solar radiation) can change sufficiently rapidly in a non-monotonic manner between two discrete observations that the average would give a far truer impression than just the discrete observations. ---
**Author:** ![sunny](https://sea2.discourse-cdn.com/flex020/user_avatar/community.tempest.earth/sunny/32/16368_2.png) [@sunny](https://community.tempest.earth/u/sunny)\ **Post date:** [January 28, 2021, 12:25pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/8 "2021-01-28T12:25:29Z")
I just found out that weatherflow servers do store all the 60 second data. This is data I _just_ retrieved from the server. Note the 60 seconds difference in the timestamp (4th number in a row). 1551312000 is computer language for 28 Feb 2019 0:00. I’m glad they do 😃 (weatherflow didn’t say it doesn’t store 1 minute Z0 data forever, It might be that in addition to that they store the Z1, Z2, Z3 and Z4 data) I just got the impression that all the old 1 minute data would be replace by Z4 in the end. Not so! ```auto 15472,obs_sky,1,1551312000,0,0,0,1.56,2.05,2.73,41,3.42,1,0,0,,,,,, 15472,obs_sky,1,1551312060,0,0,0,0.89,1.62,2.15,334,3.42,1,0,0,,,,,, ``` ---
**Author:** ![tempest20](https://avatars.discourse-cdn.com/v4/letter/t/ac91a4/32.png) [@tempest20](https://community.tempest.earth/u/tempest20)\ **Post date:** [April 5, 2023, 3:40pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/9 "2023-04-05T15:40:21Z")
Can we get an update on this for the Tempest Weather System? ---
**Author:** ![peter](https://sea2.discourse-cdn.com/flex020/user_avatar/community.tempest.earth/peter/32/1365_2.png) [@peter](https://community.tempest.earth/u/peter)\ **Post date:** [April 5, 2023, 5:19pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/10 "2023-04-05T17:19:43Z")
Nothing has changed. Although this post refers to Sky and Air devices, which have now been combined into a single Tempest, the details for each observation remain exactly the same ---
**Author:** ![tempest20](https://avatars.discourse-cdn.com/v4/letter/t/ac91a4/32.png) [@tempest20](https://community.tempest.earth/u/tempest20)\ **Post date:** [April 5, 2023, 5:27pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/11 "2023-04-05T17:27:16Z")
Interesting, thanks. Maybe @WFmarketing should update the post body to make this more clear. ---
**Author:** ![tempest20](https://avatars.discourse-cdn.com/v4/letter/t/ac91a4/32.png) [@tempest20](https://community.tempest.earth/u/tempest20)\ **Post date:** [April 5, 2023, 5:29pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/12 "2023-04-05T17:29:10Z")
@peter Do you know if “closest observation” is still used? Or did they make changes based on your suggestions? ---
**Author:** ![peter](https://sea2.discourse-cdn.com/flex020/user_avatar/community.tempest.earth/peter/32/1365_2.png) [@peter](https://community.tempest.earth/u/peter)\ **Post date:** [April 5, 2023, 7:52pm UTC](https://community.tempest.earth/t/data-archive-buckets-explained/1063/13 "2023-04-05T19:52:12Z")
As far as I can tell, “closest observation” is still being used