Tempest API understanding

[Nate]

As I work on integration, I have a number of questions about the Tempest API. Sorry for the long post.

Does /observations/device/{device_id} give me the same data I get from UDP broadcasts? It looks similar with the addition of “NC rain accumulation”. I’d like the docs to be explicit on which values are the values from my device(s) and which have been massaged.

Why are my NC rain fields null? Nearcast rain is important to me and one of the reasons I bought a Tempest. Is it because I’ve only had my device 1.5 days and they won’t be null later? Is it because there isn’t good NC data in my location and they will always be null? Why doesn’t the documentation mention which values can be null and what null means?

19 - NC Rain Accumulation (mm) is over what time period? The last minute? The docs should say (as the UDP docs do, if it’s the same data, which the docs should also say).

I need NC rain data for yesterday. What is the best way to get that? Should I use the last UDP broadcast I received yesterday? Should I use day_offset and then look at the local day NC rain value of only the last entry? Should I use /better_forecast and precip_accum_local_yesterday? Does that use NC data?

My yesterday is actually 5am to 5am, so ideally I could ask your API for that. Otherwise I would calculate an average myself, except my NC rain values are null.

Why doesn’t /observations/station/{station_id} provide NC rain data? /better_forecast doesn’t have it either. If these use NC data, the documentation should say so.

What is is_precip_local_yesterday_rain_check? I would really like full documentation, without assumptions that the API user already knows what everything is. I had to Google what “rain check” is and found this article. It still doesn’t explain what the field in question does. If true, then rain check has changed rain values? What rain values exactly?

I have to say the documentation is really lacking. There are many fields I haven’t asked about here that are unclear.

I am collecting UDP broadcast and I ignore a lot of wind speed updates. I only need observations. Is there a way to turn off wind speed updates? I don’t care about the extra UDP broadcasts, but I believe the Tempest is sending updates to the hub every 3 seconds for rapid wind speed that I don’t care about. I’ve read the Tempest might last 10 years with a couple battery changes. The battery should last longer if so many wind updates were avoided. I also don’t need lightning data.

I need a number of cumulative values for the previous 24 hours from 5am to 5am: NC rain, temp high/low, humidity high/low, wind speed average, and solar radiation average. I can collect and calculate those values from UDP broadcasts except for NC rain. I could use the web API to get NC rain data and then average it, though I might be losing some accuracy doing that. Is there a better way? I could use UDP rain data, but using nearcast data for more accuracy was the whole reason I chose the Tempest.

If I could make a suggestion it would be to provide APIs to get cumulative data about a particular time period in the past. The time a “day” starts and ends varies around the world when it comes to weather and also depends on how the data will be used (for example what time sprinklers will be turned on).

Additionally if you could calculate a reasonable evapotranspiration value it would be helpful to many. I have cobbled together code to do it, but you are the experts and I would prefer that you provide a value that is as correct and accurate as possible. I saw a reddit AmA from your people and they said you already calculate ET, so you only need to expose it. Just please write docs for it!

[iladyman]

Hi Nate,
Welcome to our forum. I understand your frustration with the documentation. And as time goes on you may notice something might change without warning which causes your previously working code to fail.
I can not answer many of your questions without doing some playing around and research so I am only replying to tell you my approach. Our 1min data is always available using the API by geting 24hr blocks. Which means I can ignore local storage and use the API for importing my data. Then I loop my data to calculate everything I want to know, which can be rain between 5am.
I realise this does not solve many of your issues.
For the one off price we paid for a Tempest I feel very fortunate with what we can do using their API.
I wish you success with your project
Cheers Ian

[Nate]

Thanks Ian. It sounds like you suggest /observations/device/{device_id} with day_offset, then calculate averages myself. That is OK, except I don’t get nearcast data. For me, fields 19 and 20 are null. How do I get NC rain data?

@iladyman

as time goes on you may notice something might change without warning which causes your previously working code to fail.

What? Wow, that should never be the case. A published API must never have breaking changes. They should publish breaking changes as a different version of the API, eg v2/observations/device/{device_id}.

If they aren’t going to version their web API, I would prefer to use UDP broadcasts, which should not have breaking changes. Do they force updates to the Tempest or hub firmware?

@iladyman

For the one off price we paid for a Tempest I feel very fortunate with what we can do using their API.

We are doing Weather Flow a big favor by buying and installing their sensors and providing them the data. They resell that data and likely make more money on that than on selling sensors. Still, it’s a nice setup that is beneficial to both parties. I just wish they would take their API more seriously.

[peter]

@Nate

How do I get NC rain data?

Where are you based? NC rain is only available in the continental US

@Nate

What? Wow, that should never be the case. A published API must never have breaking changes.

The documented parts of the API will not have breaking changes. What you have noticed though is that there are lots of undocumented parts. The reason is that WF use the same API as us users do to power their own apps. For that they use the undocumented fields and are at liberty to change them at will. You can also use the undocumented parts, but have to accept that things might change without warning.

[Nate]

@peter

Where are you based? NC rain is only available in the continental US

Ahhhh. I want NC data in Hawaii and I’m currently testing it in Puerto Rico. I guess that explains it. Is there any estimate for when Hawaii might have NC data?

I’m nervous about the accuracy of my Tempest’s rain data. Is there a recommendation on what values to use outside the continental US?

When I get current_conditions from /better_forecast, what is precip_accum_local_yesterday? Is that using rain data from a larger area, official weather stations, etc? Should I use that instead of my Tempest rain data? Maybe average the two?

My use case is knowing how much rain fell so I can decide how long to turn on sprinklers.

@peter

The documented parts of the API will not have breaking changes.

That could be OK except that none of the API documentation specifies which is “documented”. I can’t know which values I should stick to if I want to survive breaking changes.

So far I have mostly only asked questions about values listed in the documentation. Large swathes of return values are not documented at all. For example, the entire return value of /observations/station/{station_id} and most of /better_forecast.

Even values that are mentioned in the documentation are documented poorly. For example:

21 - Precipitation Aanalysis Type (0 = none, 1 = Rain Check with user display on, 2 = Rain Check with user display off)

Besides the typo, the text for 1 and 2 doesn’t explain anything. This isn’t enough to even guess what they mean. This and the other issues I’ve mentioned (NC data might be null and why, etc) are frustrating and leads me to waste your time asking for support. I still don’t have all the answers to my questions.

@peter

The reason is that WF use the same API as us users do to power their own apps.

I did get the impression the API exists for their apps and then was exposed publicly as an afterthought. IMHO it’s fine to share the API that way, but it should be treated primarily as any other public API, with complete documentation and versioning.

I hate to bring so much negativity, but I think it is important to point out how badly this is documented and that it is important. It would probably take a developer less than an hour to make it super nice. Versioning changes would be more effort, but not by much. The level of quality for the API should match the rest of what Weather Flow is doing.

[gizmoev]

@Nate

Even values that are mentioned in the documentation are documented poorly. For example:

21 - Precipitation Aanalysis Type (0 = none, 1 = Rain Check with user display on, 2 = Rain Check with user display off)

Besides the typo, the text for 1 and 2 doesn’t explain anything. This isn’t enough to even guess what they mean.

Your frustration on this particular item probably has more to do with your lack of understanding of the app than poor documentation. The item in question has to do with the NC Rain setting shown here:

And displayed as a rain drop with a check mark as shown below:

If you grabbed the data from this station the parameter returned would be 1.

For my field test station right now, the parameter would be 2. This is what its card looks like.

FWIW, the respective Station ID is at the end of the sensor name.

Not sure if HI has NC Rain. A spot check of a handful of stations didn’t show the NC Rain icon.

[peter]

@Nate

Is there any estimate for when Hawaii might have NC data?

Good question, but I’m afraid I don’t have an answer. I’m in the UK and we’ve never had NC since it was launched.

@Nate

what is precip_accum_local_yesterday?

That is the rain from yesterday (midnight to midnight in station time zone) with no NC corrections. You will probably find a lot of the answers to your original question here: Where can I find out what these "obs" values mean?.

@Nate

That could be OK except that none of the API documentation specifies which is “documented”

I suppose the point here is that if the documentation makes no reference to a particular field/value then it is undocumented. I’m sure you have seen this, but the documented parts of the API are described here: Tempest API. Since joining WF way back when it was originally launched, I can think of only one change that was made to the documented API: the addition of the NC fields to the obs array for the observation end point

[vinceskahan]

Long answer follows, but it might help if you tell folks what your ‘integration’ goals are, as it is likely folks have been there before…

@Nate

Does /observations/device/{device_id} give me the same data I get from UDP broadcasts?

Yes, plus it reports things not available in the UDP broadcasts (realtime calculated lightning and next-day NC if enabled). You might see minor timing differences if you compare websockets and UDP.

@Nate

none of the API documentation specifies which is “documented”.

I would think if it’s in the API online then it is documented. When in doubt look at the UDP API which does say which things in the broadcasts are internal use only. The websockets interface has all that stuff too.

@Nate

Why doesn’t the documentation mention which values can be null and what null means?

C’mon man. You’re a developer. If you see null it means no data. Not a reading of numerical zero. No data for that element in that time period or observation.

@Nate

I did get the impression the API exists for their apps and then was exposed publicly as an afterthought.

Disagree. They always mentioned a key part of the product was multiple public API for folks to use. It’s the reason many of us shopped WF to begin with dating back to the pre-release kickstarter days.

@Nate

I think it is important to point out how badly this is documented and that it is important. It would probably take a developer less than an hour to make it super nice. Versioning changes would be more effort, but not by much.

Which ‘this’ specifically ?

They’ve always versioned the UDP API (for example) and many of us beat on them enough to eventually have them release at least a minimal changelog for new versions so we didn’t have to figure out the differences ourselves.

I don’t recall the websockets interface changing other than adding the Tempest support several years ago, but I guess ‘yes’ a version id for that API too would have been good too.

@Nate

I could use UDP rain data, but using nearcast data for more accuracy was the whole reason I chose the Tempest.

Many of us think NC makes it less accurate, but some folks like it. Regardless you need to understand how NC works. They don’t even post-process/tweak/hack/manufacture your NC rain til the next day. Many folks just turn it off.

The old term ‘rain_check’ is now called NC rain. They changed the terminology years ago but didn’t change the API at that time, probably to not break existing code (my guess there).

Items related to rain_check (later called NC) and lightning strikes being crowd-sourced/calculated data is the only complicating factor in comparing UDP and websockets data, as those elements use WF’s proprietary magic math. Many of us have asked over the years for a way to turn it on/off for both as well as to also know how ‘much’ it tweaked the raw sensor data. That seems to have gotten a pocket veto ‘nope’ to previous requests.

@Nate

If they aren’t going to version their web API, I would prefer to use UDP broadcasts, which should not have breaking changes. Do they force updates to the Tempest or hub firmware?

What makes you think a UDP firmware change won’t ever break things ? As a developer you should deal with unexpected input. And yes. They push firmware changes quietly whether you like it or not.

@Nate

I am collecting UDP broadcast and I ignore a lot of wind speed updates. I only need observations. Is there a way to turn off wind speed updates?

Turn off rapid_wind in the app, then it’s less frequent.

[Nate]

@gizmoev

Your frustration on this particular item probably has more to do with your lack of understanding of the app than poor documentation.

If the API had proper documentation then I wouldn’t need to ask about the data it provides.

@gizmoev

The item in question has to do with the NC Rain setting

OK, so Precipitation analysis type tells the state of the Show NC rain setting on the mobile app. The app has 2 states (on/off) while the setting has 3, so that’s odd. I can’t know if the setting affects anything, such as the other data returned, or if it’s solely for the app.

Apparently it is called “rain check” in some places, “NC” in others, and “_final” in others. Inconsistencies make documentation even more important.

Thanks for the screenshot. I thought I had clicked through all the UI in the app, but missed getting to that screen. I got it now and disabled lightning, nice. I’d also like to disable wind, or at least frequent wind updates.

@peter Do you use rain data from the Tempest directly? I’ve read (eg from WF) that it can be inaccurate and NC is preferred. With that unavailable, I’m trying to figure out the next most accurate rain data. I could use the WF forecast or other web services, or I could get a bucket style rain sensor.

@vinceskahan Interesting that people want NC off. I could see both being useful.

@peter

I suppose the point here is that if the documentation makes no reference to a particular field/value then it is undocumented.

Yep, I’ve been using that page. By your definition /observations/station/{station_id} and most of /better_forecast are undocumented.

@vinceskahan

C’mon man. You’re a developer. If you see null it means no data. Not a reading of numerical zero. No data for that element in that time period or observation.

Returning null is a special case. It must be documented which values can be null, and what that special value means when it appears. Software that uses the API needs to handle these special values. Without documentation, someone who does get NC data will never know it could be null and their integration will fail when a surprise null appears. Who knows what other fields can be null? No one does, because there isn’t any documentation!

In this case we have guessed that it means NC data is not available for my location. It could also mean NC hasn’t been run yet, or not enough rain data has been collected by my new Tempest, or the WF servers are down, or anything else. I shouldn’t have to guess. I also shouldn’t have to write code to handle temperature or some other field being null for some godawful reason, unless that can actually happen.

@vinceskahan

They always mentioned a key part of the product was multiple public API for folks to use.

If it’s a key part of the product then why not, you know, spend 30 minutes documenting it.

@vinceskahan

Which ‘this’ specifically ?

The REST Reference. I’ll look at the other API docs more closely, as you suggested, and try to guess at what might transfer.

@vinceskahan

Turn off rapid_wind in the app, then it’s less frequent.

That sounds great! Where can I find that setting?

@vinceskahan

And yes. They push firmware changes quietly whether you like it or not.

That’s a bummer. Since I can’t get NC data, there’s not much incentive to send my data back to WF, just app access. I could block the hub’s WAN access.

[Nate]

@vinceskahan

it might help if you tell folks what your ‘integration’ goals are, as it is likely folks have been there before…

While I hope it’s useful for WF to hear the pains of a developer seeing their API for the first time, I’m happy to explain what I want:

I want to measure 1) rain, and 2) evapotranspiration (ET). When rain - ET < 0 I will run sprinklers in my yard to make up the lost water.

For ET, I need these values from a 24 hour period: temp min/max, humidity min/max, average wind speed (at 2 meters), average solar radiation, albedo, elevation, and latitude. Getting these values is pretty straightforward. Calculating ET is tricky, but I’ve got the math done.

For rain, I’m not sure. Maybe you guys can share some advice? I want a measurement as accurate as possible for my actual yard. Is the Tempest good enough? The solid state sensor is cool, but would a rain bucket ticker be more accurate? Hawaii tends to have very localized weather so maybe NC wouldn’t be great for me, even if it were available.

[vinceskahan]

I always turned rain check (back then before they used NC as the term) off as for me it was just bogus. It was a day late and never lined up anyway with reality as measured on reference gear I was using (a CoCoRAHS gauge and a Davis VP2 station).

@Nate

I could block the hub’s WAN access.

Then it won’t send anything to WF and nothing will be available via APIs querying their servers, and you won’t get any background tuning of your gear or firmware updates (which they do), your clock will likely start to slew, and it is uncertain if the Hub will even send UDP if it doesn’t think it can talk to the WF servers.

My memory is hazy re: what the behavior is, but I ‘think’ it buffers UDP too with some amount of storage awaiting internet connectivity to return. You’d want to test to make sure experimentally what to expect there. Short answer is just leave Internet connectivity on if you can so you get the most predictable results.

There are dozens and dozens of discussions here about how the gear acts, but the forum software is kinda rough to search.

I always found the UDP API pretty good re: understanding what was available, and while I didn’t use it, the websockets API looked ok to me. There are some oddities in their data structures occasionally but at this point they are what they are so I just dealt with it in my UDP listener/simulator tool.

Anyway - when in doubt use the UDP API docs to figure stuff out. They’re pretty good.

[background disclaimer/rant mode on]

FWIW, I long ago sold all my WF gear but I do still support a family member in PA that I gifted my old air+sky original system to and that ancient gear still runs fine on batteries.

Why did I sell ? It just wasn’t the right gear for my needs. I’m a bit of an old school ‘it should just be accurate out of the box with no Internet needed’ guy and the WF architecture doesn’t fit me.

Some data points re: what I found…

• In lots of testing here the temp+humidity were provably good plus/minus the Tempest case heating up slowly and cooling slowly and affecting readings in bright sunlight. WF has algorithms to tweak the temp value in realtime based on other readings such as wend, but it struck me as a hack trying to fix a bad hardware case with software based on other readings (wind) that I didn’t know was good or not. They lined up at night vs. three different types of similar sensors so I believe the sensor itself is ok.

• rain was just always awfully wrong and lightning was to me ridiculous by any definition since it was crowd-sourced. Rain Check (NC) to me was just a total after the fact hack.

• I also could never believe the light sensor either as it always reported impossible readings.

• the above testing was with as many as three Sky, one Air, and two Tempest at the same time (I’m a recovered Field Tester from back then) versus hand-built arduino rigs, Davis, and Ecowitt stations.

That said, WF has always been very nice people with nice stable APIs in my opinion, and they have ‘thousands’ of happy customers so they’re hitting a market niche for sure. Still hoping someday they come out with a next-gen LAN-only gear that my fossil retired instrumentation engineer brain likes from an architecture standpoint so I can try a future product someday. For now I just run weewx vs. my trusty old Davis VP2 from 2009.

[vinceskahan]

Forgot to add - if you’re going to work that hard, you might just run weewx and the excellent UDP driver from @vreihen that just plain works. There are extensions for calculating ET and the like and basically you wouldn’t need to write any code, you’d just integrate things others have already written. There are also integrations for MQTT and Home Assistant and the like too. Works great.

For rain, pick yourself up a $35 CoCoRAHS manual gauge (link)and see for yourself. Maybe with your kind of rain the Tempest is accurate enough. For me near Seattle it wasn’t any good with the gray months of gloom and misty stuff we get here, but in any kind of reasonable on/off type rain with nice big drops maybe it will be ok for your needs.

Also check out https://www.wxforum.net for a more general site with lots of smart people and lessons learned about this kind of stuff. Lots of discussion about pros/cons of various gear and setups worth surfing through.

[Nate]

Thanks for all the info @vinceskahan! I’ll definitely dig in to those links.

In general I agree shoring up data isn’t great. I’m sure there are tradeoffs. I like the sold state sensors, maintenance is low. If the adjustments are still good enough for my usage, great. That remains to be seen. I have some secondary sensors for temp and humidity.

I grew up in Seattle, so I know what you mean. It was surprising the first time I was elsewhere and the rain only lasted a few minutes. In Seattle if it starts raining, that’s life for the next few days, sometimes weeks. Miserable place, 0 out of 10 stars, cannot recommend. The city seems to be on fire these days but hiking is great, I’ll give it that (more to WA than the city).

Hawaii has it all, from a literal rain forest to desolate desert. In my location rain is very localized. Often it’s over in 15 minutes. I’ve seen it rain for 30 seconds.

[vinceskahan]

I dunno - I have Mt Rainier looming out the front window and I can hear the fog horns in Commencement Bay at night (I’m in Federal Way). I can go to Rainier, St. Helens, and hike at Snoqualmie Falls in the same day. That ain’t bad. An amazing 50 degF this morning here. Might be the year without a summer yet again. Air conditioning is for weenies

(originally from a few miles from Valley Forge and had a Revolutionary War battlefield across the street from the house, so lots of scenery here and lots of history there).

[iladyman]

Hi Nate,

@Nate

It sounds like you suggest /observations/device/{device_id} with day_offset, then calculate averages myself. That is OK

No I do not use station. I always use device. Reason being that I have many devices and the station (your hub) returns whichever device has been set by Weatherflow. I have an old model Tempest (called a ‘Sky’) measuring rainfall on a short pole in my garden where it does not get any false rain from vibrations. Because my high pole is in an extreme vibration location with a wind sock on the same pole. And I have several Tempests or Skys measuring wind in several different locations connected to a single station.

My call to the API for a 24 hour period between 5am a couple of days ago when I had rain is this:
curl -X GET --header ‘Accept: application/json’ 'https://swd.weatherflow.com/swd/rest/observations/device/12506?time_start=1685041140&time_end=1685127540&token= (insert token here) ’ > /var/tmp/wfdevice12506.json

Find your Tempest deviceID in settings, station, status.

I then loop through and calculate air density and some other personally interesting variables including adding the rain fall.

Due to one of my Tempests having a failed barometer ignore the flat brown line and the Baro reading in the table on the right.

There are two temperature lines because I am measuring temperature in my Stevenson screen and also on a high pole above my roof. Which is why some fields (dew point) are duplicated.

I suggest purchasing a manual funnel rain gauge and installing it beside your Tempest to be able to accurately know if your Tempest requires calibrating. I find different locations around my garden vary by as much as 20% in my funnel gauges and watching the rain during the windy storms I can see there are rain shadow areas and also from rain bouncing and blowing off the roof there are areas with extra rain. And then you are able to send in the readings from the funnel to have the Tempest rain calibrated to your installation. A Tempest on a different type of pole can require different calibration. To calibrate it you send your funnel gauge readings into a form. (I will send you a link when I edit this message from another computer unless some one beats me to it)

edit: Rain Report Accumulation Data

cheers Ian

[Nate]

@vinceskahan To be fair just about anywhere in WA that isn’t the city is a lot better. I’ve still had enough of the crazy and will never live in WA again.

@iladyman Neat chart! Thanks for sharing your setup.

I found this page for registering a colocated rain gauge, so I’ll set that up. I do have a colocated tipping bucket gauge. Maybe I get a funnel gauge too.

I’ve only had one good rain since getting the Tempest. The bucket shows 23.1mm while the Tempest shows 29.4mm. One other day the bucket shows 0.25mm and the Tempest 2.3mm. Maybe small amounts have larger inaccuracy.

[tempest20]

No, the documentation is terrible. If you don’t agree, then you must not have a lot of experience with other APIs that are well documented.

[Nate]

Agreed 100%. The silly thing is the API is small and could be documented extensively in an afternoon. That WF doesn’t bother is a bummer and takes a little bit of the shine off of the other things they are doing so well.