Rest API with time_start and time_end

I am experimenting with the Rest API to extract all observations between a specified time_start and time_end. Unfortunately I’m struggling to understand the output. For example, if I submit the following request:

https://swd.weatherflow.com/swd/rest/observations/device/1110?api_key=20c70eae-e62f-4d3b-b3a4-8586e90f3ac8

I get output from the latest observation with (in this case) a UNIX timestamp equal to 1524033360 or 04/18/2018 @ 6:36am (UTC). Now, if I subtract 10800 seconds from that timestamp in order to use time_start=1524022560 (or 04/18/2018 @ 3:36am (UTC)) and time_end=1524033360 to extract all observations in the three hours preceding the latest observation using this command:

https://swd.weatherflow.com/swd/rest/observations/device/1110?time_start=1524022560&time_end=1524033360&api_key=20c70eae-e62f-4d3b-b3a4-8586e90f3ac8

Then the first observation returned has a UNIX timestamp of 1524023100 which corresponds to 04/18/2018 @ 3:45am (UTC). The question then is what has happened to the observations between 3:36am (UTC) and 3:45am (UTC), and why are they not being returned by the API request?

If for the sake of argument I change time_start to 1524020000 (04/18/2018 @ 2:53am (UTC)):

https://swd.weatherflow.com/swd/rest/observations/device/1110?time_start=1524020000&time_end=1524033360&api_key=20c70eae-e62f-4d3b-b3a4-8586e90f3ac8

Then I can find the observation at 3:36am (UTC), so it is not an issue that the observation doesn’t exist. However, again the first observation returned by the API request has a UNIX timestamp of 1524020580 or 04/18/2018 @ 3:03am (UTC), a whole 10 minutes after the specified time_start.

Any help on understand where these missing observations are would be much appreciated!
Many thanks.

2 Likes

First, 152402000 is actually 2018-04-18T02:53:20+00:00 so if you add 40 seconds to the second time example you will see the both examples are exactly 540 seconds off.

They both skip the fitst nine minutes. That’s very strange and I never paid any attention to that. David (@dsj) or Corrine (@corrineb) will have to inquire about this.

3 Likes

Yes, you’re right! I was posting quickly before going to work and hadn’t spotted that.

2 Likes

@dsj and @corrineb, any update on this?

Hey @peter,

We are still looking into the issue. Will update you soon.

2 Likes

Hi @corrineb. I appreciate everyone at WF is probably incredibly busy, and this is likely not the issue at the top of your to-do list, but would you be able to provide an update on when this bug will be fixed?

Another thing I have noticed is that if you use time_start, time_end or day_offset in your query, all the timestamps in the resultant JSON message are rounded to the minute. Others may have a different opinion, but for me I am not sure that this should be the expected behavior. The timestamp should still respect exactly when the observation was taken.

Many thanks!

1 Like

Hey @peter,

We took a look and were able to identify an issue with the API and the time_start parameter. We are incorrectly handing a start time of 3:36 AM UTC and starting to pull observations at 3:45 AM UTC. You can test by setting time_start to 4/18/18 3:30 AM UTC and you will see that you will correctly get observations back starting at 3:30 AM UTC.

https://swd.weatherflow.com/swd/rest/observations/device/1110?api_key=20c70eae-e62f-4d3b-b3a4-8586e90f3ac8&time_start=1524022200&time_end=1524033360

We’ve added this bug to our list and it will be fixed in a future API update. Thanks for letting us know!

2 Likes

Thanks for getting back @corrineb and updating me on the issue. I will try and work around the bug in the meantime, but hopefully you’ll be able to get it fixed soon :slight_smile:

1 Like

I am having a similar issue… when I query with time_start of 1528730520 (Monday, June 11, 2018 10:22:00 AM GMT-05:00 DST) and time_end of 1528730880 (Monday, June 11, 2018 10:28:00 AM GMT-05:00 DST) , I get NO results. However, if I query with a ‘day_offset’ of 0 (today), I get data…

Ideas?

–Sam

1 Like

There is definitely something wrong. In my 38 years of database programming I don’t see how that can NOT work.

1 Like

If I use a time_start of 1528700040 (Monday, June 11, 2018 1:54:00 AM GMT-05:00 DST) and that end_time… I get data, but starting at 1528700400 (Monday, June 11, 2018 2:00:00 AM GMT-05:00 DST).

–Sam

1 Like

There is a big bug in the API when it comes to specifying start and end times. It seems like the day offsets of 0 (today) and -1 (yesterday) work, but nothing else.

@corrineb mentioned earlier in the thread that the API was incorrectly starting to pull observations at 3:45 am UTC. I’m not sure if you can use this to create a workaround. I have just given up trying to use this functionality in the API for the time being, and am waiting for the WF team to issue an update to fix the bug. It means, for example, that I can’t calculate a numeric value for the 3 hour pressure trend, or the total rainfall for the year.

Unfortunately, I don’t imagine that fixing this is particularly high on their to-do list!

There should be data every 60 second. I wonder if data is held and written later.

To help WF diagnose the issue…

I choose to get values starting at:

Monday, June 11, 2018 10:16:09 AM GMT-05:00 DST,

But I get values returned starting at:

Monday, June 11, 2018 10:30:00 AM GMT-05:00 DST

Gonna change over to getting ‘todays’ data until the time frame requests are fixed.

–Sam

1 Like

Does ‘todays’ data end with the most recent observation, or does it end earlier? It would help to check how quickly the 60 second data is made available to API when using data ranges.

Doing some quick testing with the online REST API tests… it’s popping the most recent ‘minute’ observation as soon as it happens.
–Sam

1 Like

And the most recent ‘minute’ observations is also popping as soon as it happens if you use day_offset=0?

Generally, it’s looking like the ‘latest’, minutely data is not THIS minute, but LAST minute.

–Sam

1 Like

I guess that makes sense. A usual API call gets you THIS minute, whilst using day_offset=0 gets you all data from the beginning of the day to LAST minute.

Shoulda clarified… I am using the start_time and end_time, which are working right now… and they give LAST minute… same as day_offset=0;

It’s something I can live with, since I can’t get the real 3-second data… :wink:

1 Like