Lightning Strikes Weewx Database

Developers Third-Party Integrations
1

I am trying to track Lightning in my weewx database.
Can someone explain to me how the data is stored in these observations.
When the strike data is inserted, it is showing as 20.9 for strike count.
I am trying to create a graph that shows the total strikes for each month.

I’ve tried the weewx google group with no success.

Thanks

lightning_strikes = lightning_strike_count.AR-00015732.obs_air
avg_distance = lightning_strike_avg_distance.AR-00015732.obs_air
lightningYYY = distance.AR-00015732.evt_strike
lightningZZZ = energy.AR-00015732.evt_strike

2

First off, I assume that you extended the WeeWX database schema to hold these values, since it does not have lightning data support out of the box.

WF sends two different UDP packets with lightning data. One is evt_strike, which is sent immediately whenever a lightning strike is detected:

distance.AR-00015732.evt_strike
energy.AR-00015732.evt_strike

The second packet is sent every minute as a summary in obs_air:

lightning_strike_count.AR-00015732.obs_air
lightning_strike_avg_distance.AR-00015732.obs_air

I have no idea how often the obs_air figures reset, be it every minute/packet, hourly, daily, etc. (Confession - I never extended my WeeWX database to try it out, since we don’t have frequent T-storms here to test it.)

I just pulled a few UDP obs_air packets from my two running Airs just now, and the two zeroes are the lightning data here:

{"serial_number":"AR-00012619","type":"obs_air","hub_sn":"HB-00001310","obs":[[1566510332,995.40,24.87,98,0,0,3.50,1]],"firmware_revision":20}

{"serial_number":"AR-00003375","type":"obs_air","hub_sn":"HB-00001310","obs":[[1566510354,995.50,25.10,89,0,0,3.50,1]],"firmware_revision":20}

{"serial_number":"AR-00012619","type":"obs_air","hub_sn":"HB-00001310","obs":[[1566510392,995.50,24.87,98,0,0,3.50,1]],"firmware_revision":20}

{"serial_number":"AR-00003375","type":"obs_air","hub_sn":"HB-00001310","obs":[[1566510414,995.50,25.11,89,0,0,3.50,1]],"firmware_revision":20}

{"serial_number":"AR-00012619","type":"obs_air","hub_sn":"HB-00001310","obs":[[1566510452,995.50,24.87,97,0,0,3.50,1]],"firmware_revision":20}

{"serial_number":"AR-00003375","type":"obs_air","hub_sn":"HB-00001310","obs":[[1566510474,995.50,25.13,89,0,0,3.50,1]],"firmware_revision":20}

{"serial_number":"AR-00012619","type":"obs_air","hub_sn":"HB-00001310","obs":[[1566510512,995.50,24.87,97,0,0,3.50,1]],"firmware_revision":20}

{"serial_number":"AR-00003375","type":"obs_air","hub_sn":"HB-00001310","obs":[[1566510534,995.50,25.14,89,0,0,3.50,1]],"firmware_revision":20}

I had suggested in the past to follow Matthew Wall’s AS3935 raw sensor driver’s database schema extensions and names, but have never looked at the driver code to see how he is calculating averages or if he is storing the raw strike data anywhere.

https://github.com/weewx/weewx/wiki/as3935

Long story short, both lightning observation types are commented out in the WF WeeWX driver’s sample sensor_map because I am not sure where to stick the data or where/how I would ever use it historically. From a SQL perspective, my thought is that throwing the evt_strike figures into a separate database/table might be useful in that you can compute your own averages from the raw data.

If anyone comes up with a solution to store/report on the data, please let me know and I’ll include it in the docs…

3

Thanks for the insite. I extended the database for the values and I have been collecting data for about a week now. I am using the belchertown skin to display all of my data. I was using the charts to plot the lightning data. I just haven’t found a good way to count each record and display that as the total strikes. It wants to use the column data so it reports the wrong information.

4

I’m using neowx skin but that’s irrelevant I think. And I only store the counts (per minute), not the distance. In the plots, I aggregate the data by 10-min intervals. In the table, I show total strikes of the day (sum), number of strikes in the last hour and most intense time of the day (with most strikes/min).
The “code” is nothing special, just using weewx standard calls.
I have another setup with my home-made AS3935 and use it in the same way except the RPi calculates the strikes/min and provides the data to weewx. Here, I have complete list of strikes (with their timestamps) stored in a plain text log file on a RPi.

1 Like
5

You’re perhaps focussing too much on Belchertown. Focus on your database content. What does your archive database have in it for the recorded strike data ? Do you see any ‘counts’ that look like anything other than whole numbers in your database ?

I have my data recorded in influxdb and the counts look right to me. No fractional counts.

6

It looks like each record (distance.AR-00015732.evt_strike)is considered a strike.
The record holds the distance and time of the strike. Could I add another column and populate a 1 in it. This would represent the strike count. I could then sum that within the chart. I just don’t know how to do that with in the code.
I can easily plot the distances. I am just having a hard time summing up the records to display the total strikes within the chart.
I am seeing fractional counts for this observation ightning_strike_count.AR-00015732.obs_air. When I do a sum of lightning_strikes from the database for august 2019 it is 32.1 strikes/.

7

If you are not logging the energy data in evt_strike packets (since what it even represents in the AS3935 is up for debate), hijacking energy.AR-00015732.evt_strike from the LOOP packets may get you half way there.

strike_count=energy.AR-00015732.evt_strike

We don’t care what’s in it, just that it exists and contains a value. The next step is to modify WeeWX’s StdCalibrate section to make it always be 1.

[StdCalibrate]
[[Corrections]]
strike_count = 1

If that doesn’t work, this one looks valid from the examples in the link above:
strike_count = strike_count - strike_count + 1

Let me know if this crazy roundabout idea works…

8

“great scott” I think your on to something. LOL
I will update the code tonight and try it. Of course we are not set to get any thunderstorms. I might try it on another column.

Thanks again

1 Like
9

use a piezo lighter near the air, it’ll trigger like a lightning

10

I made the database updates. I just need an ignightor to trigger the lightning I’ll keep you posted.

11

We had a nice storm last night . The weather flow app reported 600 lightning strikes. The weewx database logged only 41. I’m not sure that the udp packet below is sent with every strike.
distance.AR-00015732.evt_strike
energy.AR-00015732.evt_strike

This worked well for recording the 1 strike per entry.

strike_count = strike_count - strike_count + 1

StdCalibrate]
[[Corrections]]
strike_count = 1.

It strike count was just too low.
Thanks

12

next time you have a story coming, run my UDP listener and look for just evt_strike and it’ll show you every event that the hub broadcasted…

That said, if you have a slow raspi on wifi, there is never any guarantee that your pi won’t miss something that was broadcast, depending on how good your wifi is at your site. If you have at least a pi3 ‘wired’ to your network you should catch everything.

13

Ok thanks, It successfully installed. When I run

pi@raspberrypi : **~ ** ~ python listen.py --decoded
-bash: /home/pi: Is a directory

I get this. These is where I get lost. I am new to the whole raspberry Pi.

14

when i run :
**pi@raspberrypi: sudo python /home/pi/wf/listen.py --decoded
Traceback (most recent call last):
File “/home/pi/wf/listen.py”, line 558, in
s.bind((ADDRESS, MYPORT))
File “/usr/lib/python2.7/socket.py”, line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use

15

(Being new to the pi is a stressful thing. Do the free introduction to linux course at edx.org - it’s a free course, just don’t sign up for the $199 certificate of course. It’ll lower your blood pressure. Really !)

Hmmm - that’s really interesting that you get address in use. I can’t even make that happen on purpose, I am running weewx with the WF UDP driver ‘and’ up to 3 copies of my listener just to try to find a way make it break that way, and I can’t. Wow.

Run sudo netstat -lnp udp and see how many processes are listening for port 50222.

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      562/sshd
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      803/exim4
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      791/nginx: master p
tcp6       0      0 :::22                   :::*                    LISTEN      562/sshd
tcp6       0      0 ::1:25                  :::*                    LISTEN      803/exim4
tcp6       0      0 :::80                   :::*                    LISTEN      791/nginx: master p
udp        0      0 0.0.0.0:50222           0.0.0.0:*                           1297/python3
udp        0      0 255.255.255.255:50222   0.0.0.0:*                           1055/python3
udp        0      0 0.0.0.0:68              0.0.0.0:*                           522/dhcpcd
udp        0      0 192.168.1.177:123       0.0.0.0:*                           541/ntpd
udp        0      0 127.0.0.1:123           0.0.0.0:*                           541/ntpd
udp        0      0 0.0.0.0:123             0.0.0.0:*                           541/ntpd
udp        0      0 0.0.0.0:44239           0.0.0.0:*                           347/avahi-daemon: r
udp        0      0 0.0.0.0:5353            0.0.0.0:*                           347/avahi-daemon: r
udp6       0      0 :::39965                :::*                                347/avahi-daemon: r
udp6       0      0 fe80::c21c:6534:e40:123 :::*                                541/ntpd
udp6       0      0 ::1:123                 :::*                                541/ntpd
udp6       0      0 :::123                  :::*                                541/ntpd
udp6       0      0 :::5353                 :::*                                347/avahi-daemon: r
raw6       0      0 :::58                   :::*                    7           522/dhcpcd

See the two 50222 entries ? The first one listening for 0.0.0.0:50222 is weewx. The second one is my listener if you have it running.

16

I didn’t have the ip address in the script. Once I added it. I was able to run it.
I have this running on 50222
udp 0 0 10.0.0.32:50222 0.0.0.0:* 14305/python
udp 0 0 255.255.255.255:50222 0.0.0.0:* 8528/python

Where does the log output to?

Thanks for the information on the course. I will be taking that for sure.

17

I’m not sure I get what you did, but if you have a pi with both wifi and wired ethernet running the listener can be a little confused. I have a command-line option to limit it to one listening address (rarely needed)

If you run the listener with the --raw option and no other options, you should see a lot of output to the screen. The --decoded just makes it easier to tell things apart. Other options let you include/exclude types of observations and optionally publish to MQTT if you’re so inclined.

All the output goes to the screen (it’s a bit verbose) unless you are using it to publish to MQTT using that set of command-line options from the readme file.

If you can trigger strikes manually as somebody else mentioned, then you ‘should’ see it see the events hopefully.

18

I hardwired the Pi and turned off the wifi.
I added my IP address to the python.py
line 82
ADDRESS = ‘10.0.0.32’
When it was blank I got the error.
when I run sudo python /home/pi/wf/listen.py --decoded
I get a cursor with nothing. No output just a blank screen.
I ran sudo netstat -lnp udp and it was running.
I have to quit the terminal to get anything to work.

19

You don’t need to edit the code, just add a command-line switch per this description.

I don’t know how to help with “to get anything to work” as that’s not descriptive enough for me to even guess what you’re seeing. Only thing I can suggest is reading the docs for the app on the github page, and particularly these examples of how you might do things, but they might be above your Linux skill level I think.

Basically you can run something like:

nohup some_command_here >/some/outputfile 2>&1 &

That says “run some command and don’t hangup when my window closes, and send the standard output to /some/outputfile and send the error output to the same place, and do this all in the background so my shell is still usable”. Really. Trust me :slight_smile:

That leaves a process running forever. You’d need to kill the process to stop it (or reboot, but reboots are soooooo Windows and this is Linux).

Anyway, to monitor rapid_wind continually to a file that will grow pretty rapidly since rapid_wind is every few seconds, you would do:

# log just rapid_wind
nohup python listen.py --decoded --limit='rapid_wind' > /tmp/wf.log 2>&1 &

# log just lightning strikes
nohup python listen.py --decoded --limit='evt_strike' > /tmp/wf.log 2>&1 &

# log just two kinds of things
nohup python listen.py --decoded --limit='rapid_wind evt_strike' > /tmp/wf.log 2>&1 &

That help any ?

2 Likes
20

I ran this:

I’m not sure if i installed it wrong.
I followed the instructions by running
ran this:
pi@raspberrypi: ~$ python --version
python2 - version 2.7.9

ran this:
pi@raspberrypi: ~$ sudo apt-get install -y python-pip && sudo pip install paho-mqtt
-mqtt
Reading package lists… Done
Building dependency tree
Reading state information… Done
python-pip is already the newest version (9.0.1-2+rpt2).
The following packages were automatically installed and are no longer required:
lxkeymap python-cairo python-gobject python-gobject-2 python-gtk2
python-xklavier realpath
Use ‘sudo apt autoremove’ to remove them.
0 upgraded, 0 newly installed, 0 to remove and 12 not upgraded.
[3]+ Exit 1 nohup python /home/pi/wf/listen.py --decoded --limit=‘rapid_wind’ > /tmp/wf.log 2>&1
Requirement already satisfied: paho-mqtt in /usr/local/lib/python2.7/dist-packages

ran this:
**pi@raspberrypi:nohup python /home/pi/wf/listen.py --decoded --limit=‘rapid_wind’ > /tmp/wf.log 2>&1 &
output:
[3] 5627
[2] Exit 1 nohup python /home/pi/wf/listen.py --decoded --limit=‘rapid_wind’ > /tmp/wf.log 2>&1

when i open the wf.log file
this is in the file:

nohup: ignoring input
Traceback (most recent call last):
File “/home/pi/wf/listen.py”, line 589, in
s.bind((ADDRESS, MYPORT))
File “/usr/lib/python2.7/socket.py”, line 228, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use

thanks for all your help