ArchiveSW - Display & Data Archive Storage

vinceskahan · November 7, 2019, 11:45pm

Hey Gary do you have a page of screenshots of your display someplace we can see ?

GaryFunk · November 8, 2019, 5:07am

I don’t but I can easily create one.

Are there any specific screens you want to see?

eric · November 11, 2019, 5:13pm

I’m on v1.8.16.030 (for sure I force updated files)
Running Altertables still gives me crashed tables

2019-11-11 18:07:33 Error alterTableIndex (create index): HubStatus timestamp_serial_number
  HA_ERR_CRASHED_ON_REPAIR
2019-11-11 18:07:33 Error alterTableIndex (create index): HubStatus serial_number
  HA_ERR_CRASHED_ON_REPAIR
2019-11-11 18:07:33 Error alterTableIndex (create index): HubStatus datestamp
  HA_ERR_CRASHED_ON_REPAIR
2019-11-11 18:07:33 Error alterTableIndex (create index): HubStatus timestamp
  HA_ERR_CRASHED_ON_REPAIR
2019-11-11 18:07:33 Checking table index: PrecipEvent
2019-11-11 18:07:33 Checking table index: RapidWind
2019-11-11 18:07:33 Error alterTableIndex (create index): RapidWind serial_number
  HA_ERR_CRASHED_ON_REPAIR
2019-11-11 18:07:33 Error alterTableIndex (create index): RapidWind hub_sn
  HA_ERR_CRASHED_ON_REPAIR
2019-11-11 18:07:33 Error alterTableIndex (create index): RapidWind timestamp
  HA_ERR_CRASHED_ON_REPAIR
2019-11-11 18:07:33 Error alterTableIndex (create index): RapidWind datestamp
  HA_ERR_CRASHED_ON_REPAIR

and the devices tab ends dead (can’t load page)

Can you please have a look at a spare moment ? No sure I’m the only one but would appreciate your script to be at your level of exigence (perfect)

chuckmc · November 11, 2019, 9:20pm

I’m also seeing an error. Also on v1.8.16.030
Clean install on a new sd card.

Sorry for the small text. It reads :
2019-11-11 16:17:49 Error in insertHubStatus (4): ER_TRUNCATED_WRONG_VALUE_FOR_FIELD: Incorrect integer value: ‘[479,2]’ for column archivesw.HubStatus.mqtt_stats at row 1

GaryFunk · November 11, 2019, 9:24pm

Did you run altertables? The field type needs to be changed.

chuckmc · November 11, 2019, 9:31pm

Yes but I just ran it again.

And the log file Shows an error: Error changeTable: DailySky

GaryFunk · November 25, 2019, 3:32am

I have a new repairtable.js ready for download.

USAGE: node repairtable hubevents
replace hubevents with the name of the table that is bad.

wget -N https://fsoft.com/archivesw/repairtable.js

chuckmc · November 30, 2019, 12:19am

Thanks @GaryFunk,
I’ll give it a try soon.
Chuck

todd.lorey · December 2, 2019, 4:30am

I tried using the new repairtable.js but it didn’t seem to fix things. Also ran Altertables and UpdateFiles along with some reboots/restarts of the archive/server for good measure.

I’m having the same issues as @chuckmc and @eric . In general though, the program is working well in that I’m primarily using it to upload to CWOP and PWS.

I tried running the repairtable on most of the items in the altertables.log. I figured things like PrecipEvent, AirBackfill and similar ones didn’t need it and weren’t contributing to the errors.

GaryFunk · December 2, 2019, 5:07am

Are you still seeing errors writing to DailySky?

todd.lorey · December 2, 2019, 5:19am

Yeah, that’s what the AlterTables.log files says.

But in the Archive.log I’m still getting this error:
2019-12-01 21:15:48 Error in insertHubStatus (4): ER_TRUNCATED_WRONG_VALUE_FOR_FIELD: Incorrect integer value: '[479,102]' for column archivesw.HubStatus.mqtt_stats at row 1

Here’s the AlterTables.log entries:

2019-12-01 20:22:20 Starting AlterTables v1.8.16.030
2019-12-01 20:22:23 Dropping table index: DailySky
2019-12-01 20:22:23 Dropping table index: PrecipEvent
2019-12-01 20:22:23 Dropping table index: SkyObservation
2019-12-01 20:22:23 Dropping table index: StrikeEvent
2019-12-01 20:22:23 Checking table index: AirBackfill
2019-12-01 20:22:23 Checking table index: AirObservation
2019-12-01 20:22:23 Checking table index: DailyAir
2019-12-01 20:22:23 Checking table index: DailySensor
2019-12-01 20:22:23 Checking table index: DailySky
2019-12-01 20:22:23 Checking table index: DeviceEvents
2019-12-01 20:22:23 Checking table index: DeviceStatus
2019-12-01 20:22:23 Checking table index: FWUpdate
2019-12-01 20:22:23 Checking table index: HubEvents
2019-12-01 20:22:23 Checking table index: HubStatus
2019-12-01 20:22:23 Checking table index: PrecipEvent
2019-12-01 20:22:23 Checking table index: RapidWind
2019-12-01 20:22:23 Checking table index: SkyBackfill
2019-12-01 20:22:23 Checking table index: SkyObservation
2019-12-01 20:22:23 Checking table index: StrikeEvent
2019-12-01 20:22:23 Checking table index: Xrain
2019-12-01 20:22:23 Checking table index: Z_version
2019-12-01 20:22:23 Error changeTable: DailySky
2019-12-01 20:22:23 ChangeTable: DailySky
2019-12-01 20:22:23 ChangeTable: HubEvents
2019-12-01 20:22:23 ChangeTable: HubStatus
2019-12-01 20:22:24 Process changes complete
2019-12-01 20:22:24 Update Version complete

GaryFunk · December 2, 2019, 2:32pm

It’s failing on HubStatus. I’ll look at that nasty see what I find.

todd.lorey · December 2, 2019, 3:25pm

Just a shot in the dark…

The HubStatus message changed to add an extra value to the mqtt stats section, so now it’s an array:

Status (hub) [type = hub_status]
{
  "serial_number":"HB-00000001",
  "type":"hub_status",
  "firmware_revision":"35",
  "uptime":1670133,
  "rssi":-62,
  "timestamp":1495724691,
  "reset_flags": "BOR,PIN,POR",
  "seq": 48,
  "fs": [1, 0, 15675411, 524288],
      "radio_stats": [2, 1, 0, 3],
      "mqtt_stats": [1, 0]
}

GaryFunk · December 2, 2019, 3:38pm

Right. Altertables was to correct the field but for some reason it did not.

todd.lorey · December 2, 2019, 5:42pm

If it helps, here’s a real UDP packet for hub status, as captured by my OpenHAB system. I’m sure you have them too but maybe there’s something weird with mine??

{"serial_number":"HB-00003177","type":"hub_status","firmware_revision":"114","uptime":2903890,"rssi":-33,"timestamp":1575308221,"reset_flags":"BOR,PIN,POR","seq":290085,"fs":[1,0,15675411,524288],"radio_stats":[2,2,26,3],"mqtt_stats":[483,102]}

And one for the sky and air device status. Maybe this has something to do with the Devices tab not displaying correctly?

{"serial_number":"SK-00003301","type":"device_status","hub_sn":"HB-00003177","timestamp":1575308527,"uptime":916744,"voltage":3.33,"firmware_revision":43,"rssi":-30,"hub_rssi":-37,"sensor_status":0,"debug":0}

{"serial_number":"AR-00002283","type":"device_status","hub_sn":"HB-00003177","timestamp":1575308518,"uptime":46663345,"voltage":3.44,"firmware_revision":20,"rssi":-61,"hub_rssi":0,"sensor_status":4,"debug":0}

GaryFunk · December 3, 2019, 6:09pm

For those that have the HubStatus error, please get this file and run it.

wget -N https://fsoft.com/archivesw/altertables.js

todd.lorey · December 3, 2019, 6:46pm

Gary,
EDIT: not sure why the big font! Can’t seem to change it.
Also, I noticed that the version of altertables is the same: 1.8.16.030 /edit

I tried it and I’m still getting the same errors.
Here’s some snippets from the log files:

2019-12-03 10:35:56 Starting UpdateFiles v1.6.16.030
2019-12-03 10:35:56 Getting files for update 1.8.16.030
2019-12-03 10:35:57 Checking for new Script files
2019-12-03 10:35:57 Checking for new JavaScript files
2019-12-03 10:35:59 Received new file ==>: /altertables.js
2019-12-03 10:36:08 Checking for new Config files
2019-12-03 10:36:11 Checking for new Panel files
2019-12-03 10:36:13 Checking for new Views files
2019-12-03 10:36:28 Checking for new Desktop files

–

2019-12-03 10:40:31 Starting AlterTables v1.8.16.030
2019-12-03 10:40:32 Dropping table index: DailySky
2019-12-03 10:40:32 Dropping table index: PrecipEvent
2019-12-03 10:40:32 Dropping table index: SkyObservation
2019-12-03 10:40:32 Dropping table index: StrikeEvent
2019-12-03 10:40:32 Checking table index: AirBackfill
2019-12-03 10:40:32 Checking table index: AirObservation
2019-12-03 10:40:32 Checking table index: DailyAir
2019-12-03 10:40:32 Checking table index: DailySensor
2019-12-03 10:40:32 Checking table index: DailySky
2019-12-03 10:40:32 Checking table index: DeviceEvents
2019-12-03 10:40:32 Checking table index: DeviceStatus
2019-12-03 10:40:32 Checking table index: FWUpdate
2019-12-03 10:40:32 Checking table index: HubEvents
2019-12-03 10:40:32 Checking table index: HubStatus
2019-12-03 10:40:32 Checking table index: PrecipEvent
2019-12-03 10:40:32 Checking table index: RapidWind
2019-12-03 10:40:32 Checking table index: SkyBackfill
2019-12-03 10:40:32 Checking table index: SkyObservation
2019-12-03 10:40:33 Checking table index: StrikeEvent
2019-12-03 10:40:33 Checking table index: Xrain
2019-12-03 10:40:33 Checking table index: Z_version
2019-12-03 10:40:33 Error changeTable: DailySky
2019-12-03 10:40:33 ChangeTable: DailySky
2019-12-03 10:40:33 ChangeTable: HubEvents
2019-12-03 10:40:33 ChangeTable: HubStatus
2019-12-03 10:40:33 Process changes complete
2019-12-03 10:40:33 Update Version complete

    2019-12-03 10:44:31 Error in insertHubStatus (4): ER_TRUNCATED_WRONG_VALUE_FOR_FIELD: Incorrect integer value: '[496,102]' for column `archivesw`.`HubStatus`.`mqtt_stats` at row 1

Archive.err
TypeError: Cannot read property 'length' of undefined
    at Query.<anonymous> (/home/pi/archivesw/archive.js:933:16)
    at Query.<anonymous> (/home/pi/archivesw/node_modules/mysql/lib/Connection.js:525:10)
    at Query._callback (/home/pi/archivesw/node_modules/mysql/lib/Connection.js:491:16)
    at Query.Sequence.end (/home/pi/archivesw/node_modules/mysql/lib/protocol/sequences/Sequence.js:83:24)
    at /home/pi/archivesw/node_modules/mysql/lib/protocol/Protocol.js:236:14
    at process._tickCallback (internal/process/next_tick.js:61:11)
TypeError: Cannot read property 'end' of undefined
    at tcpExit (/home/pi/archivesw/archive.js:1260:12)
    at onExit (/home/pi/archivesw/archive.js:1240:2)
    at process.<anonymous> (/home/pi/archivesw/archive.js:1224:5)
    at process.emit (events.js:203:15)
TypeError: Cannot read property 'end' of undefined
    at tcpExit (/home/pi/archivesw/archive.js:1260:12)
    at onExit (/home/pi/archivesw/archive.js:1240:2)
    at process.<anonymous> (/home/pi/archivesw/archive.js:1219:5)
    at process.emit (events.js:203:15)

GaryFunk · December 3, 2019, 7:49pm

Ah, Crap. The idiot sitting at my computer updated the file but forgot to put it on the server. I’m at a VAMC getting a spinal injection. I’ll be back home in two hours.

todd.lorey · December 3, 2019, 7:52pm

Spinal injection?! That sounds fun.
No worries. Sounds like you might have bigger fish to fry. Get to it when you can.
Thanks again!!

GaryFunk · December 3, 2019, 11:31pm

The new file is up. You should get version 1.8.16.032

Topic		Replies	Views
Smart Weather Archive for Cheap Local Storage Tempest Owners wfarchive , archivesw	64	5127	April 11, 2018
WeatherFlow, PWSweather.com, and Rachio Developers	36	4351	August 1, 2020
Announcements & Updates WeatherFlow PiConsole api , websocket , wfpiconsole , python	59	97629	April 29, 2026
WeatherFlow PiConsole - Archive WeatherFlow PiConsole	2757	29805	February 10, 2023
Modified PiConsole that gets data from UDP WeatherFlow PiConsole tempest	20	1157	October 10, 2023

ArchiveSW - Display & Data Archive Storage

Related topics