Parser updates and new delayed packet filtering feature

I've updated the APRS device identification module Ham::APRS::DeviceID to include detections for new APRS devices (KissOZ, anyfrog, unknown mic-e, SARTrack, Altus Metrum, SM2APRS, aprsc, NW Digital Radio UDR56K). The new version, 1.06, has been published on CPAN for your open source pleasure, and installed on aprs.fi.

I also published version 1.19 of the Ham::APRS::FAP packet parser used by aprs.fi. It only includes a small fix to the binary value telemetry bit order in Base91 comment telemetry. But at least that keeps me in the regular 1 release per year schedule! Just in time!

The larger feature in the aprs.fi upgrade is that it now utilizes the sequence number present in Base91 comment telemetry for detecting delayed packets.

APRS packets notably lack any sort of sequence number that could be used to detect old duplicate packets arriving late, or to place them in the correct order. Newer Byonics trackers can transmit telemetry (battery voltage, temperature, etc) within position packets using the new Base91 comment telemetry format, which is quite tightly packed, and also includes a sequence number in a range of 0...8280. This makes it really easy to detect a packet having an older sequence number than the previously received packet. There are already a lot of those devices in use.

aprs.fi has now been updated to make use of the sequence number when available. It's more reliable and faster than the old methods of detecting too high speed or duplicate packet content. Due to the amount of broken igates and digipeaters delaying packets for minutes or tens of minutes (often due to a buggy Kantronics KPC3+ in KISS mode) it can make sense for a tracker to transmit the sequence number alone without any actual telemetry! Packets dropped due to this will be shown with this error message:

Delayed or out-of-order packet (sequence number)

In practice it will usually be shown together with the error message about dropped telemetry, since the telemetry content is also ignored due to the duplicate sequence number: 

2012-12-31 00:24:51 EET: N0CALL-9>SY0UWY,WIDE1-1,WIDE2*,qAR,N00CALL:
`pOnqgd>/'"KQ}MT-RTG|!D&='a|!w;a!|3
[Duplicate telemetry sequence, Delayed or out-of-order packet (sequence number)]

You can use the Decoded mode of the raw packets view to see sequence numbers and telemetry values decoded from each packet.

Time zone selection on aprs.fi

Most of yesterday I spent on implementing proper time zone selection on aprs.fi. You can now select whichever local time zone you wish using a (hopefully) user friendly map tool.

Click on Preferences (on the right side of the real-time map), select the Units and time tab, and click the Change button next to the currently selected time zone. That'll bring up the new time zone selection tool.

Point the mouse roughly at your location (the state or country which uses your preferred time zone), click, click Save to return to the main Preferences view, and click Save again.

You can also select either the time zone or country from the drop-down menus. Or, if browser geolocation works for you, click the I'm Feeling Lucky button. It's not very accurate (I'm being placed in Moscow for example), but it probably works for more than 90% of the population.

I didn't do all of this from scratch - I copy with pride. Thanks go to these two excellent open source projects: timezonepicker  and TimezoneJS.Date

aprs.fi upgraded: track tail length adjustable

Kukkatarhan Cissus sings:
"We wish you a merry Christmas!"

I upgraded the aprs.fi service yesterday evening. I rebooted both main servers after upgrading operating system and database components. After that I upgraded the aprs.fi software itself.

It's been a long while since the last significant upgrade. There have been small patches and quiet bug fixes here and there. My excuse is the time I've put into completing and deploying aprsc, a new open-source APRS-IS software written in C together with OH2MQK. It's now in use on about 1/3rd of all APRS-IS servers and appears to work fine.

There is now a separate track tail length drop-down selector on the right side of the map. It's now possible to select "show last positions of all stations in the are for the past 24 hours" but still only show a track tail for the past 30 minutes, for example. Drawing tracks for 24 hours tends to clutter the map a lot and it can also be very slow on many devices.

The old wildcards-on-initial-navigation bug is finally fixed. If you go to http://aprs.fi/OH7* (use a wildcard character, or otherwise multiple stations come up from the initial search) clicking on the callsigns within the popup did not work. Fixed!

Fixed exporting of stations which have transmitted telemetry.

Lots of improvements on Tetra protocol support and closed enterprise service model (not visible on aprs.fi).

Improvements in command line tools for service maintenance.

Small performance improvements here and there.

I also upgraded the operating system and database (the usual security fixes and some small performance improvements).

We've got new kittens too. Been taking a lot of kitten photos!

Power meter monitoring using Munin & an Arduino

It's been a bit quiet on the blog lately, but that's just because I've been a bit busy with both aprs.fi and other related projects. I've been writing the new aprsc APRS-IS server software together with OH2MQK, went to DCC 2012 in Atlanta to do presentations about both aprsc and aprs.fi, aprsc has been deployed to about 30% of APRS-IS servers and I've had to fix some bugs too. I'll tell you more about those things later – this blog post is about one of the "other projects", a one-nighter to monitor the electric power consumption of your home using an Arduino, a phototransistor, and Munin. I have to admit it took another day to package and document it for you, and write this blog post, and that it can take another night to adjust the physical installation of the sensors.

The Arduino C code, the two Perl scripts used on the Linux box to get the numbers to Munin, and a README text file with more instructions is available on Github. Click on the ZIP button to download it all.

Here's an example diagram drawn by Munin, with annotations for some events we triggered. Munin might not be the best way to collect and report measurements like this, but it's certainly one of the easiest methods to get data graphed on the web using a Linux box. apt-get install munin munin-node and you're ready to go!

The local power company remotely measures power usage in 1-hour steps (using a GSM/3G connection), lets me view that data on their web site (after logging in, of course), and advertises that as being "very accurate" and "high resolution". I don't think it's much to brag about! Munin polls every 5 minutes which makes it really easy to see how much each device or action consumed energy.

Most modern power meters installed by the utility companies have a red LED which blinks according to the power consumption. The meters installed in our small apartment building blink 1000 times per kWh (kilowatthour) – it reads "1000 imp/kWh" next to the led.

To count the blinks you'll need a photodiode or phototransistor. I used an Osram SFH 300 sold by the local electronics store (Partco). It's sensitive to visible light (not just infrared, like some models).

I'm measuring consumption of 4 apartments (with the consent and for the benefit of my neighbors) and the building's consumption (heating system, outdoor & basement lights, etc), so I attached 5 phototransistors.

I used an Arduino Duemilanove, but it's current replacement model Arduino Uno or just about any other model will work just fine.

The Atmel microcontroller has internal pull-up resistors, so the phototransistors can be connected directly to the board without any other components. The Uno board costs about 20€ or $25 USD, has USB and a good set of analog and digital I/O pins. It's powered by USB – no wall wart required.

It's very easy to program in C using the Arduino IDE, which comes with plenty of simple example code. If you're a programmer with no electronics experience, or an electronics guy with no programming experience, or something in between, you'll love the Arduino.

Here's the box attached to the wall using 3M Dual Lock tape.

Since we measure the blinks using optical coupling – there is no electrical connection between the meter and the photodiodes – installing the instrumentation is completely safe and legal. It's about as safe as taking photos of the meter, which is very safe indeed.

With the small bias voltage given by Arduino's internal pull-ups, the photodiodes are a bit insensitive, so they need to be aligned very closely and sharply at the blinking red led of the meter.

Remember that it might not be wise to publish real-time measurement data on the Internet, since everyone could see when you're home. But that applies to APRS on your car, too.

My meter actually has two blinking leds. The one on the left is for kiloVoltAmpsReactivehours (kvarh), which indicates phase-lagged power consumed by an inductive load. My power company isn't yet charging me for that separately, so I'm not measuring it.