Friday, March 13, 2015

Device identification database updated, available for other apps

Most APRS devices and applications transmit an unique AX.25 destination callsign in all their packets, so that receiving stations can figure out which application or device is transmitting each packet. Bob Bruninga maintains a tocalls.txt index file, which lists all the assigned destination callsigns.

Those devices which use the Mic-E encoding to transmit position packets encode the latitude and a little bit of the longitude within the destination callsign, in which case something else has to be done for device identification. Mic-E device IDs are encoded around the comment text, with one character in the beginning of the comment text and zero to two characters in the end of the comment. The Mic-E type codes are indexed in mic-e-types.txt.

Now, when an application such as aprs.fi wishes to automatically decode the destination callsigns and type codes to readable application names, as seen on the aprs.fi station information page, the author of that application needs to collect all the device identifiers from those two files, and somehow convert them to application source code, or a configuration file that can be read by the application. The master files are written with human interpretation in mind, and it's rather hard to make an application automatically parse out the identifiers from them. It needs to be done manually, and whenever new devices or applications are published, all the applications wishing to detect the new ones need to get an update. All softwares authors need to get notified that there are some new devices, and then somehow add the devices in their respective configs. That's quite a lot of extra work that would be better spent writing some fancy new features instead.

OH7LZB-7, correctly identified as a Kenwood TH-D72,
at Mikkeli International yesterday afternoon.
Aircraft and training provided by MIK at Helsinki-Malmi.

For aprs.fi, I initially made a Perl module, Ham::APRS::DeviceID, and published it as open source, so that other software authors could use the index too, and skip the manual labour-intensive part of parsing the master text files. For some obscure reasons a number of programmers have chosen to use other programming languages than Perl, and the module was of limited usefulness for them.

To improve the situation, I converted the device index in YAML (Yet Another Markup Language), which is easy to read and edit by humans, and also easy to read and edit by computer programs. I also wrote a little converter program which parses the tocalls.yaml file and outputs the same data in JSON and XML formats, which are popular file formats for passing data between computer systems.

I then updated the original Perl module to read the YAML file, and removed the old database that was embedded in the code. DeviceID.pm 2.00 and later use tocalls.yaml. I also updated aprs.fi to use the new version of the Perl module, and tocalls.yaml. The update brought a number of new devices to aprs.fi, including the newer Yaesu radios.

Other programmers who wish to do device identification are welcome to download the YAML, JSON or XML files and use those. It should be straightforward to automatically update the device index during an application build, or even automatically update the index directly in the application.

New devices should still be first added to Bob's files, and only then to tocalls.yaml. If you have a new device which has already been added to Bob's index, please make a pull request or an issue ticket to update tocalls.yaml accordingly.