Tuesday, May 9, 2017

Important Hints to iGate developers

By connorgoodwolf - Own work, CC BY-SA 4.0
There are a lot of APRS iGate implementations out there, and new ones are popping up every few months these days. A lot of the new ones are made for various microcontroller chips and development boards (Arduino, ESP8266 and friends). Some are made for SDR sticks, some are built in mobile phone applications. Many of them reduce the amount of hardware needed to run an iGate by fully removing the need for a TNC, a full-blown computer, or an actual amateur radio transceiver.

Very cool stuff!

It would seem like writing an iGate would be very easy: receive packets off the air, put them in TNC2 format, and pass them to the APRS-IS servers over TCP (specifications: connecting, design, details). But it turns out that there are quite a few things that can go wrong in the implementation, and in fact, some things seem to be going wrong the same way fairly often. Some bugs are fairly common and some seem to pop up in multiple different iGates, old and new.

Currently the most popular ones are due to the false assumption that all packets are either plain 7-bit English ASCII text with no NUL bytes or other binary stuff, or some specific Unicode encoding such as UTF-8. These assumptions lead to the packets getting truncated at the first NUL (binary 0) byte, or otherwise modified in a way that scändic and øthér ïnternátiønal characters get mutilated if they don't happen to use the same encoding. Sometimes a truncation of a packet at the first NUL may lead to the packet being concatenated together with the next packet.

I get to decode the corrupted packets on aprs.fi, and I also get to process the corrupted ones in the aprsc APRS-IS server software which is fairly common these days. If the bugged packets are not detected and handled, there usually are some folks complaining to me about the odd results, since they are often received from aprsc and viewed on aprs.fi.

I can't blame the authors of the new applications – I have to admit that APRS documentation is not great, and it's easy to stumble across the same bugs if you haven't fixed them once before.

Every now and then I spend an evening trying to identify some of these bugs in the APRS-IS client applications and file bug tickets to the software authors. Last night I filed 3 packet corruption bug reports on Github for 3 different iGate applications. These are not the first ones, I've probably filed 30 in the past. It's not hard to spot these once you know what to look for.

I'd like to take this chance to thank to all the authors who fix the bugs in a timely manner and release the fixed versions!

Based on this experience I wrote a little document listing common iGate bugs: IGATE-HINTS.md. It'd be great if all APRS software authors could glance through it and make sure these bugs are not present in their own software. If you spot someone writing a new iGate, please pass on the link to the new author – hopefully these bugs will then be avoided right from the beginning.

Unfortunately some buggy iGate applications are orphaned and no longer developed. A few of those applications are fairly popular, too. It'd be great for the network if iGate operators could gradually upgrade to applications which still get updates, and don't exhibit any of the packet-corrupting bugs.

A lot of iGates are also not getting updated, even though they run software which would have updates available. Some iGate operators feel that the iGates work, sort of, since most packets go through them just fine, most of the time. I think I'll try to make aprs.fi highlight some of the more common issues on the station info page in the future.

If you know a little Python or C, and you've got a little extra time, let me know – there are some bugs out there in the Open Source Land which could use a helping hand.

Saturday, March 25, 2017

New version of aprs.fi's APRS packet parser released: Ham::APRS::FAP v1.21

Yesterday I uploaded a new version of the APRS packet parser used by aprs.fi, Ham::APRS::FAP, to the CPAN. It's open source, so if you're a little bit weird like me and enjoy programming in Perl, this might be useful. Here's a full list of changes:
  • Improve make_position() to support HMS UTC timestamp. make_position() now returns the packet type character so that it can signal the presence of a timestamp.
  • Improve make_position() to support comment string, !DAO! extension, altitude encoding. Fix rounding errors in lat/lon/speed. Support generating packets with no speed or course. Take optional parameters in a hash parameter. Implement unit tests for make_position().
  • Set up Travis automatic running of the (existing) unit test runs.
  • Additional character escaping in regular expressions to deal with deprecated functionality in Perl 5.22.
The packet parser is one of the more complete APRS decoders, although I'm not sure if any of the parsers deal with all of the APRS features and packet variants yet.

Monday, December 12, 2016

aprs.fi moving to TLS

In an effort to increase security on the web at large scale, web browser vendors and other organisations such as Google are making changes which encourage web sites to move to TLS/SSL encryption. Even web sites which previously did not seem to need it – ones with static content only, and ones without any login / password functionality. This is good and fine – even if it's not a banking web site, it's good that third parties along the network can not observe or modify the content being downloaded. The Chrome web browser has started to label non-encrypted sites with an informative '(i)' symbol which warns the user that "Your connection to this site is not private", and will eventually make those warnings stronger. Google gives better ranking in the search results for https sites.

A real, practical issue right now is that the geolocation Javascript API is no longer available on non-HTTPS sites in recent Android and Chrome versions. This actually broke map center and tracking functionality on the aprs.fi web site.

I wholeheartedly support this movement, it will make the Internet a better place!

These days, with performance-improving developments such as ECDHE, GCM mode AES and hardware accelerated AES, running TLS on a web server is not much of a performance issue any more. Most of the CPU time will be spent on application logic, anyway.

The fun part is that HTTP/2, a new protocol used by modern web browser to access web sites, is only used over TLS/HTTPS – it is not available over plaintext connections. HTTP/2 is faster than older HTTP versions, and a surprising side effect is that a web site may well open up faster over HTTP/2 + TLS than over HTTP 1.1 without the encryption!

Picture not related. I just took it last summer. Kyyttö cows © Sappion luomu.
Before now, aprs.fi has only used TLS/HTTPS for its login and user account management pages. Fairly soon I will have a maintenance break on the aprs.fi servers, upgrade the operating system to the next major release, and install a new version of the aprs.fi software which supports access over both HTTP and HTTPS. To reduce duplicate content (same stuff being available over both HTTP and HTTPS) it will prefer HTTPS and nudge clients that way every now and then, but initially plaintext access should be possible, too. Later on, if there are no surprises, the nudges will gradually become stronger.

There are a few issues which need to be addressed. There are possibly a few Amprnet users accessing this site over amateur radio frequencies. On the other hand, they're then practically surfing the Internet over radio, and probably doing a few requests to other encrypted sites now and then, too, so maybe it's not a big problem for them.

Another thing is that apparently users in China can't access the Google Maps API over HTTPS, so those users would still need the plaintext access for now. I might make the zh.aprs.fi site plaintext only, and bump those users that way, or something along that way. Maybe the Amprnet users can use that, too?

Sunday, October 9, 2016

aprs.fi iPhone/iPad app update: v1.6.2

Version 1.6.2 of the aprs.fi iPhone/iPad app went out yesterday evening. I've been adding a few features and fixes here and there on the weekends, but most of the larger changes in the code are actually under the hood and not yet visible for the users. I've also spent a good amount of time upgrading the aprs.fi web site backend and fixing a few bugs here and there.

Here are the visible changes in 1.6.2, all of them were recently requested by users of the app:

  • Tapping a station on the map multiple times switches the track line colour for that station.
  • Track line width can be adjusted in Settings. The default size is slightly thicker than the previous default.
  • The maximum tracked station tail/track length is now 6 hours.
  • Previously selected stations and addresses can be deleted by swiping the respective table row to the left.
  • Previously selected address search results are retained even if the application is killed manually by the user.
  • Some small visual adjustments (more space between "Beacon now!" and "Delete station" buttons, etc).

Some folks are also asking for a feature to track multiple stations at the same time, but that can already be done, since version 1.0 – just tap the '+' button on the additional stations info view to add them in tracking. This video demonstrates tracking many stations in the iOS app.

Here are some of the new features added in previous versions this summer:

  • Setting to hide/show station callsign labels on the map
  • Setting to disable (accidental) map pinch rotation
  • Latest packets of an APRS station can be viewed by clicking a new button in the station information view. Tap a packet to decode it using the aprs.fi packet decoder.
  • Filtering feature to control what is shown on the map, for hiding weather and AIS stations, for example. Complex custom filtering will be available in the future.
  • A feature to display road traffic information (traffic jams). The information can be hidden in Settings.

I'm also currently working on APRS-IS beaconing and messaging features, and arbitrary date/time range selection (with long time ranges), but they're not complete yet. Larger features take more time.

The app is already getting quite happy reviews, but it still needs a few features to really make it complete. The 5-star average rating and these reviews are for version 1.6.1, from users from the USA:

Sunday, December 13, 2015

aprs.fi iOS app for iPhone and iPad!

The new, official aprs.fi iOS application went live on the App Store a week back. It runs on the iPhone and iPad, and currently supports iOS 7 to iOS 9. Purchase once – run on up to 10 devices associated to the same iTunes accounts!

It provides immediate, near-real-time visibility to APRS traffic around you, and has quick search-as-you-type station and address search functions. Zoom around the world as easily as on aprs.fi, or look up stations by their callsign. Multiple stations can be tracked at the same time.

Telemetry, weather, and APRS station statistics can be viewed as graphs.

The new high-resolution symbol graphics look crisp on Retina displays. iOS 9 Split-Screen multitasking is supported on applicable devices (iPad Air 2, iPad Pro, iPad mini 4, and newer). Landscape and portrait modes work too, of course.

Position beaconing to aprs.fi works great, although, as usual, GPS use in the background reduces battery life noticeably. The minimum transmit interval slider can be used to reduce transmission rate, allowing the GPS to turn off for longer periods of time. The app does not require location information, but it can be helpful for automatic map centering and calculating distances.

A little expectation management needs to be done at this point: Some future features will only be available for a small additional yearly price, through an in-app purchase. That price will be lower than the purchase price of the app. The rationale behind this is simple: There's only a rather small number of APRS iOS users around the world, and once most of them have bought the app once, there will be no more income from the app, ever. Having a small, steady income nicely keeps up the motivation in maintaining and improving the application in the future. A few APRS apps have already been practically abandoned on the app store, with no updates in one, two or three years. The original app, simply named "APRS" recently got deleted and replaced with something completely unrelated.

Some new features, and all bug fixes of course, will be free updates. The current version does not even include any support for in-app purchasing yet – it'll maybe come up some time next year, after the base features are ready.

Purchase the app now, and you'll get a nice APRS web site for free!

Frequently Asked Questions

What about Android?

Yes, maybe later. It takes a lot of time to produce these things. I concentrated on iOS mostly because all my devices happen to be iOS, I have some previous experience on iOS development, and APRSDroid is already so good.

Filtering, I wish to hide AIS vessels and/or weather stations?

Yes, that's on the top of my list of things to do, I can't live without them either.

Can not beacon to APRS-IS?

Not yet! This is the aprs.fi app, not an APRS-IS app, so it talks primarily to the aprs.fi service, not other services. Beaconing to APRS-IS will come later, stay tuned.

Connecting to the aprs.fi database makes the immediate real-time view happen, so that there's no need to wait 30 minutes for everyone to transmit their position once after opening the app. On the downside, if aprs.fi happens to be down, the app doesn't do much either. Luckily aprs.fi has proven to be very stable during its operation since 2007.


Yes, of course, later.

Tuesday, November 3, 2015

New symbol graphics and better support for mobile devices

Old symbols, scaled up, pixels obvious
[updated 2015-11-03: the set is now available on github.]

I've just upgraded aprs.fi to use my new APRS symbol graphics set. The new symbols are drawn in vector format (as opposed to a raster format at a fixed resolution), allowing them to be rendered at larger and smaller sizes without distortion or blurriness. The new symbols are slightly larger than the old ones, making them easier to recognise on modern displays having smaller pixels than the old ones. They're also available in double resolution so that they're properly sharp on the 4K/retina displays found on many modern tablets, phones and computers!

New symbols, scaled up - no pixelation!
One downside is that they are just a little bit bigger than the old ones, taking up more space on the screen. On the other hand, the old ones were a bit too small on modern displays, and the very small resolution did make it hard to understand what the symbol tried to mimic.

The new aprs.fi symbol set is available as open source on GitHub, in both vector (Adobe Illustrator/PDF) and raster (PNG) formats. Other APRS applications may then use them, too, at no cost. Raster renderings are available in 24x24, 48x48, 64x64 and 128x128 pixel resolutions - drawing from raster sprites in apps is usually quicker and easier than working with the vector source material. Having the vector sources makes it possible to improve them and and replace individual symbols easily. If you need to render other resolutions or make some other fine tuning, you can run Illustrator for free for 30 days. The symbol set release even comes with a little piece of javascript which crunches out the 3 PNG files (primary, secondary, overlay characters) at the 4 resolutions in a few seconds.

Naturally I did not draw all of the symbols myself. Many are loosely or strongly based on the original symbol graphics, primarily to keep the familiar and consistent look. Some symbols I obtained from other sources, such as Wikipedia. In those cases I picked SVG versions which allow commercial reuse (source known, and the work is placed on public domain, or with a CC license which allows adaptation and commercial reuse). In any case, the source and copyright information is documented separately for each symbol.

The aprs.fi symbol graphics set does not contain additional symbols for overlays yet, mostly because it takes lots and lots of time to draw them, and the effort it took to create this set was pretty high already. Maybe later!

There is one obvious difference in the new symbol set: the "ham store" symbol has been replaced with a more generic "store" shopping cart, reflecting the current \h symbol definition in the master index. Please use the 'H' overlay character to specify an amateur radio shop.

To complement the symbol graphics, I've previously published a machine-readable (CSV/JSON/XML/YAML) APRS symbol description index, which is easier to integrate in applications than Bob's master list.

Improved mobile device support

aprs.fi will now work better than before on mobile devices. I fixed the signup/login flow and most of the text and data table views to scale more nicely on small devices, allowing  horizontal scrolling of tables. It'll need some more work to make it very nice, but this is a good start.

Friday, October 9, 2015

OpenStreetMap available again

Earlier this year aprs.fi was unavailable for a short while after Google disabled Maps API access due to mixed OSM/Google content being visible on the site.

I now spent a couple evenings setting up OSM again in a way that would not interfere with Google's policy of not allowing Google content such as Street View or address search results to be shown on top of non-Google maps. The result of that work is now live on aprs.fi.

OpenStreetMap maps are again visible, but Street View buttons and controls are hidden while in OSM mode.

While using OSM, address searches are done by using the GeoNames database. City lookups work quite well, but address searches do not seem to work, at least not for Finland. City/country lookups do not return information on how large the found place is, so zoom level does not adjust automatically to cover the place.