Magic UTF-8 support, weighted callsigns and other updates

The aprs.fi service now supports UTF-8 in most APRS packet types, such as messages, comments and status messages. Using UTF-8 enables proper universal messaging for APRS users. The UTF-8 encoding of Unicode is backwards compatible with ASCII, and it can be transmitted without problems on the APRS-IS and the APRS radio frequencies. It does not require the end-user to manually switch between "compatible ASCII" and "unicode" mode depending on the language or destination, since English text (like this) is transmitted in exactly the same way in both ASCII and UTF-8. It also supports all of the special characters in other languages (åäöæø ÅÄÖÆØ ßüÜ). UTF-8 works on findu.com, too.

I strongly recommend all APRS authors to implement UTF-8 encoding in their APRS software for proper internationalization support! To get some UTF-8 testing packets, send an APRS message to the callsign UTF-8, and my testing responder will respond with four messages containing English text, Scandinavian and German special characters, and a few Japanese words.

In addition to UTF-8, I also added a magic recode feature (inspired by the irssi IRC client), which tries to guess the character set of the original packet based on the character distribution. It currently attempts to support ISO-8859-7, ISO-8859-15, CP437 and CP850. Because of the overlaps between these character sets, this feature can not be made to work perfectly - the task is impossible. It's guesswork. To get your characters to show correctly, please transmit them in UTF-8.


Inspired by weighted tag clouds, callsigns shown in the Other SSIDs list are now weighted according to the time of the last position update from that callsign. If the position was updated within 24 hours, the callsign becomes bigger than usual, and the other way around. For example: KJ4ERJ, WB4APR

The position history database table was migrated to a partitioned table. Or rather, the migration was started - new data is being inserted in a partitioned table, and old data is still in the unpartitioned one. This is a very technical change which does not make a lot of difference to most of you folks, but improves things a lot on the server side. I can now delete old history data in 1-day chunks very quickly with a minimal performance hit. Inserting new data requires less disk I/O and smaller memory buffers (thanks to the partitioned indexes).

I've also upgraded aprs.fi to use Ham::APRS::FAP version 1.13, upgraded the web server software, fixed some XHTML syntax, done some performance tweaks on the real-time map, and added support for APRS targets without a current known position (required for receiving telemetry from a station without a known position).

This upgrade was a big one, and required about an hour of downtime. Sorry for the trouble. I hope I didn't break anything this time.