Back button should now work!

Ocicat kitten, 5 weeks, (c) Heikki Hannikainen

Last night I spent a couple hours trying to improve URL navigation on the aprs.fi real-time map. The back and forward buttons have behaved quite weirdly for some time, and that simply needed to be fixed. The upgrade went in just a few minutes ago.

With the help of the HTML5 history API I think I mostly got it right now. It is of course likely that I missed some corners, so keep the feedback coming - either in blog comments or posts on the discussion group.

Basically, the browser history now only gets a new entry when the user initiates a "major change" in the map status: a new callsign or a new address is searched. Those new history entries can be navigated using the web browser's back and forward buttons.

Minor state changes, such as changing the time range / tail length parameters, and any changes not initiated by the user, should only update the current browser history entry. After first looking up station A, then looking up station B, and then changing time range, clicking back will take you to station A. Clicking forward will then take you to station B - with the changed time range! Try it out, you'll see what I mean.

Naturally this won't work so nicely with steam-powered web browsers, or the oldest boat anchors out there.

Christmas is coming, and postcards need to be printed, so I was required to take some kitten photos with seasonal props yesterday.

aprs.fi upgraded: track tail length adjustable

Kukkatarhan Cissus sings:
"We wish you a merry Christmas!"

I upgraded the aprs.fi service yesterday evening. I rebooted both main servers after upgrading operating system and database components. After that I upgraded the aprs.fi software itself.

It's been a long while since the last significant upgrade. There have been small patches and quiet bug fixes here and there. My excuse is the time I've put into completing and deploying aprsc, a new open-source APRS-IS software written in C together with OH2MQK. It's now in use on about 1/3rd of all APRS-IS servers and appears to work fine.

There is now a separate track tail length drop-down selector on the right side of the map. It's now possible to select "show last positions of all stations in the are for the past 24 hours" but still only show a track tail for the past 30 minutes, for example. Drawing tracks for 24 hours tends to clutter the map a lot and it can also be very slow on many devices.

The old wildcards-on-initial-navigation bug is finally fixed. If you go to http://aprs.fi/OH7* (use a wildcard character, or otherwise multiple stations come up from the initial search) clicking on the callsigns within the popup did not work. Fixed!

Fixed exporting of stations which have transmitted telemetry.

Lots of improvements on Tetra protocol support and closed enterprise service model (not visible on aprs.fi).

Improvements in command line tools for service maintenance.

Small performance improvements here and there.

I also upgraded the operating system and database (the usual security fixes and some small performance improvements).

We've got new kittens too. Been taking a lot of kitten photos!

Small fixes on Monday morning

On Saturday and Sunday I did a little coding, and installed the new version before leaving for work this morning.

When an user tries to save a filter list twice with the same name, and error message is now displayed. It turns out that a lot of users press the "Save as new list" button after making any changes to the existing list, and end up with 20 lists (or more!). That button is only for creating a new list. Any changes to existing lists are saved immediately without pressing any additional buttons.

The Preferences view got a new hint text pointing users towards the new Filter tool button. A lot of users are complaining about the "missing filters".

When an user account is deleted, data related to that account is now more thoroughly deleted. That includes web stations, alerts, favourites, and so on.

Alert configuration changes should now actually do all the necessary database changes to enable new alarms and disable old ones.

The real-time map should now more reliably draw a symbol on the last point of a track. An old bug sometimes left the last point without a symbol.

When a client computer wakes up in the morning from a good night's sleep mode, the data reload should now be a quick one, instead of a very slow one.

Session cookie processing was also improved, and memcached was upgraded.

Again, it doesn't make sense to take a photo of software, and a blog needs to have some photos, so kittens it is again. This is FIN*Kukkatarhan Bellium, a chocolate-spotted Ocicat, posing in Mikkeli a few weeks back.

User preferences saved with user account

I did a quick upgrade last night. A few bugs were found immediately after the upgrade and needed to be fixed but no serious harm was done.

The upgrade includes revamped session and preferences storage. The user-visible change is that user preferences are now stored in the server backend and attached to your user account if you're logging in when saving the preferences. They will be used on other computers, browsers and mobile devices as long as you're logged in with the same account.

The preferences can still be saved if you're not logged in, in which case they're attached to a browser cookie.

A less visible change is that the settings and session cookies set in the browser are now much shorter - just random keys, instead of containing the whole settings or session data. This makes the HTTP requests smaller, which improves network performance marginally. On a slow mobile connection it might actually make a difference.

I don't have a related photo to attach, so kittens it is again! Buxus and Brassia, aged 6 weeks.

Map of the APRS-IS

Have you ever wondered what the APRS-IS looks like? Well, here you go (click on the link, 1.2 megabytes, 3154 x 3072 PNG image):

Map of the APRS-IS

• Servers with less than 5 clients are not shown.
• Only servers running javAPRSSrvr are shown.
• Darker colour indicates more clients connected.
• Green servers are core servers, orange ones are T2 hubs, red ones have a T2* name (most of them belong to the Tier2 network), purple ones are WXQA servers, the rest of them are blue.
• Inter-server links between hubs and core servers are blue, the rest of the links are green.

Another project I'm working on: we've got 6 ocicat kittens!  I've set up a live kitten camera. Cheap Chinese webcam (foscam) + ffmpeg + ffserver does the trick.

Position ambiguity support

Happy new year!

Today I implemented ambiguous position support in aprs.fi. Many APRS transmitters using MIC-E or uncompressed packets can be configured to intentionally transmit less precise positions. This may seem a bit backward at first, but there are perfectly good reasons to do so. Some people might want to transmit a rough location of their car without revealing the exact parking spot where their expensive radio gear spend their night in. Some might like some aspects of APRS but wish to adjust the level of privacy by hiding their precise location.

Ambiguity is configured by setting the number of digits which will be truncated from the end of the position. Plaintext APRS positions are transmitted in degrees and decimal minutes (DD° MM.mm'), with two decimals of minutes. When ambiguity is set to 1, it'll be truncated to DD° MM.m', 2 will transmit DD° mm', 4 will transmit DD° only, resulting in a resolution of 1 degree.

Stations transmitting ambiguous positions are now shown with a purple dashed border around the callsign label. The precision of the position can be visualized by hovering the mouse pointer on top of the station symbol – a purple rectangle will pop up, showing the area of ambiguity. The station symbol is drawn at a slightly randomized position within that area. Also, the info page now says "Position ambiguous: Precision reduced at transmitter by 3 digits, position resolution approximately 18.5 km."

There is one known big bug in this code: the ambiguity is not yet stored together with each position in the position history database. Only the ambiguity of the most recently received position is stored. If you turn ambiguity on or off, the effects on aprs.fi might be slightly weird when looking up old tracks. I'll fix that later when I next change the position history table schema.

I've been told some Kenwood mobile rig owners have accidentally enabled ambiguous positions in their transmitters without realizing what the setting actually does. This change should make the effect visible in a more meaningful way. On the TM-D710, the setting lives in menu 606 (BEACON INFORMATION), POSITION AMBIGUITY.

Off-topic news: Kukka (FIFE GIC Moosegrove Orqideas Doiradas), our lovely chocolate spotted ocicat, is expecting kittens in late January. The sire is CFA CH Windhaven Salt and Pepper, an immigrant from the USA (family tree). She's gained quite some weight, as can be seen in this photo. Kittens for sale later this spring, details at the cattery site!

Temporary markers and AIS receiver position uploading

Here's the list of changes in today's upgrade. A few of the smaller bug fixes were already deployed on Friday and Saturday.

You can now drop temporary markers on the map by right-clicking the map and selecting "Add marker". The markers can be moved around, just drag-and-drop.

It's now possible to conveniently upload the position of an AIS receiver. The instructions are on the AIS feeding page, step-by-step instructions, step 8.

At this point you probably figured that the context menu (which opens up on right-click) and the marker feature add a good bit of infrastructure, and can be later used for a bunch of other nice features like object/item uploading and home position marking. Stay tuned.

The Facebook app's canvas page failed to load since a couple weeks. Fixed!

Humidity parsing of h0 (100%) was fixed for normal APRS weather packets. Negative Fahrenheit temperature parsing was fixed in peet bros ULTW packets. These fixes went to the Ham::APRS::FAP svn trunk, and will be included in the next FAP release.

Magic UTF-8 support, weighted callsigns and other updates

The aprs.fi service now supports UTF-8 in most APRS packet types, such as messages, comments and status messages. Using UTF-8 enables proper universal messaging for APRS users. The UTF-8 encoding of Unicode is backwards compatible with ASCII, and it can be transmitted without problems on the APRS-IS and the APRS radio frequencies. It does not require the end-user to manually switch between "compatible ASCII" and "unicode" mode depending on the language or destination, since English text (like this) is transmitted in exactly the same way in both ASCII and UTF-8. It also supports all of the special characters in other languages (åäöæø ÅÄÖÆØ ßüÜ). UTF-8 works on findu.com, too.

I strongly recommend all APRS authors to implement UTF-8 encoding in their APRS software for proper internationalization support! To get some UTF-8 testing packets, send an APRS message to the callsign UTF-8, and my testing responder will respond with four messages containing English text, Scandinavian and German special characters, and a few Japanese words.

In addition to UTF-8, I also added a magic recode feature (inspired by the irssi IRC client), which tries to guess the character set of the original packet based on the character distribution. It currently attempts to support ISO-8859-7, ISO-8859-15, CP437 and CP850. Because of the overlaps between these character sets, this feature can not be made to work perfectly - the task is impossible. It's guesswork. To get your characters to show correctly, please transmit them in UTF-8.


Inspired by weighted tag clouds, callsigns shown in the Other SSIDs list are now weighted according to the time of the last position update from that callsign. If the position was updated within 24 hours, the callsign becomes bigger than usual, and the other way around. For example: KJ4ERJ, WB4APR

The position history database table was migrated to a partitioned table. Or rather, the migration was started - new data is being inserted in a partitioned table, and old data is still in the unpartitioned one. This is a very technical change which does not make a lot of difference to most of you folks, but improves things a lot on the server side. I can now delete old history data in 1-day chunks very quickly with a minimal performance hit. Inserting new data requires less disk I/O and smaller memory buffers (thanks to the partitioned indexes).

I've also upgraded aprs.fi to use Ham::APRS::FAP version 1.13, upgraded the web server software, fixed some XHTML syntax, done some performance tweaks on the real-time map, and added support for APRS targets without a current known position (required for receiving telemetry from a station without a known position).

This upgrade was a big one, and required about an hour of downtime. Sorry for the trouble. I hope I didn't break anything this time.

Beta test: Multiple target tracking on live map

I've done some rather heavy changes in the map code, which might well break some things. The big visible change is that you can now look up and track multiple targets (up to 20) in the real-time map. I'm mostly worried that it'll slow down things too much, but we'll see...

To try it out, surf to the beta site and enter a comma-separated list of callsigns (OH7LZB-9,OH2RDK,OH2RDS), or click 'start tracking' on multiple targets!

It's also running with a completely different web server software than before. We are still fixing bugs with the new setup, so the beta service might be unavailable at times. Please report any bugs as comments on this post, or send a private email to the address shown on the profile page.

Thank you!

- Hessu and the bug-eating cats

Status and comment texts

As a little early morning exercise I've made aprs.fi show the status message in the info balloon of the current position on the real-time map, and also in the KML. Status message is shown in purple, and the comment text is shown in green.

There has been some confusion about these messages. There are three kinds of "status/comment" messages you can attach to your position. For example, SM4IVE-9 (info) is sending two of them.

The comment text is sent together with the position, in the end of the position packet. Here's an example packet with a comment text of www.sm4ive.com:

SM4IVE-9>APERXQ,WIDE2-2,qAR,SM5NRK-2:!5908.38N/01532.45E>000/000/A=000148www.sm4ive.com

The status message is sent as a separate packet which starts with a '>' character:

SM4IVE-9>APERXQ,WIDE2-2,qAR,LA6TMA-1:>{AT0B4}aprstracker-0.11-16f648

The Mic-E status message is encoded in a mic-e packet using just a few bits, and can contain one of these 8 standard messages: Off duty, En route, In service, Returning, Committed, Special, Priority, Emergency. 7 custom messages (Custom 0 to Custom 6) are also defined. All Mic-E packets contain this status message, and it only consumes a couple of bits in the message, so this requires the least bandwidth from the APRS channel. On the other hand, it can only express the few predefined values.

I would recommend using only the comment text, since it is sent in a single packet together with the position. The status message is sent in a separate packet which increases congestion.

If a status message is required (for example, if the text really needs to be so long that it doesn't fit in the comment text), the status message should not be sent too often. Certainly not as often as the position packet.

In the following photo Armi frowns upon seeing a long, static status packet:

Small changes (and a small cat)

During the weekend I replaced the date browsing menu with a new AJAX-based one. The downside is that the new one doesn't tell you how many positions were received during a year, month or day. The upside is that it's much, much faster than the old one. The old one was generated completely when you looked up a station on the map, which in turn required scanning the whole position history of the station, which caused a lot of disk IO, which filled up the caches. The new one just quickly checks whether there are any points during the year, month or day, and does it only when needed.

The new one only shows the days when new points have been inserted. For a fixed station like a digipeater it shows the initial date when the station was first heard by aprs.fi - the old one showed all of the days between the initial packet and the latest packet at the same position.

I also described the date selection menu in the FAQ. It turns out a lot of people don't know about it. Would some people like to try to write a beginner's manual?

The info page now shows the latest telemetry from the station instead of the number of telemetry packets stored. It's much more useful and much quicker to look up.

Most importantly - there is now a GIF animation in a couple of places! Honestly, I never thought I'd bother with such an useless thing which takes such an enormous amount of time to produce. But I accidentally figured out how Gimp does it, so I just had to try it once. Sorry!

In case you were wondering - a new kitten arrived on Saturday evening! The photo is maybe a bit off-topic, but it sure is cuter than the computers and radios.

Planned upgrade: new account system

It's been a bit quiet here for some time, since I've been working on some larger changes in the aprs.fi back-end system (and learning how to fly, crash and repair an RC airplane). I'm planning to install the changes on the production servers early on Tuesday or Wednesday morning (7th or 8th of July 2009, around 4:00 UTC). The outage shouldn't be a long one, if all goes well.

One of the visible changes is a new authentication and authorization layer, which is based on a conventional email address + password model with a team-based authorization system. It'll enable some new features and services later (once I get around to implementing those) - in the first phase it just lets you sign up for submitting AIS data without manual work on my part.

There's no need to worry, as a normal user viewing data on aprs.fi, you won't need to sign up for an account. Logging in is completely optional, and I won't encourage or beg everyone and their cat to sign up. In fact, the "login with your nickname to see the map" model will go away - you'll end up directly on the map page when you arrive on the site, and the "most usual" site entrance path will be easier and quicker than before.

I will be adding an "introductory" entrance page for first-time visitors later, so that there's a chance to let them know what this site is all about. But it'll only be shown once for each visitor.

Date range with weather and telemetry lookup

I'm on a 2-week sick leave (my gallbladder was removed last week), and have some spare time for enhancements and bug fixes. Rest of the time I'm entertaining Armi the cat, who joined us on Friday.

This morning I added arbitrary date range lookups to the weather and telemetry pages. It allows you to look up a detailed graph between any two dates or times. It also has a little calendar widget for your convenience.

I've also done some XHTML validity fixes for better browser compatibility, switched to a smaller position marker on the static maps, and added some navigation links in the bottom of most pages. I also fixed the bug of date selection menu not appearing when clicking on 'start tracking' on the live map.

Also, the /info/ page of a weather station now shows a brief 'latest weather' report instead of the number of weather reports in database. It's probably much more interesting, and it's much quicker to load, too, since it's almost always cached in memory and doesn't require a potentially large disk read.