aprs.fi discussion group

To facilitate peer support and discussion about aprs.fi I have created an aprs.fi discussion group on Google Groups. A bit like Yahoo groups, you can send and receive group mails using email (just like a regular mailing list), or you can use the web interface, or read the posts using an RSS reader.

If you've got questions on how to use aprs.fi, or if you'd like to share tips or tricks, please join the group and send an email there. I'd also like to encourage experienced users to answer any questions posted by others, if you happen to know the answer. I'd greatly appreciate your support - any time I don't need to spend writing emails I can use for developing new features.

CWOP, Traffic view and Service status, and More

Since CWOP (Citizen Weather Observer Program) users moved to their own IS server pool, aprs.fi has not collected weather data from them. I've connected aprs.fi to the CWOP servers to collect that data again, and the CW* stations should now show up. If you don't wish to see them on the map, you can always hide the WX stations in the Preferences.

Several users have also requested that support for the Google Maps Traffic would be added. It's there now, along with a couple of other popular overlays behind the More button. The traffic data is collected using crowdsourcing methods (Google's view), so it's available on those roads where people use Google Maps to navigate (excluding iPhone users).

The heard and gated maps have also been fixed a bit, they no longer load all the other stations in the view. That was a stupid bug on my part. These views have also received some technical improvements and speedups. As a side-effect I erased the 48-hour history tables, so those views (and the "heard by" tables in the info pages) will be a bit empty for the next day or two. Sorry about that.

The info pages now show symbols together with the callsigns in the tables. Hmm, sorting by symbol doesn't seem to work, though.

There's also a new service status view.

Sortable tables and log Y scale

The tables shown on the info page are now sortable - click on the column header to sort, and click on it again to sort in descending order. The "nearby stations" table doesn't sort nicely, though, since it's actually two tables embedded in a single table, and it's sorted as if it was a single table of content. I'll have to figure out something for it.

I've also changed the igate / digipeater receiver performance graph to use a logarithmic Y scale, so that distances with smaller amount of activity are more visible.

Merry Christmas!

APRS receiving performance graph

The graphs page of APRS igates and digipeaters now shows a new graph which tries to describe how well the station's receiver is performing by plotting the number of position packets received from a given distance during a month. There will be multiple lines, one per month, over the past few months. Here are a few examples:

OH2RCH close to downtown Helsinki, but there have been a few bad GPS fixes:

OH7AA fairly quiet on 144.800, peaks generated by stationary digipeaters:

OH6RDK is a digipeater without any activity within 20 km range (the receiver probably does work at that range, though):

VK2TV-4 has an HF receiver:

As usual, there are a few catches:

• The graphs tell more about the amount of activity within a given distance, than the receiver performance at that range. A receiver in the middle of a big city might hear well from a long distance, but there is simply so much traffic close to the receiver that the linear Y scale of the graph hides the position packets from the distant stations. Maybe I should switch to a logarithmic scale, or would that be too confusing?
• Stations transmitting invalid or fake coordinates will skew the stats. If the transmitting station claims to be 1000 km away (due to a bad GPS fix), my best guess can only be that the station actually is that far away.
• Items and objects are not shown here, since they are usually far away from the transmitter which transmitted the packets.
• AIS support is still in the works, I need to add a method to configure the coordinates of the receiver.

Magic UTF-8 support, weighted callsigns and other updates

The aprs.fi service now supports UTF-8 in most APRS packet types, such as messages, comments and status messages. Using UTF-8 enables proper universal messaging for APRS users. The UTF-8 encoding of Unicode is backwards compatible with ASCII, and it can be transmitted without problems on the APRS-IS and the APRS radio frequencies. It does not require the end-user to manually switch between "compatible ASCII" and "unicode" mode depending on the language or destination, since English text (like this) is transmitted in exactly the same way in both ASCII and UTF-8. It also supports all of the special characters in other languages (åäöæø ÅÄÖÆØ ßüÜ). UTF-8 works on findu.com, too.

I strongly recommend all APRS authors to implement UTF-8 encoding in their APRS software for proper internationalization support! To get some UTF-8 testing packets, send an APRS message to the callsign UTF-8, and my testing responder will respond with four messages containing English text, Scandinavian and German special characters, and a few Japanese words.

In addition to UTF-8, I also added a magic recode feature (inspired by the irssi IRC client), which tries to guess the character set of the original packet based on the character distribution. It currently attempts to support ISO-8859-7, ISO-8859-15, CP437 and CP850. Because of the overlaps between these character sets, this feature can not be made to work perfectly - the task is impossible. It's guesswork. To get your characters to show correctly, please transmit them in UTF-8.


Inspired by weighted tag clouds, callsigns shown in the Other SSIDs list are now weighted according to the time of the last position update from that callsign. If the position was updated within 24 hours, the callsign becomes bigger than usual, and the other way around. For example: KJ4ERJ, WB4APR

The position history database table was migrated to a partitioned table. Or rather, the migration was started - new data is being inserted in a partitioned table, and old data is still in the unpartitioned one. This is a very technical change which does not make a lot of difference to most of you folks, but improves things a lot on the server side. I can now delete old history data in 1-day chunks very quickly with a minimal performance hit. Inserting new data requires less disk I/O and smaller memory buffers (thanks to the partitioned indexes).

I've also upgraded aprs.fi to use Ham::APRS::FAP version 1.13, upgraded the web server software, fixed some XHTML syntax, done some performance tweaks on the real-time map, and added support for APRS targets without a current known position (required for receiving telemetry from a station without a known position).

This upgrade was a big one, and required about an hour of downtime. Sorry for the trouble. I hope I didn't break anything this time.

Maintenance break on Saturday 5th of December

On Saturday, roughly between 18 and 19 UTC, I upgraded operating system components and installed security patches. These upgrades required taking the service down for some time, and even data collection was affected for a short while. Sorry for the trouble!

Before the reboot, the master server had an uptime of 681 days. Thanks go to the UPS and the diesel generator - there have been a few small power outages in Espoo during that time.

In the near future I'll be installing a new master server for aprs.fi, the hardware is already waiting for the OS installation in the basement.

I've also bought an 80 GB Intel X25-M SATA solid-state drive (SSD), a very quick flash-based hard disk-like device. The amount of disk I/O operations is starting to become an issue on aprs.fi, so I'm going to try moving the busiest database tables (which are also the smaller ones) on SSD. We'll see how that improves things! It might even allow me to implement some new features (which require additional I/O capacity).

Silja Europa steering & comments

Silja Europa (live position), a rather large passenger ferry (photos), has some steering issues, and is currently doing circles near Åland. News in Finnish: hs.fi, iltasanomat.fi. There are 1659 people onboard, but there is no immediate danger, since the ship can still steer using the propellers, although in a rather coarse manner.

Another small bit of news is that I've disabled anonymous commenting on this blog due to comment spam - I'm getting tired of manually deleting all the spam. From now on, you'll need a Google account to comment, or an account on one of the supported OpenID-enabled services. Sorry for the trouble.

During the last couple weeks I've been working on making the service UTF-8 clean for proper universal text messaging. It almost works in my testing installation, but there are still some quirks I need to clean up.

I've also installed a new database replica server at another physical site in downtown Helsinki, so that I'm able to bring the service alive without significant data loss in case of a complete hosting site loss (fire, flooding, extended network outage). The main servers are located in Espoo, Finland.

Ham::APRS::FAP 1.13 released

Version 1.13 of Ham::APRS::FAP was uploaded to CPAN today. The tests seem to be passing nicely, and the code has been running on aprs.fi for a long while, so the upgrade should be safe.

Ham::APRS::FAP is the APRS packet parser used by aprs.fi. If you wish to decode APRS packets in Perl, this is what you need. It's pretty fast, it's stable, and it can report it's errors in a developer-friendly way. And it's free (as in free speech).

Version 1.13 contains the following small fixes:

• Allow anyone to update telemetry parameters, skip the source callsign check
• Allow a PHG of 0000 for deleting PHG
• Added new error code sym_inv_table for invalid symbol table char
• Added local time zone parsing to object timestamps
• Fixed comments parsing for last resort !-location packets
• Parse APRS message rejects (negative acknowledgments)

Some of the improvements were implemented by Tapio, OH2KKU, while others were done by myself.

Embedded maps tracking fixed

The embedded map broke with the upgrade on Saturday - it stopped tracking the configured station (he_track parameter). That's fixed now. Sorry about that, I need to focus a bit more on testing before releasing...

Statistics after the upgrade

On 31 Oct I switched from Apache to nginx. Here are some graphs from one frontend. Which would you prefer, based on these graphs?




I only added the temperature plugin yesterday (before the upgrade), but ... the CPU core temperatures did drop by 2°C. Now, does nginx reduce global warming?

Multiple targets on live map (and other small stuff)

On Saturday I did the planned upgrade. The big new user-visible feature is that you can now track multiple targets on the map. Just type in multiple callsigns separated with a comma: Rautauoma, OH2RDK, or click on 'start tracking' more than once. This code has been running on the beta site for a week now.

There are also some smaller improvements, like symbols being shown in the "other SSIDs" list, callsign links in APRS paths shown in the info balloons, fixed web search, and a JavaScript memory leak workaround for IE.

Oh yeah, referenced digipeaters and igates are now shown when you look up a day in the history. There is little bug in there, though - it shows their current positions, instead of their positions on the selected day. But hey, they rarely move.

There were also some rather large changes under the hood, causing a longer outage than usually. I replaced the Apache web server software with nginx, which allows me to run with HTTP keep-alive again (without running out of memory), and should generally handle larger amounts of users without that much trouble. It also helps with resource management - I'm now running separate web server processes for each service instead of a single Apache process pool for a large number of virtual servers. Now, if one service gets a beating, the others won't be affected so easily. Upgrades and configuration errors in one service won't break others, either. The downside is that name-based vhosts are out of the question.

Today's photo was taken on 11th of August at the summer cottage in Maaninka. EXIF says 9:39:17 PM. It's freezing and very dark in Helsinki today, so I'd rather go back a few months.

PS. I'll be in Bordeaux, France for the next 2 weeks or so.

Upgrades on Friday or Saturday

I'm planning to upgrade aprs.fi on either Friday evening (30 Oct 2009 around 1500z) or Saturday morning (31 Oct 2009 around 0700z). I'll be reconfiguring the service to use a completely different web server software than before, so there will be an outage of some sort. If everything goes well, it won't be more than a few minutes. There might be some surprises on the way, since I don't have a proper lab setup to run the new configuration with a simulated heavy load.

Beta test: Multiple target tracking on live map

I've done some rather heavy changes in the map code, which might well break some things. The big visible change is that you can now look up and track multiple targets (up to 20) in the real-time map. I'm mostly worried that it'll slow down things too much, but we'll see...

To try it out, surf to the beta site and enter a comma-separated list of callsigns (OH7LZB-9,OH2RDK,OH2RDS), or click 'start tracking' on multiple targets!

It's also running with a completely different web server software than before. We are still fixing bugs with the new setup, so the beta service might be unavailable at times. Please report any bugs as comments on this post, or send a private email to the address shown on the profile page.

Thank you!

- Hessu and the bug-eating cats

Multiple targets on static maps

The static map page can now display up to 20 targets at the same time! Just enter the calls, separated by a comma. It works with wildcards, too, letting you select multiple matches simultaneously. For example:

RAUTAUOMA, OH2RDK, oh2rd*

The exact matches (rautauoma and oh2rdk) are preselected, and the wildcard matches are not.

Implementing this on the live map will be slightly more complicated, but unavoidable. :)

PS. Got my OT2 running with my Garmin Nüvi 350 today. Details will be posted later. It's very cool indeed.

Password recovery fixed

Oops, forgot to mention yesterday that the password recovery functionality was broken since Thursday, and was fixed in yesterday's update.

Repeater antenna installation day

For a change the photos in this blog post are not completely off-topic - we were installing new antennas on the OH2RCH repeater and igate site on Saturday. The operation was very well planned, all the required bits and pieces were there, the weather was excellent, and the execution was professional (like the components).

An FM broadcast antenna, which was used as the 145 MHz repeater transmitter antenna (together with a circulator and dummy load to handle the SWR), was removed and replaced with a proper 2-element stacked dipole for 145 MHz. Also, a 432 MHz repeater antenna with 4 stacked dipoles for RX and 2 dipoles for TX was added. And an extra 145 MHz dipole on the side of the mast. The 145 MHz RX antennas on the top of the mast, which are shared by the 2M repeater and the APRS igate, were not touched at this time. The new single dipole could be used for a transmitting igate to support 2-way messaging. There's another igate handling that in the area already, though.

The work happened at 70 meters above ground level, right above the second guy wire attachment point. Here is Antti, OH2MNI, attaching the 145 MHz dipole pair to the mast:



And here you can see yours truly, holding the braking rope which keeps the 2M antenna far from the mast while the antenna is being pulled up, so that it doesn't break all those microwave link antennas on the way up. On my right side is one of the guy "wires" - there are 6 of these on three sides of the mast, a total of 18.

And here, on the left side of the mast, are the completed antennas. On the top right side of the mast is a fancy new wind turbine, which is generating 48V DC to help power the commercial network equipment on the site. Green energy, global warming, you know the story. It was installed this summer, and it's already broken - the bearings are making so much noise that the neighbors are more than slightly annoyed.

Thanks go to: OH2LAK (the brains of the operation), OH2FDA, OH3GMZ, OH2MNI, OH2GLG, OH3GMZ and Markku V (the mast pro). More photos by OH2LAK and OH2MNI...

New bad GPS fix detector algorithm installed

I've just installed my new bad GPS fix detection algorithm. It should detect bad fixes about as well as before, but produce less false positives. The new algorithm looks at the previously received packets instead of the previously accepted packets, and is also slightly adaptive, taking into account more history than just the previous single accepted position.

It should work better for jets (traveling close to 1000 km/h), although during the takeoff acceleration some points might be dropped. After some initial test flights we'll be fixing that. :)

It should also better handle the case where the initial transmission happens to be somewhere far off. It seems like there are a bunch of stations which always wake up in Tokyo and then start transmitting their correct position in the US or Europe. Probably the GPS manufacturer has decided to show it's office location instead of the standard 0/0 lat/lon, and either does not indicate the bad fix in the NMEA sentence, or the tracker ignores that bit of information and transmits the bad position. These should now jump to the correct position after just a couple of packets.

The algorithm also ignores positions which were sent more than 2 hours ago, so if you take an intercontinental flight and start transmitting your new position immediately, it should just work!

Feedback is more than welcome!

Status and comment texts

As a little early morning exercise I've made aprs.fi show the status message in the info balloon of the current position on the real-time map, and also in the KML. Status message is shown in purple, and the comment text is shown in green.

There has been some confusion about these messages. There are three kinds of "status/comment" messages you can attach to your position. For example, SM4IVE-9 (info) is sending two of them.

The comment text is sent together with the position, in the end of the position packet. Here's an example packet with a comment text of www.sm4ive.com:

SM4IVE-9>APERXQ,WIDE2-2,qAR,SM5NRK-2:!5908.38N/01532.45E>000/000/A=000148www.sm4ive.com

The status message is sent as a separate packet which starts with a '>' character:

SM4IVE-9>APERXQ,WIDE2-2,qAR,LA6TMA-1:>{AT0B4}aprstracker-0.11-16f648

The Mic-E status message is encoded in a mic-e packet using just a few bits, and can contain one of these 8 standard messages: Off duty, En route, In service, Returning, Committed, Special, Priority, Emergency. 7 custom messages (Custom 0 to Custom 6) are also defined. All Mic-E packets contain this status message, and it only consumes a couple of bits in the message, so this requires the least bandwidth from the APRS channel. On the other hand, it can only express the few predefined values.

I would recommend using only the comment text, since it is sent in a single packet together with the position. The status message is sent in a separate packet which increases congestion.

If a status message is required (for example, if the text really needs to be so long that it doesn't fit in the comment text), the status message should not be sent too often. Certainly not as often as the position packet.

In the following photo Armi frowns upon seeing a long, static status packet:

USNG/MGRS grid and static map improvements

aprs.fi now has US National Grid support, it's selectable from Preferences -> Units and time -> Locator type. It took a surprisingly high amount of work to get this working somewhat right, and it still doesn't work in the polar areas. Each country has it's own national grid system, and many countries have more than one. I don't think I will be implementing any more of these, it would take a LOT of effort to support all of them.

I've improved the static map view a bit. From now on, the map will always be centered on the current location of the tracking target. The 'fit as many points as possible' centering algorithm of Google left a moving target outside the map view way too often, especially in the higher zoom settings. I also reduced the amount of zoom selections to make it quicker to navigate on mobile devices, and made the zoom selector show the currently selected value.

The user account password storage was improved, too. It now uses the RFC 2898 PBKDF2 algorithm to better protect the passwords against an offline dictionary attack. The computationally intensive hash algorithm intentionally makes it slower to perform the cryptographic operations required to check whether the given password is correct for the user. In case someone manages to break in the aprs.fi servers and steal the password database, it'll take a very long time for him to try all the dictionary words against the password hashes. This is basically a safeguard to protect those users who are stupid enough to use the same password on aprs.fi and other services (like their email accounts).

Label colour, messaging capabilities and case sensitivity

Today I installed an upgrade which contains a few small visual enhancements and a couple of bigger architectural changes under the hood. I hope the latter didn't break anything.

The callsign / object name labels shown on the map now have colours which partially match "APRS symbol attribute" colours in Bob Bruninga's original APRS software:

• White: APRS stations which are capable of messaging
• Gray: APRS stations which are not capable of messaging
• Violet: Objects
• Purple: Items

AIS stations have green labels and weather stations have yellow labels, although that conflicts with Bob's spec. I'm running out of colours and I don't have any idea which objects are sent by *you*. Hmm, well, maybe I could compare the source callsign with your aprs.fi login...

aprs.fi now updates the case of the callsign / target name to match exactly what is transmitted, even though the service is still case insensitive. So, if you're transmitting an APRS object named 'Oh7lzb' now, and you switch to a normal APRS station (not object) with the callsign 'OH7LZB', the service will now show the updated upper-case 'OH7LZB'. Before now, it retained the old mixed-case name. Both will refer to the same station and tracking history on aprs.fi, as before.

The path adviser on the info page now warns you about funny path elements like WIDE1-3.

The bigger under-the-hood changes make the web UI marginally quicker, easily expandable, and enable more fine-grained per-feature access control and site configuration.

Small changes (and a small cat)

During the weekend I replaced the date browsing menu with a new AJAX-based one. The downside is that the new one doesn't tell you how many positions were received during a year, month or day. The upside is that it's much, much faster than the old one. The old one was generated completely when you looked up a station on the map, which in turn required scanning the whole position history of the station, which caused a lot of disk IO, which filled up the caches. The new one just quickly checks whether there are any points during the year, month or day, and does it only when needed.

The new one only shows the days when new points have been inserted. For a fixed station like a digipeater it shows the initial date when the station was first heard by aprs.fi - the old one showed all of the days between the initial packet and the latest packet at the same position.

I also described the date selection menu in the FAQ. It turns out a lot of people don't know about it. Would some people like to try to write a beginner's manual?

The info page now shows the latest telemetry from the station instead of the number of telemetry packets stored. It's much more useful and much quicker to look up.

Most importantly - there is now a GIF animation in a couple of places! Honestly, I never thought I'd bother with such an useless thing which takes such an enormous amount of time to produce. But I accidentally figured out how Gimp does it, so I just had to try it once. Sorry!

In case you were wondering - a new kitten arrived on Saturday evening! The photo is maybe a bit off-topic, but it sure is cuter than the computers and radios.

New symbols, overlays and FAQ

I'm back from a couple weeks of vacation. Went to the north, visited the northernmost corner of EU and the northernmost corner of mainland Europe. It turned out that there's some APRS coverage in Magerøya and elsewhere in the northern Norway! None of the interesting animals shown in these travel photos tried to eat me.

I've just installed a new build of the software with a couple of features I've implemented yesterday evening and tonight.

I've regenerated the whole map symbol (icon) set from the Revision H APRS symbols provided by Stephen H. Smith, WA8LMF. The symbols shown by aprs.fi should now match those shown by findu.com, UI-View (with Stephen's symbol files) and other programs running with up-to-date symbols.

Thanks to Stephen for maintaining the symbol tables!

The downside is that the new symbols are a bit smaller than the old ones, and they suffer more from the artifacts caused by rotation (for course/heading display).

The upside is that I've also implemented symbol overlays! For example, OH2RDG now has the letter N overlaid on top of the solid green star, both on the map, in Google Earth, and in the info pages.

There's also a short FAQ page.

Arctic Sea position speculation

Quick recap: A cargo vessel called Arctic Sea (MMSI 215860000) was probably hijacked on July 24th 2009 near the eastern coast of Sweden. This was big news in northern Europe, since hijackings generally happen near the Somali coast, not over here. The ship has a Russian crew of 15, it appears to be owned by a Finnish company, and the owners of that company are of Russian origin. The Finnish media had considerable trouble trying to figure out the true owners, and the owners were really hard to interview. The ship deported from the harbor of Pietarsaari on 22nd of July and carries 6500 tons of Finnish timber, worth of about 1.3 MEUR.

The really odd thing is that the ship didn't go to the nearest Swedish port, but continued towards Africa as if nothing had happened. Very strange indeed. Either the hijackers were still on the ship, or the crew is taking part in the plot.

Latest news (Ransom demanded): BBC, CNN, YLE.

There have been a few questions about AIS positions of Arctic Sea shown on aprs.fi.

Q: Why is the track not shown for the moment of hijacking between Gotland and mainland Sweden?

A: There are no AIS receivers in the area which would directly send AIS reports to aprs.fi. These receivers are run by volunteers (thank you!), and each volunteer chooses where to submit AIS data. There is a receiver in the area, but it is submitting data to MarineTraffic only, and while MarineTraffic and aprs.fi exchange AIS data, aprs.fi is not getting the reports of all of those receivers. The Swedish maritime officials have an AIS receiver network of their own, and they've reported it ran circles and stopped for a while.

Q: Is the position shown for Saturday, 15th of August, valid?

A: Technically, it's possible, but I personally would find it very unlikely. It is easy to fake and it doesn't make any sense for the hijackers to publish their true position like this.

The position report was sent by an anonymous receiver station to MarineTraffic. It is quite easy to send fake data to MarineTraffic over the Internet, since they allow unauthenticated UDP packets containing NMEA strings to be sent to the service. aprs.fi does not allow unauthenticated UDP packets, all AIS submissions are tied to a specific receiving station using a password. Of course any one of those stations could feed us invalid positions, but at least we have some idea of the originator.

If the hijackers (or someone else) wanted to play tricks, they could also go to a shop selling marine radio equipment, buy an AIS transmitter, configure Arctic Sea's MMSI number (and other correct data) in it, give it an incorrect position by crafted NMEA strings (fake GPS receiver on the serial port of the AIS transmitter) and have it transmit the packets on the correct AIS frequency. If they've got the money and motivation to hijack ships with guns and speedboats, they've certainly got the guts to buy or steal AIS equipment. They could also grab the AIS transmitter from Arctic Sea, and take it to another position using a speedboat.

The French navy says there were 3 military vessels in the claimed position on the Bay of Biscay, heading for the Baltic sea, and they didn't see the hijacked ship. And they didn't see it on their radar, either.

The coast guard of Kap Verde claims to have seen the vessel about 800 km off the coast of Cape Verde, which is some 3600 km away from the Bay of Biscay.

In any case, this is starting to become a good plot for a movie.

Mic-E status message supported

I've just added support for Mic-E message bits. These are sent only in the Mic-E formatted packets, and I've been told it's quite easy to switch between the different messages in the Kenwood TM-D700 / TM-D710 rigs. I don't know how, since I don't own any of these rigs, and they're a bit expensive. If you're working for Kenwood, please consider sending me one for review and compatibility testing, there are over 130 000 potential APRS users visiting this site every month (absolute unique visitors according to Google Analytics). You can find my email address behind the 'complete profile' link on the right. Thank you. :)

The Mic-E status message is encoded in the packet using just a few bits, and can contain one of these 8 standard messages: Off duty, En route, In service, Returning, Committed, Special, Priority, Emergency. 7 custom messages (Custom 0 to Custom 6) are also defined.

The current message is shown on the info page of the sending station, and the info balloon of the current position on the real-time map. Old values of the message bits are currently not stored in the database.

Database upgrade, bug fixes and a feature

This morning I upgraded the master database engine. The upgrade stopped data updates for a few minutes, but the site was viewable during the upgrade.

I also installed new aprs.fi software on the servers. The new version adds support for the APRS equivalent of the email / newsgroups x-no-archive header, "!x!", which can be placed in the beginning of the comment text of a position packet. It prohibits aprs.fi (and findu) from archiving the position packet in the database. This is quite an old feature in findu, and has not really been documented anywhere (except the APRSSIG archives).

The !x! string is different from using NOGATE or RFONLY in the digipeater path. NOGATE/RFONLY prohibit gating the packet to the APRS-IS completely. !x! does not forbid igates from gating the packet on the Internet, it just requests that the packet is not stored in databases for a long time.

Please note that not all igates or databases support these features. Amateur radio transmissions are defined to be in the public domain (by FCC rules in the US, and by respective legislation in most other countries, and I suppose, by international regulations). Anyone can receive them, and retransmit, publish or store them as they wish.

As Steve Dimse put it on the APRSSIG: "if you do not want your data appearing on the internet the only guaranteed way is not to transmit it!"

I also added a search button to message and raw packets browsing views, and fixed a bug in clickable callsign links in digipeater paths. And did a rather massive source code tree reorganisation, making it a bit more tree-like.

Database upgrades

Today I upgraded the slave databases to a new major release of the database engine. I'd like to use some of the new features to increase the performance of the system.

The upgrade itself, and the necessary conversions, wouldn't have caused any outages, since I can tell my software to do queries on another database server while upgrading one (in fact, they will automatically fail over to another server if one is down). But, as usual, there was something I overlooked, and for a few minutes about 50% of the /info/ page requests failed and complained about problems with looking up nearby cities. I had to improve the stored procedures a bit to get them working on the new database engine version.

Please, drop a note in the blog comments, if you notice any other problems.

Account system upgrade completed

The planned upgrade has been now been done. It only took some 15-20 minutes to resolve the few issues that popped up. More details about the changes can be found in the previous blog post.

If you already have an account on aprs.fi for posting AIS data, and you don't have your login password (it's different from the AIS feeding password, and no, you didn't have one before since there was no such thing before today), try logging in with your email address and any bad password, and you'll get to the "forgot my password" path which lets you reset the password to a new one.

As usual, there are quite a few new strings to be translated. From now on, you'll have to sign up for an account and log in to access the translation tool. I hope you don't mind.

Planned upgrade: new account system

It's been a bit quiet here for some time, since I've been working on some larger changes in the aprs.fi back-end system (and learning how to fly, crash and repair an RC airplane). I'm planning to install the changes on the production servers early on Tuesday or Wednesday morning (7th or 8th of July 2009, around 4:00 UTC). The outage shouldn't be a long one, if all goes well.

One of the visible changes is a new authentication and authorization layer, which is based on a conventional email address + password model with a team-based authorization system. It'll enable some new features and services later (once I get around to implementing those) - in the first phase it just lets you sign up for submitting AIS data without manual work on my part.

There's no need to worry, as a normal user viewing data on aprs.fi, you won't need to sign up for an account. Logging in is completely optional, and I won't encourage or beg everyone and their cat to sign up. In fact, the "login with your nickname to see the map" model will go away - you'll end up directly on the map page when you arrive on the site, and the "most usual" site entrance path will be easier and quicker than before.

I will be adding an "introductory" entrance page for first-time visitors later, so that there's a chance to let them know what this site is all about. But it'll only be shown once for each visitor.

New simultaneous viewers record and related slowness

Seems like we hit a new high of over 1000 simultaneous map viewers today, mostly thanks to Dayton Hamfest, and a popular live Hamfest video feed with an embedded map. They're giving away freebies.

A couple of components started hitting file descriptor limits, which had last been upgraded over a year ago. Too many simultaneous connections per process. This made the site perform very, very slowly. I quadrupled the limits, and the site started to perform quickly again, I hope that's enough for more than a year to come. Well, I have to admit that it would actually be a nice surprise if the site would be so popular that it wouldn't be enough...

I also fixed a bug in the "first heard" algorithm pointed out by Ian, VK1IAN. The digipeater alias GATE was not treated as a special digipeater alias (like WIDE, RELAY and TRACE are), and an igate which first heard a packet with a GATE in the digi path was not given credit for hearing it first.

Another fix that went in was a filter which takes out complete APRS packets which have somehow made their way to the comment field of another APRS packet. Apparently something is loosing CR LF sequences between packets (could be my code...), which causes packets to go into the comment of the previous one. Before I find the actual bug I've added a filter to strip these off.

Dayton Hamvention APRS view

Dayton Hamvention is here again on next weekend. I've set up a separate page showing live APRS activity at Hamvention.

Bob Bruninga is expecting to have APRStt running there, too, so if you're visiting Hamvention, you might be able to place yourself on the map using just a DTMF-capable 2m transceiver.

I've also added a new parameter 'he_maptype' in the APRS map embedding interface which allows you to select satellite, physical or hybrid map views in addition to the normal street map.

Date range with weather and telemetry lookup

I'm on a 2-week sick leave (my gallbladder was removed last week), and have some spare time for enhancements and bug fixes. Rest of the time I'm entertaining Armi the cat, who joined us on Friday.

This morning I added arbitrary date range lookups to the weather and telemetry pages. It allows you to look up a detailed graph between any two dates or times. It also has a little calendar widget for your convenience.

I've also done some XHTML validity fixes for better browser compatibility, switched to a smaller position marker on the static maps, and added some navigation links in the bottom of most pages. I also fixed the bug of date selection menu not appearing when clicking on 'start tracking' on the live map.

Also, the /info/ page of a weather station now shows a brief 'latest weather' report instead of the number of weather reports in database. It's probably much more interesting, and it's much quicker to load, too, since it's almost always cached in memory and doesn't require a potentially large disk read.

Google Maps loading problem fixed

Google released Google Maps API version 2.151 today, which somehow broke map loading for many users (including myself). I received a number of complaints that the map doesn't load. The Firefox error console gave the error "window.jstiming is undefined".

I've now manually downgraded to Google Maps API version 2.150 which seems to work better.

Sorry for the trouble, and thank you for your patience!

Some database cleanups

Since the popularity of the site is increasing I need to do some optimizations to keep the site running smoothly.

As announced before, I ran some database restructuring batch jobs during the weekend. The last one was actually started on Monday morning and the execution was completed early Tuesday morning. I hope the slowdowns were not too noticeable, I tried to make them run slowly enough to not interfere with normal use.

I'm currently looking at ways to clean up some unused data from the aprs.fi database. There's quite a lot of automatically or erroneously generated targets stored in the database, most of which are not really looked at by anyone. They just inflate my tables and make the prefix browsing view very cluttered. The performance impact caused by the extra targets isn't really noticeable, but it's just not very nice to have them there.

If you're keeping an eye on the statistics, you might have noticed that today I deleted about 60000 targets. Those were all APRS objects, which had only a single point stored (non-moving), which were last announced over a week ago, and which were announced for less than 12 hours. Almost all of them were automatically generated earthquake and severe weather warning objects, which don't really need to be here any more.

There's also an awful lot of regular APRS targets which have a corrupted source callsign. Most usually one or two characters have been lost from the beginning of the callsign. For example: 1VAJ (J41VAJ), 3GXT-2 (W3GXT-2) and 3TVX-9 (VE3TVX-9). You can find these by looking at the prefix browsing view, picking a strange prefix starting with a number, and looking up the correct original call in the "nearby stations" list of the info page. I wonder how the callsigns get mangled like this, and how could I remove these from the database without accidentally removing some valid data.

Slowdowns caused by maintenance work

I'm doing some changes in the database structures during this Friday evening and Saturday morning, so there might be some slowdowns in the aprs.fi service. In particular, updated data might take a few minutes to be inserted in the database. No data should be lost, though, it'll just be delayed. Thank you for your patience!

APRS in русский язык

Thanks to UZ2HZ, RA1AMW, UA3IRS and UR7IMM, aprs.fi is now available in the Russian language! The site has now been translated to 20 languages. Although there's not so much APRS activity in Russia yet, Russian is the primary language of about 164 million people living in quite a few countries. Even here in Helsinki, it's the third most common language.

There are spots of APRS coverage in (at least) St. Petersburg, Moscow and Novgorod.

Single-target APRS packet paths

The APRS packet path plotting now works while tracking a single target. As a natural side-effect, all of the stations used as digipeaters and igates by the tracked target will also be shown on the map.

On Saturday I also made some significant changes in the backend code, which should make it scale out a bit better. I added some more in-memory caching which should speed things up so that the addition of things like APRS packet paths will not cause a significant performance hit.

"First heard" algorithm fix, and APRS-IS server failover

I've just fixed the "first heard by" algorithm to mark the igate as the first receiving station, when an intact WIDE1-1,WIDE2-1 path is seen. It was incorrectly guessing that the packet might have been digipeated because there is a WIDE2-1 in there, but the untouched WIDE1-1 clearly says that the packet has not been digipeated yet.

I've also added support for failing over to secondary APRS-IS servers automatically, so now I don't have to do anything manually if my own APRS-IS server fails (or is taken down for maintenance). This should prevent today's outage from happening again.

A little outage in APRS data collection

While I was at the hosting site adding memory to the second aprs.fi frontend today, I also did maintenance on the Sun Enterprise 4500 server which hosts finland.aprs2.net (the Tier 2 APRS-IS server which aprs.fi uses as the data source) and the aprs.fi backup database. So I configured the primary aprs.fi server to connect to another APRS-IS server while finland.aprs2.net was down, but made a mistake in the configuration, so the APRS-IS data collection broke. I didn't notice the alarms which it triggered, because I was working hard with the memory upgrade and the Sun reboot.

As a result, aprs.fi didn't collect APRS data between 11:35:43 UTC and 14:21:13 UTC. Sorry about that, I'll try to be more careful next time.

The memory upgrade itself went fine. The server should perform quite well for a while now.

Frontend memory upgrade on Sunday

I'll be upgrading the second aprs.fi frontend web server from 4 GB to 8 GB of RAM tomorrow. I'll do the installation some time during Sunday afternoon Finnish local time (that'll be Sunday morning in the US). I'll move it's IP address to the primary frontend server for the duration of the upgrade, so the service should be available all the time (unless something unexpected happens).

The upgrade should speed up the second frontend a bit (not that it would really have been slow before), and make it do the job for another year or so. It's been a gig or two on the swap before, but there hasn't been significant swap I/O happening normally, so it hasn't really been a problem. This is just a proactive upgrade to keep it performing as well as before, with the increased usage.

Multi-hop APRS path display

I've now improved the APRS packet path display to show multi-hop packet paths. The other limitations are still there: If one of the hops is well outside the current view, that hop will not be displayed on the packet path display.

Hrvatski APRS

Thanks to Gordan, 9A6AIV, aprs.fi is now available in the Croatian language! That brings us to 19 languages, which I find quite amazing.

APRS path lines

Quite a few of you have asked for some sort of visualization for APRS packet paths. I've just finished a partial implementation of it, and installed it on the production aprs.fi system.

It currently shows the path of a single packet, when you point an APRS station or a track point with the mouse cursor. In the screen shot on the right I had the mouse cursor pointing to the oldest waypoint of OH7FDN, and that position packet was relayed by the digipeater OH7RAA, and forwarded to the APRS-IS by the igate OH7AA.

There are a couple of limitations in this implementation:

1. It currently only shows the first digipeater found in the path, and the igate.
2. The digipeater and the igate must be visible on the map (or just outside the visible map area) - you might have to zoom out to make them visible.
3. The digipeater and the igate must be visible on the map - if you're tracking a single station (you've just searched for a callsign), you'll have to click on "stop tracking".
4. It currently only works in the real-time view - it doesn't work if you select a day in the past (because it only shows a single station in that mode).

I'll try to improve it in the future, and get rid of these limitations. This is just a start... feedback is welcome!

Features like this always come with a performance penalty. To provide the additional eye candy I'll have to pass on more data to the web browser. It will consume a bit more memory, and make the web browser slow down some more. I certainly hope it's worth it.

Deleting old PHG data

You can now delete your PHG circle by sending the PHG extensions with zero power, zero gain: 'PHG0000'. Some users have been testing PHG with their mobile station's callsign, and need to hide the old circle after finishing with the testing.

If you simply stop transmitting the PHG data, it will still be present in the aprs.fi database. There are a bunch of digipeaters using multiple different comment texts in a round-robin fashion, and some of them only transmit the PHG extension once per three position packets. So, if I'd delete the PHG data from the database whenever the PHG extension is missing, those digis would only be shown with a PHG circle 1/3rd of the time.

Magyar translation

Thanks to HA5CQZ (flicker) and HG1DFB, there's now a Hungarian translation of aprs.fi running! Thank you, guys!

Facebook APRS integration

Today I released the Facebook aprs.fi map application. If you're on Facebook, you can use the application to show the position of your APRS tracker / vehicle on your profile page. Yes, you need to sign up on Facebook to get a glimpse of it. Sorry about that.

The map is somewhat configurable, supports several different map types, zooming, different measurement units and coordinate formats. It runs on the same quick and scalable backend servers with aprs.fi.

There are some issues with the combination of Google Static Maps and Facebook, so it might stop working once it becomes a bit more popular. We'll see later.

I'm sure a lot of you will shake your heads in disbelief, and quietly comment something about the uselessness of this feature, but let's face it, Facebook is a big thing, and I just wanted to try it out, see what's going on in there, and figure out how things can be integrated with it.

aprs.fi also has a facebook page now. I don't know what it's good for, but at least you can mark yourself as a fan of the site to show your support. And maybe exchange some related photos, links and videos there, or use the discussion board. I'm sure it has a better discussion board feature than this blog's comments.

Slovene / slovenščina translation completed

Thanks to S53TK, S55M, S56G and hihi, there's now an almost complete Slovene version of aprs.fi! Thank you!

Technical details: what's behind the scenes

Some of you might be interested in knowing what aprs.fi runs on top of, so here's some technical information about the technology, design and implementation of my little APRS site.