Tuesday, December 25, 2007

Svenska, Japanese and French translations

OH6NT made a complete translation to Swedish (translated all 255 strings which are currently available for translation - help & option windows are not available yet)! JA7UDE and JO2VTM have translated most of the Japanese strings, and other languages have received updates too. I also installed the French translation, although it has only 35 strings translated so far.

These contributions really make me happy this Christmas. It's really nice to receive all this help. It takes a good day's work to properly translate the site to a language. Thank you, everyone.

Thursday, December 20, 2007

Polish and German translations installed

Thanks to SQ6NTI, there is now a partial translation to Polish (167 of 255 strings translated). HB9TQJ has translated 101 strings to German. HB3YKO has also started a French translation. Great work, thank you!

APRS parser module used by aprs.fi released

Here's a little Christmas present for the programmers out there:

Last night the APRS parser library used by aprs.fi was uploaded to CPAN. It's written in pure Perl, and it's reasonably complete, stable and fast. It's licensed under the same open terms as Perl itself (you pick: Artistic or GPL). It is written by Tapio Sokura, OH2KKU, with some additions by myself (weather packet parsing, machine-readable return messages, unit tests). Without this module, and the time Tapio took to write it in the first place, I wouldn't have gotten this far with aprs.fi. Parsing APRS packets is not easy.

It does miss some features (like telemetry parsing, and a few other smaller things), but I believe it to be the best parser currently published as open source. It validates input packets in a rather strict way and actually tells you what is wrong with each rejected packet. It does not crash when it sees something unexpected or corrupted. It returns numeric data in SI units. It's good enough to parse the APRS-IS feed in real time on any modern computer. It might even run on Windows with Perl (cygwin?) installed.

To install it on an Unix-like system (like Linux) with Perl using CPAN (as root):

root@proge:~#> perl -MCPAN -e shell
... if you're running the CPAN client for the first time, answer the questions, and once you're in the CPAN shell: ...
cpan> install Ham::APRS::FAP

You can also download the Ham::APRS::FAP module and browse it's contents on the CPAN web site. Installation instructions can be found in the README file.

Oh yeah. FAP stands for Finnish APRS Parser (or Fabulous APRS Parser). Ham::APRS::Parser was taken.

Wednesday, December 19, 2007

Dutch translation

Thanks to PG1G and PA3BWE, aprs.fi now speaks Nederlands (Dutch)! Thanks guys, that was really quick.

The translation seems quite complete, only a couple of the really long strings are missing. And the Options and Help windows, which are not (yet) available in the translation tool. But this is, in my humble opinion, very cool. Japanese, anyone?

The language shown to the user is selected by the preferred languages list of the web browser (Firefox: Edit -> Preferences -> Advanced -> General -> Languages -> Choose, IE: Tools -> Internet Options -> General -> Languages). The default setting of a browser usually follows the operating system's language settings.

Saturday, December 15, 2007

A little community effort: Translating aprs.fi!

A certain percentage of non-English speaking users seem to have some trouble using the site. The web site's statistics show that people who prefer viewing pages in English usually stay on the site longer. Even many users who speak English well would prefer to view the site in their native language. It is now possible for anyone to participate in a little community effort and translate the site to their own language.

So, if you speak some other language than English or Finnish fluently, and if you'd like to help your fellow hams a bit, please take a look at the brand new aprs.fi Translation Tool! Please read the instructions carefully.

This is a brand new application, and hasn't been widely tested before now. Some bugs will probably be found soon, but I'll try to get them fixed quickly! Feedback is welcome, post a comment to this blog post or send an email to the address listed in my blogger profile. Thank you!

Friday, November 30, 2007

AIS from Pietarsaari, and tracking links fixed on MSIE

OH6AG kindly started to provide AIS reception from Pietarsaari. Their receiver hears traffic from the harbors of Pietarsaari and Kokkola, and most of ships from Kvarken (Merenkurkku).

I also did a small but important fix on the map javascript code - the "start tracking" and "stop tracking" links now also work on Internet Explorer 6. For some reason the quite correct way of assigning an onclick event to a span did not work on that single browser - it did work on every other browser (including IE7). And it only failed to work in the innerHTML dynamically generated by the javascript for the target info pop-up balloons - it works just fine in, for example, the Options link on the right side of the map (which is dynamically generated on the server). So I put in a <a href="javascript:function()"> tag, which seems to work on at least Firefox, IE6/7 and Safari. Please let me know if this change broke some other browser.

Wednesday, November 28, 2007

New AIS sites, and lack of other news

New AIS receiving sites have been added in Thunder Bay, Lake Superior, Ontario, Canada (thanks to Robert, VA3ROM) and Felixstow, Suffolk coast, UK (thanks go to Ian, G4ZJH).

There's not much else to report on. I've written a lot of code for the new localisation tool, and it should be ready for prime time pretty soon.

Saturday, November 10, 2007

APRS parsing logic changes

Today I did two changes which were recently requested by aprs.fi users.

The first one was actually done last weekend, but it needed some more testing and cleaning up, so I only deployed it on the production server today. The MIC-E parser now accepts some forms of corrupted packets. There are a number of broken igates and digipeaters out there, which corrupt MIC-E packets which often contain non-printable, non-ASCII characters in the 0x1C-0x1F range. At least aprsd has this bug (there is a patch in it's bug tracking system which fixes it) - it replaces the characters with spaces (0x20), and it seems that some igates or APRS-IS servers then replace the spaces with a single space, so that the MIC-E data is really broken.

I've added a kludge which manages to parse the location from some of these corrupted packets. It's ugly, but it usually works. Replacing corrupted or missing data with something I've guessed is not a good idea, but this will have to do for a while.

The second one was pointed out only last night by Chris, K6DBG. The speed of a target is now calculated from the positions of the current packet and the previously transmitted packet, instead of the previously accepted and stored packet. This is significant, because packets indicating that the target is moving faster than 500 km/h are discarded. These are usually caused by corrupted packets, or bad GPS data. Some APRS trackers transmit their position even if the GPS clearly indicates that it has no valid fix. Sometimes the GPS suddenly gives out a single invalid point which is hundreds or thousands of kilometers away.

If the invalid position was the first one transmitted by the station, it was accepted and stored to the database, and it could take days (to move to the true position at 500 km/h) before a new correct position packet was accepted. Now, the second correct position packet is already accepted and stored.

Tuesday, November 6, 2007

Weather graphs are back

I managed to extract all data from the corrupted database, except for the weather history data from a single weather station (MMUN in Mexico), so I didn't need to use old backups. The weather history is currently being loaded in the running database. I started the load just after midnight (after getting home from the Kent concert), it's now 2 PM and 74970000 of 106248685 rows have been loaded already, so the load should be completed this evening. That's right, there are currently 106 million weather reports stored from 8728 weather stations.

Then database load is a bit slow, since I made the loading script sleep most of the time - it loads the reports of a single weather station, and then sleeps three times the time it took to perform the previous load. This prevents it from slowing down the database too much, and interfering the normal operation of the aprs.fi site. Once it's finished, the restore process is completed.

Monday, November 5, 2007

aprs.fi is running again

I've now finished loading the backup database dumps, recovering all data before the crash last night, except for the weather history. It seems I'll be able to restore almost all of the weather data too (all of 6 months, except for a small minority of weather stations), but it'll have to wait until tomorrow. The recovery took all of today, since there simply is a lot of data, and I didn't have a hot replica of the database - the recovery had to be done using text dumps of the database. The operations took hours each, so I had good time to go out for a walk, watch a movie, and take part in a local HF contest while waiting for each step to finish. :)

I'll give replication a higher priority, this wasn't much fun as such.

Sunday, November 4, 2007

aprs.fi down due to storage problems

Last night the EMC storage system (a half-rack full of hard disks, connected to the database server with Fibre Channel) where the aprs.fi database is stored, has returned some corrupted data, and the ZFS filesystem has detected checksum errors. As a result, the database is now down, and I'm restoring a backup copy. This takes some time, as the database is rather large. The service should be back up later today.

This sucks. Seems I really need to set up online data replication to another database machine. As long as the database is down, no APRS-IS data is stored.

Sorry for the trouble!

Sunday, October 28, 2007

New view: moving targets

Just a few minutes ago I published links to a new page: http://aprs.fi/moving/ shows APRS and AIS targets which have moved within the last 5 minutes. It's an easy way to find a "live" tracking target. And some drifting stations, which in reality are very much fixed and non-movable.

Today I also did some work trying to speed up the generation of the info pages. They do a lot of database lookups and feel a little bit slow. Managed to optimize the nearest cities lookup by about 200 milliseconds, which is very much noticeable. It probably cannot be optimized much more (other than by adding faster hardware, or by removing information from the page).

Also had a few contacts in the CQ WW SSB contest, and submitted my JARTS RTTY contest log. And visited the OH2RCH site in Juhanila, powered up the igate Linux PC which had been down for a week.

Thursday, October 25, 2007

aprs.he.fi and gaprs.net are now aprs.fi

A week ago Justin McAllister (K5EM) emailed me asking if I had any interest in taking over the gaprs.net domain, since lately he's been forwarding users with feature requests to aprs.he.fi anyway. The ownership of the domain has now been transferred and pointed to my site (although it might take until tomorrow before all DNS caches have obtained the new IP address). Thank you for the support and donation, Justin!

To make the site's URL even shorter and easier to remember, I've also obtained aprs.fi and pointed aprs.he.fi to it. That's only 7 characters to spell and remember. Old links will continue to work for the foreseeable future. The downside is that browser cookies don't work over domains, so you'll need to log in and set up the options again. Sorry about that!

So: the new "official" link is aprs.fi.

Gaprs.net and aprs.he.fi links will continue to work as before.

As I am writing this, the site is running on batteries (they're testing the UPS batteries, and after that, the diesel generator).

Monday, October 22, 2007

Character set fixes and invisible enhancements

Today I fixed most of the 文字化け issues. 乱码, 亂碼, маймуница and крокозя́бры should be gone, too. If you don't know what that means, they didn't affect you. If you still want to know, look it up on Google. 8)

No, you cannot search Japanese callsigns from aprs.he.fi, but at least trying to do so will now give a nice error message instead of letting the database say "no, you cannot compare apples with oranges" (in technical terms, of course).

Over the weekend and most of last week I also did a lot of fixing and restructuring which is not too visible right now, but supports wider localization of the service. Awful lot of rather boring and mechanical work, moving a lot of code and strings around, testing, fixing, testing, then moving some more. But it needed to be done eventually.

Localization issues are a new interesting frontier for me. Remembering the times when Finnish special characters (åäö) were rendered incorrectly more often than not, it hasn't yet stopped to amaze me how those Asian characters can "just work" on modern Linux systems, and how my web apps can be fixed to support them with so little work.

Thursday, October 18, 2007

Ships in Åbo and the ARBONET-II balloon launch

Thanks to www.aboais.net, we now have an AIS feed from Turku, showing the busy traffic of the Archipelago of Finland (Saaristomeri). Thank you, Petri!

Mark, N4CMB reports that the ARBONET team is launching a high-altitude balloon called ARBONET-II from JD Trissell Airport in Clarksville, Texas at 14:30 UTC (9:30 AM CDT) on Saturday, October 20th. The balloon will carry an APRS tracker operating on 144.390 as K5FRC-11 (aprs.he.fi tracking), a 70 cm up / 2 meter down cross band FM repeater, a 10 meter CW beacon and a plain language homing beacon that will transmit navigation data in plain English on the 2 meter band. There are cameras and environmental sensors too, so there should be some fine coverage of the event afterwards!

Monday, October 15, 2007

New web frontend server

The main aprs.he.fi server was feeling a bit sluggish - it was starting to run out of memory, and it did some swapping. This happened simply because there are so many people looking at the maps at the same time, and fetching real-time updates from the server at a rather high constant rate. So today I separated the web server to another server, and left the database running on the old server computer. This seems to help the situation quite a lot, and did not require a lot of work.

There was a very short outage caused by the change - by accident the new frontend server did not serve your requests for a couple minutes, since I had accidentally forgotten an access list in it's configuration. It only worked for me! Sorry about that.

Maybe in the future I'll try to put together a couple of dedicated frontend servers and start load-sharing web requests between them. Adding a slave database server would make the HA setup almost completely fail-safe. The software I've written seems to support scaling to a cluster with multiple servers just fine. I had hoped it would, since it was designed to, but I didn't try it until now.

Sunday, October 14, 2007

Finnish translation and more graphs

I spent almost all of saturday with my favorite text editor joe, working on these two gems.

You can now click on show graphs in the info view of a station, and get some plots on the activity of the station. The number of graphs will depend on the type of the station - if it is an igate, a digipeater, or a moving vehicle. For example, take a look at
http://aprs.he.fi/info/graphs/OH2RCH (a good local rx-only igate)!

While doing this, I enhanced the graph plotting code to do bar graphs, and to scale X better than before (filling the whole range). And added URL watermarks. These enhancements benefit the weather graphs, too.

The other big item was translating the user interface almost completely to Finnish. Language selection is based on your browser preferences (Explorer: Tools / Internet Options / Appearance: Languages, Firefox: Edit / Preferences / Advanced / General / Languages: Choose). These browser settings define the default language of most well-behaving international sites.

The weather page has mobile templates now, too, so it should look better on your phone.

Thursday, October 11, 2007

AIS from Trondheim, and lost raw packets

We now have AIS from Trondheim, Norway, thanks to Tomas Roald!

I just installed an improved database schema for raw packets, making callsign lookups for raw packets much quicker. Storing twice as much as data as before (actually storing everything twice), but retrieval is over twice as fast. I deleted the two days of history, I hope you won't mind (after 48 hours, you shouldn't).

Tuesday, October 2, 2007

APRS in a rally car

Kris (KI6IUC) reports that they'll have APRS tracking in their rally car while driving the Prescott Rally in Prescott, AZ, USA on Friday and Saturday this weekend. Cool!

Sunday, September 30, 2007

Some more AIS from Norway!

We now have AIS feeds from Aukra, Norway (thanks to LA2PJ) and Harstad, Norway (thanks to LA2MV). Great, keep them coming!

Today I coughed up a list of AIS map positions.

Wednesday, September 26, 2007

Dew points and more AIS

This morning I added dew point calculation to the weather pages and temperature graphs. It was really easy. I used the Magnus approximation referenced by the Wikipedia article.

Thanks to Kai Gunter, LA3QMA, we now have AIS data from Bergen, Norway. Most excellent! If you're running ShipPlotter somewhere and would like to share your data with us, please drop me an email (the address is in my Blogger profile).

Now I'll head towards downtown, to hear Dream Theater play live in Helsinki! Tomi OH2BNS, the author of gMFSK, is coming too.

Tuesday, September 25, 2007

Weather of area in view + bug fixes

On Monday morning I quickly fixed Konqueror/KHTML support. This might help with some older versions of the Safari browser, too (in case they had any problems - I don't know if they did).

Last night I added a small local weather info line on the right side of the map. It shows an average current weather report from the weather stations within (and near) the current map view. Before calculating the averages, some of the most extreme samples are removed - if there are 6 weather stations reporting the temperature as 0, 5, 6, 7, 7, and 15 degrees, the average will be avg(5,6,7,7) = 6.3 degrees. There are some stations sending rubbish out there.

Weather history graphs can now show temperatures in degrees Fahrenheit. Some other small weather related bugs were also fixed.

Sunday, September 23, 2007

Geographic coordinate formatting

It is now possible to select how coordinates are displayed on the site. Three formats are available:
  • Decimal:
    Degrees in decimal: 25.1235° N
  • Deg°Min'Sec":
    Degrees, minutes and seconds as integer values (DMS): 25°51'11" N
  • APRS format:
    Integer degrees with minutes in decimal: 25°51.03' N
The selection can be made in the Options window, as usual. The new default value is Deg°Min'Sec".

Friday, September 14, 2007

Mobile browser support and xhtml

Some web browsers embedded in mobile phones should now work with aprs.he.fi to some degree. I've only tested it on Nokia E61i (maps work!) and N70 (no javascript + ajax => no maps), so reports for other browsers (iPhones, Windows Mobile) are welcome. Just drop a comment to this post, please.

The live APRS maps require AJAX support (XMLHttpRequest javascript API + DOM). For Symbian-based phones (Nokia smart phones, and some others), only the very latest models, which are based on S60 3rd Edition, have this functionality. E61i, E65, E90, N95, etc. Other phones can only browse the "normal" info pages.

The work is not complete, but some of the content has been converted to templates. There are different content templates for mobile browsers (ones with tiny screens) and "normal" browsers, and separate language template files which contain all of the English strings. So, if there is a typo or bad grammar somewhere, it only needs to be fixed once. And the layout can be tuned separately for each browser type without actually touching the strings or the application code. Maybe I'll add translations to other languages some day - it would be rather trivial (although time-consuming).

Beware - browsing the maps consume a lot of bandwidth. Depending on your carrier, you might end up paying a lot for just a little browsing. Be sure to check how much you're charged per megabyte.

While working on this I converted some of the site to mostly proper strict xhtml syntax. This changes how other browsers render things, so some new bugs might have crept in. This is what broke the weather pages the other day.

PS. The E61i browser is crappy - it hangs, dies or runs out of memory ("please close some applications") after just a little browsing around in the real-time map. Think I'll have to check what Opera has to offer.

Friday, September 7, 2007

Live APRS maps on your own web page!

It's now possible to put APRS maps on your own web page. It's free, and it's easy. Show the current location of your car, airplane, boat, motorcycle or ship, or all the APRS activity in your town. Just copy and paste the following HTML code on your page (replace OH2RCH with your own APRS callsign):

<p>
<script type="text/javascript">
he_track = "OH2RCH"; // track this callsign
</script>
<script type="text/javascript" src="http://aprs.he.fi/js/embed.js">
</script>
</p>

It's also possible to start the embedded map at certain coordinates (showing all APRS targets in the area, or limiting shown targets like in the Options of aprs.he.fi). The map updates automatically (every few seconds, just like the real thing) without need for the user to reload the page to get new data. Read more detailed instructions on embedding APRS maps on your web page!


Thursday, August 30, 2007

APRS balloon flights this weekend

Mark, N4CMB reports in a blog comment that members of the North Texas Balloon Project are launching flight 15 on Saturday, September 1, 2007 at about 13:30 UTC (8:30 am CDT) from the Hillsboro Municipal Airport (32.0835N 97.0971W). The balloon's APRS tracker callsign is W5SJZ-11. It will carry beacons and a crossband FM repeater. Thanks for the note, Mark!

Ilmari-2007b will fly on Sunday, September 2, 2007 at about 06:00 UTC (09:00 EET+DST) from Sappee, KP21GJ. It's APRS tracker callsign is OH2SIX-11. In addition to the beacons Ilmari carries a crossband linear transponder (much like a satellite: it receives on 144 MHz and transmits on 437 MHz) and an 1282 MHz FM-ATV video transmitter.

Update (Monday 3rd of September): Ilmari 2007b was postponed due to bad weather, but NTBP flight 15 did fly on saturday.

Happy tracking, everyone!

Sunday, August 26, 2007

Searching APRS and AIS targets using wildcards

I've finally added wildcard character support for searching stations in the map and info views. A wildcard search in either will return a list of stations (together with their symbols, report ages and comment strings). Clicking on a station in the list will jump to the map or info view of the given station. To limit load on the server, a maximum of 300 stations will be shown.

Supported wildcard characters are ? (any single character) and * (any number of characters).
After successfully looking up a callsign in either the info or map views, links will be shown to other SSIDs of the same callsign (if there are any).

Wednesday, August 22, 2007

10 MHz HF APRS in Namibia

Mike OE3MZC, who happens to be the president of the Austrian Amateur Radio Society, is staying in Namibia as V51/OE3MZC, and driving around in a 4-wheel-drive car featuring a 10 MHz HF APRS tracker. His position reports are received by igates in South Africa (1500 km away) and Australia (11000 km, or 7000 miles away)!
I suddenly feel like setting up an 10 MHz igate somewhere.

(found this on the South African Radio League web forum.)

Monday, August 20, 2007

Gliders in Sweden

APRS trackers have been installed in a bunch of gliders (or sailplanes) in Sweden. They seem to fly around Eskilstuna, and belong to Eskilstuna Flygklubb. Their APRS callsigns follow the callsigns of the airplanes, with the "-" removed (APRS callsign SEUUU belongs to the SE-UUU aircraft). I wonder what frequency they're using, and if they're doing this with an amateur license... but it's very cool indeed.
And the list goes on...

I'm sure a good busy flying day looks very cool on Google Earth.

AIS ship tracking

AIS, or Automatic Identification System is not much unlike APRS - it is a professional tracking system for ships. It runs at 9600 bit/s (GMSK) on two marine VHF frequencies, with a nice binary protocol which features automatic synchronization and time multiplexing - even with a very high number of vessels transmitting position reports once every 2 to 10 seconds, packet collisions will not be a problem. The protocol is standardized and well-defined, but unfortunately patented.

Fortunately the protocol is unencrypted and available. There is a Windows application called ShipPlotter which can decode AIS traffic and export the received position reports using HTTP. If you live somewhere near a coast, set up a 160 MHz FM receiver with a discriminator output or data port (you can't properly decode 9600 bit/s from the audio output of most FM receivers), plug it in the sound card of your PC, and you can track ships from miles away. A single receiver on the south coast of Finland can hear ships from Tallinn and a good part of the Gulf of Finland.

Now, aprs.he.fi can receive the HTTP position report exports sent by ShipPlotter. If you run it somewhere on a 24/7 basis, please get in touch with me and share the view with others! There's currently one receiver feeding us with AIS traffic from Helsinki, Gulf of Finland (thanks Tapio!).

To see some AIS traffic, zoom to the area between Helsinki, Finland and Tallinn, Estonia and make sure you have enabled viewing AIS data in the Options!

High-altitude balloons

There are groups of hams flying weather balloons stuffed with APRS trackers, digital cameras, repeaters and linear transponders, and other cool stuff. These things are cheaper to build and fly than self-built satellites, but offer a lot of the same features and thrill. I'm aware of two projects which flew during the past couple weeks.

Ilmari-2007 (OH2SIX-12) flew in Finland on 11th of August 2007 and is planned to fly again on 2nd of September. It's equipped with a linear transponder (70cm to 2M, just like amateur satellites - actually the device was designed for a future satellite), a couple of beacons, an APRS tracker transmitting on 437 MHz, a couple of temperature sensors, and an 1.2 GHz Amateur Television (ATV) transmitter sending live video from the heights! The first flight went up to about 30 km (100 000 feet). Here are the links (sorry for the login requirement):

- The track of the first Ilmari-2007 flight
- Current location of Ilmari-2007 - use this link on 2nd of September to follow the flight in real time. The flight is scheduled at 06:00 UTC (but they usually fly late).

SABLE-3 (VA6TNY-11) also flew on 11th of August, in South Alberta, Canada. It was equipped with a 8-megapixel digital still camera taking crisp pictures once every minute, and an APRS tracker. It flew up to 117,597 feet (over 35 km), and took some really nice photos!

- The track of the SABLE-3 flight on 11th of August, 2007

These balloon flights took aprs.he.fi usage to an all-time high - there were over 100 simultaenous users tracking the balloons alone.

If you are planning to launch a balloon, please share the fun and drop me an email, or post a comment on this blog. Thanks!

My first little blog post.

Hi, and welcome to my very first blog, and it's very first posting. I intend to use this space to ramble about new features and bugs found at http://aprs.he.fi/ - my little APRS web database project, which got a little out of hand. I don't feel like implementing a nice news feature on the site itself right now, so I'll just use Blogger for now!

I'll also try to drop links to interesting locations and attractions found in the APRS world. Feel free to mail me if you spot something interesting. Thanks!

- Hessu, OH7LZB