I've just installed my new bad GPS fix detection algorithm. It should detect bad fixes about as well as before, but produce less false positives. The new algorithm looks at the previously received packets instead of the previously accepted packets, and is also slightly adaptive, taking into account more history than just the previous single accepted position.
It should work better for jets (traveling close to 1000 km/h), although during the takeoff acceleration some points might be dropped. After some initial test flights we'll be fixing that. :)
It should also better handle the case where the initial transmission happens to be somewhere far off. It seems like there are a bunch of stations which always wake up in Tokyo and then start transmitting their correct position in the US or Europe. Probably the GPS manufacturer has decided to show it's office location instead of the standard 0/0 lat/lon, and either does not indicate the bad fix in the NMEA sentence, or the tracker ignores that bit of information and transmits the bad position. These should now jump to the correct position after just a couple of packets.
The algorithm also ignores positions which were sent more than 2 hours ago, so if you take an intercontinental flight and start transmitting your new position immediately, it should just work!
Feedback is more than welcome!
The news of https://aprs.fi/ - new features and interesting attractions found in the APRS and AIS worlds.
Monday, October 12, 2009
Thursday, October 8, 2009
Status and comment texts
As a little early morning exercise I've made aprs.fi show the status message in the info balloon of the current position on the real-time map, and also in the KML. Status message is shown in purple, and the comment text is shown in green.
There has been some confusion about these messages. There are three kinds of "status/comment" messages you can attach to your position. For example, SM4IVE-9 (info) is sending two of them.
The comment text is sent together with the position, in the end of the position packet. Here's an example packet with a comment text of www.sm4ive.com:
SM4IVE-9>APERXQ,WIDE2-2,qAR,SM5NRK-2:!5908.38N/01532.45E>000/000/A=000148www.sm4ive.com
The status message is sent as a separate packet which starts with a '>' character:
SM4IVE-9>APERXQ,WIDE2-2,qAR,LA6TMA-1:>{AT0B4}aprstracker-0.11-16f648
The Mic-E status message is encoded in a mic-e packet using just a few bits, and can contain one of these 8 standard messages: Off duty, En route, In service, Returning, Committed, Special, Priority, Emergency. 7 custom messages (Custom 0 to Custom 6) are also defined. All Mic-E packets contain this status message, and it only consumes a couple of bits in the message, so this requires the least bandwidth from the APRS channel. On the other hand, it can only express the few predefined values.
I would recommend using only the comment text, since it is sent in a single packet together with the position. The status message is sent in a separate packet which increases congestion.
If a status message is required (for example, if the text really needs to be so long that it doesn't fit in the comment text), the status message should not be sent too often. Certainly not as often as the position packet.
In the following photo Armi frowns upon seeing a long, static status packet:
There has been some confusion about these messages. There are three kinds of "status/comment" messages you can attach to your position. For example, SM4IVE-9 (info) is sending two of them.
The comment text is sent together with the position, in the end of the position packet. Here's an example packet with a comment text of www.sm4ive.com:
SM4IVE-9>APERXQ,WIDE2-2,qAR,SM5NRK-2:!5908.38N/01532.45E>000/000/A=000148www.sm4ive.com
The status message is sent as a separate packet which starts with a '>' character:
SM4IVE-9>APERXQ,WIDE2-2,qAR,LA6TMA-1:>{AT0B4}aprstracker-0.11-16f648
The Mic-E status message is encoded in a mic-e packet using just a few bits, and can contain one of these 8 standard messages: Off duty, En route, In service, Returning, Committed, Special, Priority, Emergency. 7 custom messages (Custom 0 to Custom 6) are also defined. All Mic-E packets contain this status message, and it only consumes a couple of bits in the message, so this requires the least bandwidth from the APRS channel. On the other hand, it can only express the few predefined values.
I would recommend using only the comment text, since it is sent in a single packet together with the position. The status message is sent in a separate packet which increases congestion.
If a status message is required (for example, if the text really needs to be so long that it doesn't fit in the comment text), the status message should not be sent too often. Certainly not as often as the position packet.
In the following photo Armi frowns upon seeing a long, static status packet:
Tuesday, September 29, 2009
USNG/MGRS grid and static map improvements
aprs.fi now has US National Grid support, it's selectable from Preferences -> Units and time -> Locator type. It took a surprisingly high amount of work to get this working somewhat right, and it still doesn't work in the polar areas. Each country has it's own national grid system, and many countries have more than one. I don't think I will be implementing any more of these, it would take a LOT of effort to support all of them.
I've improved the static map view a bit. From now on, the map will always be centered on the current location of the tracking target. The 'fit as many points as possible' centering algorithm of Google left a moving target outside the map view way too often, especially in the higher zoom settings. I also reduced the amount of zoom selections to make it quicker to navigate on mobile devices, and made the zoom selector show the currently selected value.
The user account password storage was improved, too. It now uses the RFC 2898 PBKDF2 algorithm to better protect the passwords against an offline dictionary attack. The computationally intensive hash algorithm intentionally makes it slower to perform the cryptographic operations required to check whether the given password is correct for the user. In case someone manages to break in the aprs.fi servers and steal the password database, it'll take a very long time for him to try all the dictionary words against the password hashes. This is basically a safeguard to protect those users who are stupid enough to use the same password on aprs.fi and other services (like their email accounts).
I've improved the static map view a bit. From now on, the map will always be centered on the current location of the tracking target. The 'fit as many points as possible' centering algorithm of Google left a moving target outside the map view way too often, especially in the higher zoom settings. I also reduced the amount of zoom selections to make it quicker to navigate on mobile devices, and made the zoom selector show the currently selected value.
The user account password storage was improved, too. It now uses the RFC 2898 PBKDF2 algorithm to better protect the passwords against an offline dictionary attack. The computationally intensive hash algorithm intentionally makes it slower to perform the cryptographic operations required to check whether the given password is correct for the user. In case someone manages to break in the aprs.fi servers and steal the password database, it'll take a very long time for him to try all the dictionary words against the password hashes. This is basically a safeguard to protect those users who are stupid enough to use the same password on aprs.fi and other services (like their email accounts).
Monday, August 31, 2009
Label colour, messaging capabilities and case sensitivity
Today I installed an upgrade which contains a few small visual enhancements and a couple of bigger architectural changes under the hood. I hope the latter didn't break anything.
The callsign / object name labels shown on the map now have colours which partially match "APRS symbol attribute" colours in Bob Bruninga's original APRS software:
aprs.fi now updates the case of the callsign / target name to match exactly what is transmitted, even though the service is still case insensitive. So, if you're transmitting an APRS object named 'Oh7lzb' now, and you switch to a normal APRS station (not object) with the callsign 'OH7LZB', the service will now show the updated upper-case 'OH7LZB'. Before now, it retained the old mixed-case name. Both will refer to the same station and tracking history on aprs.fi, as before.
The path adviser on the info page now warns you about funny path elements like WIDE1-3.
The bigger under-the-hood changes make the web UI marginally quicker, easily expandable, and enable more fine-grained per-feature access control and site configuration.
The callsign / object name labels shown on the map now have colours which partially match "APRS symbol attribute" colours in Bob Bruninga's original APRS software:
- White: APRS stations which are capable of messaging
- Gray: APRS stations which are not capable of messaging
- Violet: Objects
- Purple: Items
aprs.fi now updates the case of the callsign / target name to match exactly what is transmitted, even though the service is still case insensitive. So, if you're transmitting an APRS object named 'Oh7lzb' now, and you switch to a normal APRS station (not object) with the callsign 'OH7LZB', the service will now show the updated upper-case 'OH7LZB'. Before now, it retained the old mixed-case name. Both will refer to the same station and tracking history on aprs.fi, as before.
The path adviser on the info page now warns you about funny path elements like WIDE1-3.
The bigger under-the-hood changes make the web UI marginally quicker, easily expandable, and enable more fine-grained per-feature access control and site configuration.
Wednesday, August 26, 2009
Small changes (and a small cat)
During the weekend I replaced the date browsing menu with a new AJAX-based one. The downside is that the new one doesn't tell you how many positions were received during a year, month or day. The upside is that it's much, much faster than the old one. The old one was generated completely when you looked up a station on the map, which in turn required scanning the whole position history of the station, which caused a lot of disk IO, which filled up the caches. The new one just quickly checks whether there are any points during the year, month or day, and does it only when needed.
The new one only shows the days when new points have been inserted. For a fixed station like a digipeater it shows the initial date when the station was first heard by aprs.fi - the old one showed all of the days between the initial packet and the latest packet at the same position.
I also described the date selection menu in the FAQ. It turns out a lot of people don't know about it. Would some people like to try to write a beginner's manual?
The info page now shows the latest telemetry from the station instead of the number of telemetry packets stored. It's much more useful and much quicker to look up.
Most importantly - there is now a GIF animation in a couple of places! Honestly, I never thought I'd bother with such an useless thing which takes such an enormous amount of time to produce. But I accidentally figured out how Gimp does it, so I just had to try it once. Sorry!
In case you were wondering - a new kitten arrived on Saturday evening! The photo is maybe a bit off-topic, but it sure is cuter than the computers and radios.
The new one only shows the days when new points have been inserted. For a fixed station like a digipeater it shows the initial date when the station was first heard by aprs.fi - the old one showed all of the days between the initial packet and the latest packet at the same position.I also described the date selection menu in the FAQ. It turns out a lot of people don't know about it. Would some people like to try to write a beginner's manual?
The info page now shows the latest telemetry from the station instead of the number of telemetry packets stored. It's much more useful and much quicker to look up.
Most importantly - there is now a GIF animation in a couple of places! Honestly, I never thought I'd bother with such an useless thing which takes such an enormous amount of time to produce. But I accidentally figured out how Gimp does it, so I just had to try it once. Sorry!
In case you were wondering - a new kitten arrived on Saturday evening! The photo is maybe a bit off-topic, but it sure is cuter than the computers and radios.
Tuesday, August 18, 2009
New symbols, overlays and FAQ
I'm back from a couple weeks of vacation. Went to the north, visited the northernmost corner of EU and the northernmost corner of mainland Europe. It turned out that there's some APRS coverage in Magerøya and elsewhere in the northern Norway! None of the interesting animals shown in these travel photos tried to eat me.I've just installed a new build of the software with a couple of features I've implemented yesterday evening and tonight.
I've regenerated the whole map symbol (icon) set from the Revision H APRS symbols provided by Stephen H. Smith, WA8LMF. The symbols shown by aprs.fi should now match those shown by findu.com, UI-View (with Stephen's symbol files) and other programs running with up-to-date symbols.Thanks to Stephen for maintaining the symbol tables!
The downside is that the new symbols are a bit smaller than the old ones, and they suffer more from the artifacts caused by rotation (for course/heading display).
The upside is that I've also implemented symbol overlays! For example, OH2RDG now has the letter N overlaid on top of the solid green star, both on the map, in Google Earth, and in the info pages.There's also a short FAQ page.
Sunday, August 16, 2009
Arctic Sea position speculation
Quick recap: A cargo vessel called Arctic Sea (MMSI 215860000) was probably hijacked on July 24th 2009 near the eastern coast of Sweden. This was big news in northern Europe, since hijackings generally happen near the Somali coast, not over here. The ship has a Russian crew of 15, it appears to be owned by a Finnish company, and the owners of that company are of Russian origin. The Finnish media had considerable trouble trying to figure out the true owners, and the owners were really hard to interview. The ship deported from the harbor of Pietarsaari on 22nd of July and carries 6500 tons of Finnish timber, worth of about 1.3 MEUR.
The really odd thing is that the ship didn't go to the nearest Swedish port, but continued towards Africa as if nothing had happened. Very strange indeed. Either the hijackers were still on the ship, or the crew is taking part in the plot.
Latest news (Ransom demanded): BBC, CNN, YLE.
There have been a few questions about AIS positions of Arctic Sea shown on aprs.fi.
Q: Why is the track not shown for the moment of hijacking between Gotland and mainland Sweden?
A: There are no AIS receivers in the area which would directly send AIS reports to aprs.fi. These receivers are run by volunteers (thank you!), and each volunteer chooses where to submit AIS data. There is a receiver in the area, but it is submitting data to MarineTraffic only, and while MarineTraffic and aprs.fi exchange AIS data, aprs.fi is not getting the reports of all of those receivers. The Swedish maritime officials have an AIS receiver network of their own, and they've reported it ran circles and stopped for a while.
Q: Is the position shown for Saturday, 15th of August, valid?
A: Technically, it's possible, but I personally would find it very unlikely. It is easy to fake and it doesn't make any sense for the hijackers to publish their true position like this.
The position report was sent by an anonymous receiver station to MarineTraffic. It is quite easy to send fake data to MarineTraffic over the Internet, since they allow unauthenticated UDP packets containing NMEA strings to be sent to the service. aprs.fi does not allow unauthenticated UDP packets, all AIS submissions are tied to a specific receiving station using a password. Of course any one of those stations could feed us invalid positions, but at least we have some idea of the originator.
If the hijackers (or someone else) wanted to play tricks, they could also go to a shop selling marine radio equipment, buy an AIS transmitter, configure Arctic Sea's MMSI number (and other correct data) in it, give it an incorrect position by crafted NMEA strings (fake GPS receiver on the serial port of the AIS transmitter) and have it transmit the packets on the correct AIS frequency. If they've got the money and motivation to hijack ships with guns and speedboats, they've certainly got the guts to buy or steal AIS equipment. They could also grab the AIS transmitter from Arctic Sea, and take it to another position using a speedboat.
The French navy says there were 3 military vessels in the claimed position on the Bay of Biscay, heading for the Baltic sea, and they didn't see the hijacked ship. And they didn't see it on their radar, either.
The coast guard of Kap Verde claims to have seen the vessel about 800 km off the coast of Cape Verde, which is some 3600 km away from the Bay of Biscay.
In any case, this is starting to become a good plot for a movie.
The really odd thing is that the ship didn't go to the nearest Swedish port, but continued towards Africa as if nothing had happened. Very strange indeed. Either the hijackers were still on the ship, or the crew is taking part in the plot.
Latest news (Ransom demanded): BBC, CNN, YLE.
There have been a few questions about AIS positions of Arctic Sea shown on aprs.fi.
Q: Why is the track not shown for the moment of hijacking between Gotland and mainland Sweden?
A: There are no AIS receivers in the area which would directly send AIS reports to aprs.fi. These receivers are run by volunteers (thank you!), and each volunteer chooses where to submit AIS data. There is a receiver in the area, but it is submitting data to MarineTraffic only, and while MarineTraffic and aprs.fi exchange AIS data, aprs.fi is not getting the reports of all of those receivers. The Swedish maritime officials have an AIS receiver network of their own, and they've reported it ran circles and stopped for a while.
Q: Is the position shown for Saturday, 15th of August, valid?
A: Technically, it's possible, but I personally would find it very unlikely. It is easy to fake and it doesn't make any sense for the hijackers to publish their true position like this.
The position report was sent by an anonymous receiver station to MarineTraffic. It is quite easy to send fake data to MarineTraffic over the Internet, since they allow unauthenticated UDP packets containing NMEA strings to be sent to the service. aprs.fi does not allow unauthenticated UDP packets, all AIS submissions are tied to a specific receiving station using a password. Of course any one of those stations could feed us invalid positions, but at least we have some idea of the originator.
If the hijackers (or someone else) wanted to play tricks, they could also go to a shop selling marine radio equipment, buy an AIS transmitter, configure Arctic Sea's MMSI number (and other correct data) in it, give it an incorrect position by crafted NMEA strings (fake GPS receiver on the serial port of the AIS transmitter) and have it transmit the packets on the correct AIS frequency. If they've got the money and motivation to hijack ships with guns and speedboats, they've certainly got the guts to buy or steal AIS equipment. They could also grab the AIS transmitter from Arctic Sea, and take it to another position using a speedboat.
The French navy says there were 3 military vessels in the claimed position on the Bay of Biscay, heading for the Baltic sea, and they didn't see the hijacked ship. And they didn't see it on their radar, either.
The coast guard of Kap Verde claims to have seen the vessel about 800 km off the coast of Cape Verde, which is some 3600 km away from the Bay of Biscay.
In any case, this is starting to become a good plot for a movie.
Subscribe to:
Posts (Atom)