Tuesday, July 26, 2011

New view filters, alerts, out-of-order packet filtering and other features

I've just finished installing a rather large upgrade on aprs.fi. The upgrade comes with a couple of noticeable new features which everyone should get familiar with, a bunch of smaller style fixes and some tiny bug fixes. The web service was down for a while due to the database schema changes and engine upgrades. Only very little APRS data was lost - APRS-IS traffic was buffered during the database maintenance and processed later when the database was up again.

Here's a list of the major changes:

Complex but easy filters:

The old filters (APRS, AIS, Items+objects, WX) in the Preferences have been replaced by a whole new Filter tool button on the right side of the tool bar. It's the one which tries to look like a funnel (I'll have to work on the symbol image to make it more obvious).

Let's do a little exercise to get you started!

Deselect the All stations filter, so that no stations are displayed. Then enable the Station type: Weather filter and you'll only see stations transmitting weather data.

Next, continue by enabling Station class: APRS stations. At this point you'll see both APRS stations and weather stations, which doesn't make much sense since all the weather stations are also APRS stations. But it starts to make sense again if you click the red downward-pointing arrow for the weather stations, which throws them to the bottom of the list, in the red "AND NOT" department, effectively removing them from the display.

Now, click on the plus (+) button to add your own fancy filters to the list. For example, it's possible to create a filter which only shows APRS stations using Kenwood radios which are also acting as Digipeaters, but do not have a callsign starting with "OH2". That filter can then be used in the green "display any of these" side of filters (think of logical OR if you're into programming) or the red "but not these" side (think of AND NOT).

Once you're finished designing your favourite filter list, use the Save button to save it to a new named list. The default filter will reset to it's factory settings every time the map opens (so that you'll see something if you get lost with the filters). The filters are saved together with your user account, so you'll need to log in to save them. The upside of this is that you can easily use your preconfigured filters on many computers and even mobile devices!

Personally, I've set up several commonly used filter sets in my named lists, such as "Infra" for network infrastructure: digipeaters, igates, and "Moving" for stations which have their speed larger than 2.

New graphing engine:

Telemetry, weather and station graphs are displayed by a new third-party graphing engine instead of a homebrew one. The new engine reduces load on my servers (and adds some on the web browser's side, but that's OK since there are thousands of those against my 2), generates prettier line graphs, and will enable some really fancy interactive features in the future!

Station activity alerts:

This feature is mainly for digipeater and igate operators. It lets you know by email if your station stops working, or when it starts working again. Very handy especially if you're like me and maintain digipeaters far away from your home and would not otherwise notice when they go "kaput".

Click on the Favourites (star) tool button, then select My stations and bookmarks. Add your digipeaters in the Stations I follow list, and then click Settings for each of them and tune the timings.

Please note that due to APRS-IS duplicate filtering aprs.fi often does not get to know when your igate or digipeater hears a packet and repeats it. It's a good idea to set the "has not heard station" alert for a very long timer such as 24 or 48 hours, and then go down or up from there depending on the amount of false alarms!

An additional duplicate packet filter algorithm:

In addition to the old filtering methods, out-of-order and duplicate packets are now filtered using APRS timestamps embedded in the packets. If a packet comes in that has an older timestamp than the previous packet, it is promptly discarded. This should be a very effective duplicate filter when timestamps are transmitted by the tracker.

The raw packets display can now also display the decoded timestamp when using the Decoded mode.

Facelifts:

Since most people come to aprs.fi to look up a station's position, the callsign search box has been moved up on the right-side panel, and the time range selector has been moved down.

The real-time map displays a nice "loading" image until station data has been first received from the servers.

Buttons now have a prettier and consistent style everywhere. Some other layout and style tunings have been applied, too.

Bugfixes:

The real-time map's content reloading logic has been improved. It should now be more responsive when panning and zooming around. There's still a small bug in there - sometimes stations are not reloaded on the map when the filters are changed at precisely the right time. Oh well.

The sortable tables were not working in many views. Now they are. Click on table column headers to sort, ascending or descending.

PHG overlays are now correctly hidden when zooming out to area activity view.

The duplicate stations API bug was fixed.

As always, some smaller ones were fixed too. I also did some profiling and optimisation of old code, hoping to keep the performance of the code on similar level as before, even though new features were added in the critical code path.

Upgrades of third-party components:

Web server software, some operating system components, database engine, etc.

Post a note on the discussion group when you find a bug - this is such a large upgrade that I'm certain quite a few things don't work quite right from the start.

As usual, there are translations which need to be updated with the new features. Thank you for your ongoing support!

5 comments:

Martin (VA3SIE) said...

Good job!

I like the new features very much.

There is one bug which I would like to mention.

When i look up a historical track, the station icon is for the last packet received (current time) not the last packet in the selected time range.

So for example the icon of VA2BBW in this query:

http://aprs.fi/?call=VA3SIE-7%2CVA2BBW-7%2CVA2OTA&date_start=2011-07-23%2011%3A45%3A00&date_end=2011-07-23%2019%3A00%3A00&mt=roadmap&z=16&timerange=3600

... is a car.

But I'm pretty sure if I had looked at the map at 2011-07-23 19:00:00 it would have been a walking person.

Great job!!
73
Martin.
VA3SIE.

9M2PJU said...

Good job!

73 from Malaysia

oh7lzb said...

Martin: That's right. To save disk and memory aprs.fi does not store all data separately for each and every position. For example, the symbol is stored only once for each station. It's not a bug, it's a feature, as the saying goes. :)

CT1AKV said...

I like this features too.
But, how to exclude, for instance, all CWOP (not ham)stations, or all spanish stations (EA, EB, EC, etc.)?

Thanks

oh7lzb said...

CT1AKV: Create a new filter with the + button, set it's type to Name or Source callsign, and the value to EA*, and then use the red downward-pointing arrow to move it to the red 'not' section.

This has been discussed on the discussion group, too.