The Periodic City, Part II

Here we have a week’s span of air (ADS-B, air traffic control frequencies) and vehicle (TPMS (tire pressure sensors)) traffic.  This augments the previous Periodic City post nicely.  (Right click->View Image to see a larger version)


The lulls in the ATC radio traffic “sort of” correspond to ADS-B (aircraft telemetry) traffic.  But not entirely.  There are still aircraft overhead even when MCI is dead.


The TPMS graph shows a definite pattern (x-axis is UTC).


Here is the graph for just one TPMS serial number (one tire sensor).  The two days where there is no ‘blip’ are Saturday and Sunday.  We guess this person is traveling to or from work.

A multi-function node


A NUC runs the ΓRF client.  The HackRF One provides freqwatch, scanner, and spectrum input.  The black RTL-SDR in the lower left is being used for TDOA (transmitter localization) development.  The orange ‘FlightAware’ stick and the blue filter are used to capture ADS-B.

The Periodic City


The above image is a profile of scanner module hit density over a week.  The x-axis is time, and the y-axis is hit density (hits per time period).  The time is in UTC.  The semi-periodic waveforms you see are an EKG-like graph of our city’s heartbeat!  (Right click->View Image to see a larger version)

The bottom graph measures hits on KCMARRS – a network that handles much of our law enforcement, emergency services, and government communications.  The top graph measures scanner hits on local pager networks.  Pagers are used for industrial control systems, and most of all doctors.

The periodicity of both graphs is striking.  KCMARRS peaks around 1600-1800 UTC (2200-0000 local).  It troughs around 1000 UTC.  The pagers tend to lag government communications by a small amount.

What amount of the fluctuation is due to time-dependent changes in RF propagation?  The graph below shows one of the pagers frequencies over the time period in question.  The x-axis is time, the y-axis is power.  The gap and slight shifting of signal level after the gap are due to a service interruption.


At this frequency, and presumably the others (which are close by in frequency), the power stays relatively constant over time.  So we can safely conclude that the change in hits is due to activity on the frequency.

There are likely larger wave-form periodicities that can be extracted from multi-weekly and seasonal-scale data.  Are police and doctors busier on the weekends?  Are summers much busier than winters?

An interesting application would be to hook the data up to an alarm – so that any sudden rise in the government net alerts an analyst.  Terrorist attacks and large disasters will likely trigger a sharp rise on the graphs.

ΓRF catches unusual activity bursts on KC area ham nets


We’ve set up Grafana dashboards that use data sent to the ΓRF server by the clients distributed around the city.  Some of these are used to monitor amateur radio (ham) frequencies.  It’s not unusual to see sporadic blips here and there: hams searching for other hams.  Sometimes we’ll see bursts of blips that last for awhile; these are conversations.  (Right click->View Image to see a larger version)

Something unusual occurred 18Jan2018 at about 0100UTC, and again at about 2200UTC.  Several ham radio frequencies, including repeaters and storm chaser frequencies, began to light up the dashboards.  The radio traffic lasted between six and nine hours, depending on the frequency.  Many of the frequencies seemed to cease their activity all at once.

There are a couple of theories as to what was happening.  One of our team suggested sporadic E propagation / ducting.  This had been going on in the area around that time.  Another theory was that a large, multi-repeater exercise was taking place in the area.  This is also a possibility, though a short search online didn’t provide any clues.  I was able to confirm there were indeed transmissions taking place on several of these frequencies (as opposed to noise) by using my hand-held radio – so it doesn’t seem that the cause was wide-band noise.

Either way, it was exciting to see ΓRF pick up this unusual traffic.  It will be stored in our time-series database and displayable on the graphs for the next couple of weeks.

How we got hackrf_sweep to work with ΓRF on the Raspberry Pi


ΓRF can use the HackRF to track of power over a broad frequency range.  It does this with the hackrf_sweep tool.  Currently the scanner, snapshot, and freqwatch modules use this capability.

Getting hackrf_sweep to work efficiently with the ΓRF client was not trivial.  It took hours of reading hackrf_sweep code and experimentation on the client before we reached an acceptable performance level (i.e. before were able to use the data coming in from the hackrf in near-real-time, and avoid overflowing buffers).  The time spent was worth it: we optimized so well that ΓRF can work with hackrf_sweep even on (newer) Raspberry Pis!

Here’s what we had to do to get it to work:

  1. We used the binary output of the hackrf_sweep command (instead of the default text output).  The binary output format doesn’t seem to be documented anywhere online, so it was necessary to read over hackrf_sweep.c and figure out what was going on.  You can see how we decode the binary output here.
  2. Before we were using hackrf_sweep to get frequency information, we used rtl_power (and of course, took the readings from rtl-sdr devices).  This limited the amount of bandwidth we could process at once, but it made it easy to store frequency/power pairs.  We could simply keep them in a dictionary.  A Python dictionary is efficient, but it does have a necessary amount of overhead.  This overhead precluded its use with hackrf_sweep with ΓRF in many systems.  Replacing the frequency map with a numpy array (as opposed to a Python dict), and storing power values at calculated offsets in the array, drastically sped things up.
  3. Because we were using a numpy array and offsets for frequencies, more calculations were necessary to store and find data.

The end result is an efficient ΓRF spectrum module that can provide frequency to power mapping across a relatively large bandwidth to any module that requires it, on hardware as puny as a Raspberry Pi.