Did you know you can bring down an entire HPC cluster with an old script? Well, this week I had such an experience. As the systems administrator for a seriously aging cluster with over 800 post-graduate and post-doctoral researchers, "stress" is a normal part of daily life (for future reference: it's probably killing me).
Today Mark and I spent an afternoon working on a 115 kbit/s FSK data system for high altitude balloons. Here is a video of Mark demonstrating the system:
In our previous tests, we needed -75dBm to get jpeg images through the system, much higher that the calculated MDS of -108dBm + NF. So we devised a series of tests – “divide and conquer” – to check various parts of the system in isolation.
First, some SDR noise figure tests. We added to these measurements today by trying a few SDRs with a low noise pre-amplifier. First, we measured the pre-amp NF. This was quoted as 0.7dB, we measured 2dB with the spec-an (which has 1.5dB uncertainty). We then tested combinations of the pre-amp with various SDR gains:RTLSDR G=20.......: Pin: -100.0 Pout: 15.5 G: 115.5 NF: 19.4 dB RTLSDR G=50.......: Pin: -100.0 Pout: 38.8 G: 138.8 NF: 5.6 dB RTLSDR G=50 Preamp: Pin: -100.0 Pout: 59.0 G: 159.0 NF: 2.0 dB AirSpy G=10.......: Pin: -100.0 Pout: -1.0 G: 99.0 NF: 19.4 dB AirSpy G=21.......: Pin: -100.0 Pout: 33.9 G: 133.9 NF: 6.7 dB AirSpy G=21 Preamp: Pin: -100.0 Pout: 53.1 G: 153.1 NF: 2.8 dB
The RTLSSDR was a R820T and the pre-amp model number PSA4-5043.
When the pre-amp was used, it boosted the overall gain of the system, and set the system NF to 2dB. It was great to see system NFs close to our measured pre-amp NF – gave us some confidence that our measurements were OK.
The SDRs require high RF gain levels to achieve a low NF. We did notice that at high RF gain levels, birdies appeared in the SDR output spectrum, and there were some signs of compression. We will look into SDR gain distribution more in future.
Testing The Modem Off Line
I wanted to test the modem over the RF link, and the best way to do that is with a BER test. Mark configured the Rpi Tx to send fixed frames of known test data. We used a HackRF to down-convert the received FSK test frames and store them to file.
The data rate of the link is Rs=115.2 kbit/s, which is sampled by the demodulator at Fs = 115200*8 = 921.6 kHz. However to the modem it just looks like an 8 times oversampled signal. So here is 10 seconds of the modem signal replayed at Fs=9600 Hz. You can hear the packets starting by the sound of the header. When replayed at this low sample rate, the bit rate is 1200 baud, and the packets are a few seconds long. At the full sample rate they are just 23ms long.
The non-real time reference Octave demodulator was used to demodulate the FSK sample files, pop up some plots, and measure the BER. Much easier to see what’s going on with the off-line simulation version of the modem. After a few hours of wrangling with fsk_horus.m, I managed to decode Mark’s test frames.
The tx signal we sampled was noise free, so I added calibrated AWGN noise in the Octave simulation to test BER performance of the real-time FSK modulator and transmitter. It was spot on – 1% BER at an Eb/No of 9dB. Great! I do like FSK – real world implementations (like the FSK tx chip) work quite close to theory. This verified that the hardware tx side was all OK in terms of modem performance.
I did however discover a fairly large baud rate error (sample clock offset), of around 1700ppm. This suggested the tx was actually sending at 115,200*1.0017 = 115.396 kbit/s.
Simultaneously – Mark worked out how to measure the baud rate of the RPi serial port. He used the clever idea of sending 0x55 bytes, which when combined with the RS232 start and stop bits, leads to a …010101010… sequence on the RS232 tx line – a square wave at half the baud rate. We connected a frequency counter, and measured the actual baud rate as 115.387 kbit/s, right in line with my numbers above.
The C version of the demod doesn’t like such a large clock offset (baud rate error), so we tweaked the resampling code to adjust for the error. Could this baud rate error could be our “smoking gun” – the reason such a high rx level was required to push images through?
You need to test receivers at very low signal levels. The problem was a relatively high power transmitter nearby generating the tx signal. In our case, the tx power of 25mW (14dBm) is attenuated down to -110dBm, a total of 124dB attenuation. The tx signal tends to radiate around the attenuator (it is a radio transmitter after all). When you reduce the step attenuator 10dB but the signal on the spec-an doesn’t drop 10dB, you have a RF radiation problem.
We solved this by putting the tx in a metal box in another room, then (at Matt, VK5ZM’s suggestion), connecting the tx to the attenuator and rx using coax with a intentionally high loss at UHF. This keeps the high level tx signals well away from the rx.
Mark fired up the real time code again, adjusted for the baud rate error, and suddenly we were getting much better results – good images at -94dBm. He added the pre-amp, and now we could receive images at -106dBm. This is exactly, almost suspiciously close to what we calculated:MDS = Eb/No + 10*log10(B) - 174 + NF MDS = 15 + 10*log10(115E3) - 174 + 2 MDS = -106.4 dBm
It doesn’t usually work out this well ….. guess we are getting our head around this radio caper!
Dropping the signal level to -111dBm meant just a few SSTV packets were making it through. Most of them were bombing on a CRC error. At -111dBm, our Eb/No = 10dB, or a BER of 3E-3. Now our packets are a few thousand bits long, so with BER = 3E-3, we are very likely to cop a few bit errors per packet, which are then discarded due to a CRC fail. So this fits exactly with what was observed at this signal level. Check.
Here a video of Mark running through the experimental set up:
RTLSDR Samples Captured using:rtl_sdr -s 1000000 -f 440000000 -g 20 - | csdr convert_u8_f > rtlsdr_gain20_sig.bin
AirSpy samples captured using:airspy_rx -f440.0 -r /dev/stdout -a 1 -h 21 | csdr convert_s16_f > airspy_gain21_sig.bin
Commands for complete end-to-end decoding (assuming csdr is installed):rtl_sdr -s 1000000 -f 441000000 -g 35 - | csdr convert_u8_f | csdr bandpass_fir_fft_cc 0 0.4 0.1 | csdr fractional_decimator_ff 1.08331 | csdr realpart_cf | csdr convert_f_s16 | ./fsk_demod 2X 8 923096 115387 - - | ./drs232 - - | python rx_ssdv.py --partialupdate 8
The python code is here.
Visualising our Packets
Brady, KC9TPA, the author of the C FSK modems, sent us a neat visualization of our test packet:
You can see the 0x55 bytes in the header, the unique word, then a sequence of 0x0 too 0xff, sent LSB first, with the RS232 start (0) and stop (1) bits.
Brady created that image using a screen capture of this:xd -g 10 -c 10 -s 2 115.bin | tr '1' '*' | tr '0' ' '
The Basslink cable was cut this morning in order to repair it. Concerns that this could cause Tasmanian internet speeds to go to shit appear to have been well founded (see #basslink on Twitter). In my case, my NBN wireless has gone from 21Mbps/5Mbps to 1Mbps/4Mbps, i.e. my download speed is a twentieth of what it was before, but my upload is about the same. So I may as well post some photos, right?
I spent the second half of February at SUSE Mission Control in Nuremberg, which was one of those relatively infrequent but wonderful opportunities to spend time in person with the people I work with. So it actually wasn’t a holiday (we were all there to do SUSE Enterprise Storage stuff), but I did take a few photos. Here’s the ones that didn’t come out too badly.
First, enroute via Singapore, the butterfly garden and some helpful advice on books.
The hotel itself had friendly staff, nice food, pleasant table candles, and a whole bunch of Paul-related pictures on the walls (my favourite was the non-human one).
It was fairly cold. Maybe 1-7 degrees C most days, with an occasional attempt at a snow flurry. So I got to be the Purple Ninja.
One day on the way to the office, we found the centre of town.
On the weekend, I found a lightsaber I will never be willing to purchase.
There were plenty of churches, statues and other interesting pieces of architecture.
I also discovered that beer coasters have far more structural integrity than playing cards.
The interior decor is sometimes a little odd.
The Cascade beer at Kater Murr is completely unrelated to the Tasmanian beer of the same name.
…and some buskers. The panda didn’t do much (although the kids seemed entranced), but the Jewish Jazz Trio totally rocked.
On the way back through Singapore I found an interesting machine. Usually you have to pay good money to experience pain with pleasure, but this thing was free. On reflection I’m not sure the terms “pleasure” or “pain” are appropriate – I think it’s pretty much just weird robot massage.
There are a few upcoming speaking gigs this March 2016:
Two talks in London and one keynote in Singapore, while I will also moderate the database track on the last day of the conference as well. I’ve never been to flossUK before, but the schedule looks very interesting and I’m looking forward to it. FOSSASIA is an event I’ve been going to regularly since it was held in varying places (Vietnam, Cambodia, before finally settling down in Singapore) so this is an always exciting time.
Would happily meet you to talk shop, feel free to drop me an email: colin[at]mariadb.com.
We've got a great line-up of candidates to choose from this year in the Drupal Association board election. It's a really tough choice.
But reading through the candidates statements it seems many of them seem to think that being on the board of the association somehow influences the project directly. It doesn't.
The board governs the association. It doesn't govern Drupal.
The Association doesn't govern Drupal either.
As a community - we should probably think and talk about project governance more than we do, and perhaps consider or Re-consider how the association fits into the picture.
The Association works to keep the lights on for Drupal.org, and run DrupalCon. There's other important stuff, but those two things are the Big Jobs - the heavy lifting. Last year we ran a fund-raising campaign to directly fund work on the project. That was new, and great work was done, but we also didn't decide who did that work, or what work they did.
Last year, Holly (our awesome Executive Director) wrote a great blog post that outlines what the board does.
Anyway - some rushed and random thoughts after I cast my vote.
Please cast yours!
If you have an account on Drupal.org and logged in during the past 12 months, you are eligible to vote. Please do so!
Last night Mark and I were working on a 115 kbit/s FSK modem link for balloons that uses SDR receivers and FSK modem software.
We connected the transmitter to the SDR rx via a switched attenuator, calibrated with the spec-an, and measured a MDS of about -75dBm. Based on an Eb/No of 15dB for 2FSK (to get a BER of 1E-5) I was expecting:
MDS = Eb/No + 10*log10(B) - 174 + NF
MDS = 15 + 10*log10(115E3) - 174 + NF
MDS = -108.4 + NF
So we are way off expected performance at the moment, and are testing the various components one by one.
One number I wanted to know was the Noise Figure (NF) of the SDR. Mark has a bunch of SDRs so we got on a roll and checked out the NF on all of them.
Now there are a lot of people playing with SDRs, but very few looking at the engineering behind them. It’s not that hard, as you can see from the MDS calculations above. So I thought it might be useful to explain how we measured Noise Figure (NF) and present the results.
From the Wikipedia article on Noise Figure:
The Noise Figure is the difference in decibels (dB) between the noise output of the actual receiver to the noise output of an “ideal” receiver.
An ideal receiver would have an output noise power of:
Nout_dB = 10log10(B) -174 + G_dB
The -174 dBm/Hz figure is the thermal noise density at 25C, for every 1Hz of bandwidth your will get -174dBm of noise power. It’s the lower limit set by the laws of physics. G_dB is the Rx gain. The 10log10(B) term takes into account the bandwidth of the Rx. A wider bandwidth means more total noise power.
So if you have a 1Hz bandwidth, and a gain of 100dB, you would expect Nout_NdB = 0 -174 + 100 = -74dBm at the Rx output with no signal. If you have a 1000Hz bandwidth receiver you would have NdB_out = 30 -174 + 100 = -44dBm of noise power at the output.
To determine Noise Figure:
- Sample the Rx output first with an input test signal of known power and then with noise only.
- Find the Rx gain by comparing the Rx output power to the test signal input power.
- Find the noise output power, then using the gain we can find the noise input power.
- Normalise the noise input power to 1Hz noise bandwidth and compare to the thermal noise floor.
Here is the GNU Octave script nf_from_gr.m we used, and here are the results we obtained:octave:17> nf_from_gr HackRF: Pin: -100.0 Pout: 38.1 G: 138.1 NF: 8.9 dB RTL-SDR: Pin: -100.0 Pout: 45.8 G: 145.8 NF: 6.7 dB AirSpy: Pin: -100.0 Pout: 42.0 G: 142.0 NF: 7.3 dB FunCube PP: Pin: -100.0 Pout: 31.0 G: 131.0 NF: 3.9 dB
GNU Radio was used to collect a few seconds of samples and save them to a file for processing by Octave. These results are not definitive – there are so many “knobs and sliders” on the SDRs it’s hard to say if we had them adjusted right. More gain in front of the ADC tends to reduce NF. The results seem reasonable though, no huge or negative NFs.
A good way to check our script would be to use a pre-amp with a known NF in front of the SDR, but we haven’t tried that yet.
It’s important to sample a chunk of noise without too many birdies (and SDRs seem full of them!). So the script asks you for a start “st” and end “en” range of frequencies. Here is a sample spectrum of the SDR output with the test signal (top) and just noise (bottom):
Ray Tomlinson (Amsterdam, New York state USA, 23 April 1941 – 5 March 2016) invented email in 1971. While working at a company in Massachusetts that had computers on the ARPANET (which morphed into the Internet we know), he thought it would be a good idea if employees were able to send messages to each other as well as files. He choose the ‘@’ sign separate the username from the machine (host) name which he added, enabling the system to distinguish a remote username from a local one.
Email wasn’t actually part of the actual project he was supposed to be working on… but it caught on very quickly because of its obvious benefits. Fast forward to now.. in 2015 it was estimated more than 200 billion emails are sent daily, but that number does include spam which actually makes up a large proportion (about half in 2015).
It is also estimated there are currently over 4 billion email addresses – unfortunately that doesn’t quite mean that more than half the planet’s population has email, as quite a few people have more than one email address, and also computer programs can have an email address of their own.
Tomlinson was co-author on an early standards document RFC-561: Standardizing Network Mail Headers (5 Sep 1973), which was one of the predecessors of the current email standard RFC-822: Standard for the format of ARPA Internet Text Messages (13 Aug 1982).
A couple of years ago while I was working on a project with Kevin M. Hoffman, he related a story to me about his consulting work with an agency on improving presentations to clients. The story centers around a designer who was asked to change his mode of dress. This designer was fond of the goth look, but the agency considered it inappropriate for some client meetings.
Long story short, Kevin told the staff member that he could wear whatever he wanted to, but to consider the situation in terms of his desired outcomes. Clients’ opinions about clothing and personal expression aside, what was more important to the designer: dressing a certain way, or successfully pitching the design direction he was the most excited to work on? The “what to wear” decision then becomes less about the designer’s pride or interest in retaining control, and more about getting what he wants in the situation; acting in his own self-interest to the best of his ability.
Recently, as I worked on an extended project for a client at Bearded, these ideas started percolating through my brain again, but this time with regard to design. Every designer (and really everyone involved in a design) has tendencies and predilections that, like it or not, will be guiding the design process.
Of course we make our best efforts to ground the creative process in research, interviews, and egalitarian decision-making activities (sticky notes and card sorts, anyone?). But no matter what we do, our entire process is filtered through the very fallible, very subjective minds of people.Let’s talk about your childhood
For a moment, we’ll zoom in from our mile-high view and consider a single person. Let’s imagine a child who grows up under the care of well-meaning but disorganized parents. The parents’ plans and decisions are unpredictable. The child, never knowing what will happen next, grows into a bright, talented young person with an unresolved longing for structure and order.
We might even imagine that this need to bring order from chaos, to sort out the unpredictable, messy parts of the world, is what draws them into a career as a designer.Responsible adults
As they find their professional legs, they mature into not so much the David Carson, Stefan Sagmeister, James Victore, anything goes sort of designer — more of an Erik Spiekermann, Willi Kunz, or Sylvia Harris. They’re not out to tear the design world a new one. To the contrary, they’re focused on sorting out the garbled information before them; smoothing over the rough patches, and slowly rotating their subject’s gleaming, flawless, chrome surface towards the world.
And there’s nothing wrong with this orderly sort of design. In fact, it’s very, very useful to the world. Perhaps even necessary.
Likewise there’s nothing wrong with that wild-eyed sort of design, either. One might imagine (as many do) these types of design as poles on a spectrum of useful approaches. Poles between the qualities of Consistency and Variety.Non-binary systems
As it turns out, every project demands to be somewhere on this scale. And it’s essential during the first part of every project (the research, the interviews, even the sticky notes) to figure out which spot on the spectrum is best suited to the project.
Now the extremes of this range are rarely appropriate for any project. An unwaveringly consistent design approach will often seem generic and boring. On the other hand, a design that is entirely committed to variety will make the audience struggle at every turn to orient themselves and parse the information before them.
So it seems fair to say that most designs benefit from being somewhere in the middle zone, balancing between the unhappy extremes of boredom and chaos.Advanced calibration
But what happens when our imagined designer – the one who is drawn to systems of order and control – determines through their research that their project requires a design approach that is on the more varied side of center? When the organization, in fact, will benefit from a less rigidly ordered design? This organization, it turns out, needs an approach that may not sing with as much immediate and obvious clarity, but will bring more surprise and thrill to the audience.
And so we find ourselves with a mismatch between impulses (bring order!) and outcomes (show us surprises!). The problem is that the designer’s approach is not in a conversation with the project and its goals; it’s stuck in a very old dialog with the designer’s childhood. If left unaddressed, a successful project outcome will depend on whether or not those old desires happen to match up with the project’s requirements.
If we don’t want to leave things up to chance, this situation requires the identification of both the designer’s impulses and the project’s desired outcomes, and a conscious assessment of their overlap and contradictions.
When I was in a critique at design school, one of my classmates commented on another’s work that they “really seemed to be developing a style.” My professor became suddenly incensed (a rare thing for her), and declared “you’re designers, you don’t have a style! Your style is whatever is appropriate to the project!” In that way design is very different than art. Art stems primarily from the artist’s internal world. But design does not. Design aims to solve problems outside of the designer. Whereas the artist might be seen as a sort of oracle speaking to the world, the designer is more of a tool that is useful in solving problems.
Which is why, in the cases where a designer’s internal impulses and the project’s desired outcomes are not in alignment, the designer must consider adjusting. To be a useful tool, we must recognize that some of our impulses work against the needs of the organization that is employing us, and compensate.
This is no easy task. It’s a process that requires knowing oneself, and questioning our own nature and subjectivity. But it’s only through this sort of rigorous self-assessment and awareness that we can grow beyond our limitations–becoming better designers, and perhaps if we’re lucky, more sensitive and thoughtful people in the process.
Tweet: Tweet: Tweet: Tweet: Melbourne is just the second… dhanapalan.com/blog/2016/03/0…
Tweet: Tweet: Tweet: Tweet: Mercedes kicks robots off ass… dhanapalan.com/blog/2016/03/0…
Tweet: Tweet: Tweet: Tweet: Australia struggles to bring … dhanapalan.com/blog/2016/03/0… https://t.co/lMOnSqVUJ3
Tweet: Tweet: Tweet: Tweet: Opportunities abound as new j… dhanapalan.com/blog/2016/03/0…
Tweet: Tweet: Tweet: Tweet: More inventions than Edison:… dhanapalan.com/blog/2016/03/0…
Tweet: Tweet: Tweet: Tweet: https://t.co/wRV3BIsCLL https… dhanapalan.com/blog/2016/03/0… https://t.co/rYiLM43Pz6