June 7, 2019
Yesterday I released
Hacksilver, an album of
procedurally generated "algorave" music. Some people had questions about
the technology used to write it so I thought I'd write this up.
The beats and melodies were generated using
drillbit, a LISP codebase written
in a Python variant called Hy. The project outputs Impulse Tracker mod
files which are then
played and mixed live.
The interesting parts of that codebase are in the generators folder. For
example the drill-n-bass choppage generator is
Each generator has three functions:
make-sample-set: which generates IT wav tables that are used by
the generator (e.g. individual drum kit or synth sounds)
make-pattern-settings: which sets up parameters & context that
will be re-used by the pattern generator to provide similarity across
make-pattern: which outputs the pattern data in a format easily
consumed by the Impulse Tracker file writer
Mixing and live-effects are performed in Pure
Data. Originally I was using a fully software
based mixer. However I discovered that
a nicer mode of operation is to have individual bits of sound
generating/filter hardware chained together. So I started using this
Raspberry Pi based mixer + FX unit from another
project to mix live.
One other bit of software in there is
jsfxr which is wrapped by the LISP
code and outputs 8-bit synth sounds (which are then used by the pattern
generator). Because the synth definitions are simple JSON hash maps
there is a fun pseudo-evolutionary technique I was able to use where you
interpolate between the values of two synth definitions to generate new
sounds based on two synth definitions that you like.
I also built a little hardware Impulse Tracker renderer based on a
Raspberry Pi running XMP with my
friend Dimity. It has a Pocket Operator style sync output and runs
directly into the mixer that both share the same timing and the fx can
be quantised to the music which is playing.
If you're interested in the music hardware that Dimity and I are
building and selling you can stay updated at
In the image at the top of this post the hardware Impulse Tracker
renderer is the little box on the right hand side. The RPi mixer/fx unit
is to the top right of the C64 keyboard. The Korg Nanokontrol2 strapped
to the C64 keyboard is controlling the fx and mixing parameters on the
RPi. They keyboard itself was for playing live synth sounds (a very
simple arpeggiating subtractive synthesizer built in Pd).
June 6, 2019
I just released
Hacksilver, a new album
of procedurally generated music.
It uses a whole slew of weird tech to generate the beats, melodies,
synth sounds including beat-generating
LISP, 8-bit synth generating
Data for the mixing and mastering. One thing
that was particularly fun was procedurally generating Impulse Tracker files.
Would appreciate a re-share if you know of anybody who might be into
this type of thing.
May 12, 2019
In 2014 Arvid Norberg and Steven Siloti came up with a BitTorrent extension called BEP44. The basic purpose of BEP44 is to allow people to store small pieces of information in a part of the BitTorrent network called the DHT. The DHT ("distributed hash table") is a key/value lookup table that is highly decentralized. Prior to BEP44 it was used to look up the IP addresses of peers from the hashes of the torrent they were sharing.
BEP44 introduces two new ways of storing key-value data in the DHT. The first is the ability to look up small bits of information keyed directly on their hash. The second allows for the storage of cryptographically authenticated data, keyed on the public key. What this means is if you have some public key K then you can look up authenticated blobs of data stored in the DHT by the owner of that key. This opens up a variety of useful abilities to people building decentralized applications.
For instance when you distribute a file on BitTorrent it is immutable - you can't change it - but BEP44 provides a way to tell people "hey, there is a new version of this file that I shared before which you can find over here" in a secure and authenticated way. It turns out that this basic mechanism can be used to build a wide variety of decentralized, authenticated functionality and people have built cool experiements like a decentralized microblog using it.
The purpose of this post is to show you how the datastructure works and how to apply it more widely in your own software.
What's good about BEP44
Most of the advantage of BEP44 comes from the cryptographic signing. What this offers is a way for somebody (let's call her Alice) to verify that a piece of data from somebody else (let's call him Bob) is authentic and that it has not been tampered with. You can do cryptographic signing without BEP44 of course.
However, the BEP44 specification brings some other features for building decentralized systems:
- Namespacing against a public key.
- Replay attack prevention.
- A compare-and-swap primitive.
The namespacing feature means that a single public key can have a bunch of different data blobs that others can authenticate. Because the datastructure is keyed on both the public key and a "salt" field, a single public key can store multiple blobs with different "salt" values as a sub-key.
Replay attack prevention is accomplished with the sequence field, which in BEP44 is a monotonically increasing integer. The way a replay attack works is some adversary keeps an old copy of something you have signed and then when you are offline they send it again and pretend its a new message. Because the message is signed with your key people are fooled by the adversary into thinking the old data is current. In BEP44 the sequence field prevents this because the adversary will have a data blob with a lower sequence number than the last one you shared and they can't generate a new blob with a higher sequence number because they do not have your private key. So the replayed data blob will be rejected.
Finally, the compare-and-swap primitive works by specifying a previous sequence number that the current data blob should replace. Peers will reject data blobs which don't replace the current sequence number they hold. In this way it's possible to acheive basic synchronisation and atomicity of data blobs. Using compare-and-swap guarantees that the new value you want to insert into the DHT can be calculated based on up-to-date information.
Imagine you take upon yourself the small task of building a decentralized social network. Users have a timeline of posts which they have written. When they write a new post it should be appended to their timeline and people should see the updated timeline with the new post.
In the centralized social network case this is easy. The social network provider TwitFace has an internal copy of the poster's timeline to which they add the new post. Followers are notified of the update and the updated timeline is sent to them by the provider. The way the authentication works in this case is the original poster has logged in with their password and so the provider knows who they are, but the receivers of the update must trust the provider.
How do readers know that the post is authentic and comes from the original poster? The reader must trust the provider completely. They must trust that the provider is showing them the authentic timeline of messages, that the messages have not been modified, that fake messages have not been injected into the timeline, that new messages have been added to the feed in a timely fashion, and that no message has been censored and removed from the feed by the provider.
The problem with this model is that trusted third parties are security holes. Centralized social networks do modify things people have said. They do inject posts into people's timelines (ads and worse). They do change the order and timing of posts to suit their own goals. Perhaps worst of all, they do censor posts completely.
I won't get into the politics of censorship. Suffice it to say that censorship is fine and good right up until the point where your values differ from the entity doing the censoring. Values change, mysterious outside influences are myriad, and few people have complete alignment of values even with our noble corporate overlords in Silicon Valley.
The basic goal here is that if you've chosen to read somebody's timeline you can trust that the feed of posts you are reading from them is authentic and complete and as they intended.
Happily, we can use the BEP44 technique to route around the trusted-third-party centralized-social-network-provider security hole. BEP44 provides a way to receive an update and verify it cryptographically. It provides a way to know that this is the latest and complete version. It allows the verifier to do this using only the received data structure without requiring any kind of secret server side logic or password based authentiction like you would find in a centralized social network.
How it works
At the heart of the algorithm is a small datastructure which can be shared, updated by the sharer, and then shared again. Receivers can verify the updates are authentic. The idea is that you can share a small piece of authenticated data which points to a larger piece of immutable data. For instance, you can share an authenticated datastructure with the hash of a torrent containing all of your posts (an RSS feed for instance). Receivers then know that torrent is the current representation of your timeline of posts.
The datastructure has the following fields:
- cas [optiona]
- salt [optional]
When a verifier receives a copy of this data structure they perform the following checks:
- Is the size of the
value field below the maximum size?
- Does the
value conform to the expected format / encoding?
- Is the
seq number higher than the last
seq number I saw (if any)?
cas is present does it match the previous
seq number I have?
- Is the attached signature made with the private key corresponding to the attached pubkey over the datastructure's value, seq, and salt fields?
In this final point they are checking that the structure has been digitally signed with the author's private key. This is accomplished by concatenating
salt + seq + value and then checking that the signature is valid for that data and the given public key.
In this way verifiers can know that an update from a known keypair/identity is authentic and current.
Read more posts on the subject of cryptography & decentralized systems.
May 10, 2019
My mother, Sally Kathryn (Laing) McCormick, passed away on the 25th of
April, 2019. This is the eulogy I gave at the celebration of her life on
"Your mum is the nicest person I've ever met."
I've heard these words from friends and strangers countless times during
my life. They are how I slowly came to understand that Mum was somebody
extraordinary and not to be taken for granted. I'm so glad to have been
able to express my own gratitude to her in recent years.
In Henry the Fourth Shakespeare has a turn of phrase: "like bright metal
on sullen ground." Mum was bright metal on sullen ground. She was solid
Everyone here will have their own fond, and if you knew her well,
sometimes frustrating memories of my mum. I want to focus today on those
things I came to really admire in her.
We all knew that warm side of her because she gave it freely, to
everyone, everywhere she went. No matter who you were you would get a
sweet smile, warmth, and praise. Perhaps fewer people here will know of
her strength, energy, and willpower, her humour, her infinite gratitude,
and her fundamentally uncomplaining nature.
In recent years I've found inspiration in the philosophy of the stoics
and it is hard not to notice strong parallels with the way mum carried
herself, always ready to help, always grateful, always giving. Standing
straight, or held straight, by sheer will. Until around 11 each night
when the universe would gently say "ok, it's time to sleep now Sally",
and she would nod off where she stood.
And then, as my father knows all too well, was woken every morning by
her two alarms, one at 4:30, so that she had "time to think", and one at
5 so that she could actually wake up.
But Mum was not a stoic, she was a Christian. Even though she was tested
by the Good Lord with three opinionated, and two atheist sons, her faith
never waivered in the slightest, even at the very end. As with
everything about her, it held fast and true and uncompromising
throughout her life. I am not a Christian but I admire her persistence
I am going to try to be more like my mother.
I'm going to try to be as grateful as she was. To remember that every
morning I wake, every cup of coffee, every lego spaceship built with my
kids, is a gift of limited supply. To delight and give thanks as she did
in the smallest of things, common or uncommon. Any time I said thank you
to mum, she would brush it off. Clearly, she thought it her basic duty
to this world and the people in it to give everything she had with
The most valuable thing each of us has to give is our time. Mum always
had time for people and most especially for us her sons. In this too I
hope to emulate her and give more time to my own kids, and to you, my
friends and family, and the other people in my life, just as she did.
She never ever complained. Never. Instead she did whatever was in her
control to help other people, and sometimes even things which were well
outside her control. She knew that complaining accomplishes nothing but
to make you feel worse and to burden those around. Mum never burdened
anybody. If I need help I will ask for it. If I can fix something I will
simply fix it, just as she did.
Mum had incredible energy. It wasn't the kind of energy you get from an
energy drink, or from eating high energy food, because we all know she
did not do that. No, it was the energy that springs forth at 4am at the
hospital after several sleepless nights when your sick kid needs to be
held. Its the energy you didn't realise you had until you decided to try
and overcome the fatigue with willpower. Mum made amazing use of that
particular reserve, and whilst it would probably not be advisable to dip
in to quite the same extent as she did, it's good to know that there is
always more you can give when required.
Finally, I aspire to mum's strength. She powered joyfully 100% into
everything that life handed to her, and even her last hugs were
vice-like, and with a genuine smile. Again that is not because she was
particularly physically strong, although she was, but because she was
emotionally and spiritually strong. If a mouse like my mother can be so
mighty, surely I can too. She made it apparent that it is a simple
matter of choosing to be strong.
Mum and Dad's legacy speaks for itself. The life they made for
themselves in this country. The lives they helped others to build. The
success of their sons Dirk and Mike in building a good life for
themselves, sometimes against extraordinary odds which most people will
never face. Of course I also owe a huge debt to my parents for the
wonderful life I enjoy today, a debt I intend to pay forward wherever I
can, in my mum's spirit.
At times like this the universe can seem cold, and harsh, and unfair;
but people like Mum show us that goodness and love abound in this
universe. The goodness and love is in us. We get to defy the cold and
the unfair. Like mum, each of us gets to choose to be strong, to love,
and to make reality wonderful for each other.
Thank you Mum, for caring for me, and Dirk, and Mike, and Dad, and for
Orson and Scout, and for always believing in me. Thank you for always
laughing out loud at our stupid jokes. Thank you for sitting patiently
and learning to code with me on our Apple IIe when I was 8. Thank you
for showing me the way to be a good human. I promise I will try.
March 27, 2019
This post originally appeared on the David Walsh blog.
In this 15 minute tutorial we're going to build a simple decentralized chat application which runs entirely in a web browser.
All you will need is a text editor, a web browser, and a basic knowledge of how to save HTML files and open them in the browser.
Read more posts on the subject of cryptography & decentralized systems.