independent software developer
I just released Hacksilver, a new album of procedurally generated music.
It uses a whole slew of weird tech to generate the beats, melodies, synth sounds including beat-generating LISP, 8-bit synth generating Javascript, and Pure Data for the mixing and mastering. One thing that was particularly fun was procedurally generating Impulse Tracker files.
Would appreciate a re-share if you know of anybody who might be into this type of thing.
Enjoy!
In 2014 Arvid Norberg and Steven Siloti came up with a BitTorrent extension called BEP44. The basic purpose of BEP44 is to allow people to store small pieces of information in a part of the BitTorrent network called the DHT. The DHT ("distributed hash table") is a key/value lookup table that is highly decentralized. Prior to BEP44 it was used to look up the IP addresses of peers from the hashes of the torrent they were sharing.
BEP44 introduces two new ways of storing key-value data in the DHT. The first is the ability to look up small bits of information keyed directly on their hash. The second allows for the storage of cryptographically authenticated data, keyed on the public key. What this means is if you have some public key K then you can look up authenticated blobs of data stored in the DHT by the owner of that key. This opens up a variety of useful abilities to people building decentralized applications.
For instance when you distribute a file on BitTorrent it is immutable - you can't change it - but BEP44 provides a way to tell people "hey, there is a new version of this file that I shared before which you can find over here" in a secure and authenticated way. It turns out that this basic mechanism can be used to build a wide variety of decentralized, authenticated functionality and people have built cool experiements like a decentralized microblog using it.
The purpose of this post is to show you how the datastructure works and how to apply it more widely in your own software.
If you just want a working implementation you can use, check out the decentral-utils JavaScript library which has a single-function implementation of the datastructure and algorithm which you can use in the browser. Web browsers unfortunately do not have direct access to the BitTorrent DHT, but the library is still useful for authenticating up-to-date data blobs that are passed around between browsers, for example over WebRTC.
Most of the advantage of BEP44 comes from the cryptographic signing. What this offers is a way for somebody (let's call her Alice) to verify that a piece of data from somebody else (let's call him Bob) is authentic and that it has not been tampered with. You can do cryptographic signing without BEP44 of course.
However, the BEP44 specification brings some other features for building decentralized systems:
The namespacing feature means that a single public key can have a bunch of different data blobs that others can authenticate. Because the datastructure is keyed on both the public key and a "salt" field, a single public key can store multiple blobs with different "salt" values as a sub-key.
Replay attack prevention is accomplished with the sequence field, which in BEP44 is a monotonically increasing integer. The way a replay attack works is some adversary keeps an old copy of something you have signed and then when you are offline they send it again and pretend its a new message. Because the message is signed with your key people are fooled by the adversary into thinking the old data is current. In BEP44 the sequence field prevents this because the adversary will have a data blob with a lower sequence number than the last one you shared and they can't generate a new blob with a higher sequence number because they do not have your private key. So the replayed data blob will be rejected.
Finally, the compare-and-swap primitive works by specifying a previous sequence number that the current data blob should replace. Peers will reject data blobs which don't replace the current sequence number they hold. In this way it's possible to acheive basic synchronisation and atomicity of data blobs. Using compare-and-swap guarantees that the new value you want to insert into the DHT can be calculated based on up-to-date information.
Imagine you take upon yourself the small task of building a decentralized social network. Users have a timeline of posts which they have written. When they write a new post it should be appended to their timeline and people should see the updated timeline with the new post.
In the centralized social network case this is easy. The social network provider TwitFace has an internal copy of the poster's timeline to which they add the new post. Followers are notified of the update and the updated timeline is sent to them by the provider. The way the authentication works in this case is the original poster has logged in with their password and so the provider knows who they are, but the receivers of the update must trust the provider.
How do readers know that the post is authentic and comes from the original poster? The reader must trust the provider completely. They must trust that the provider is showing them the authentic timeline of messages, that the messages have not been modified, that fake messages have not been injected into the timeline, that new messages have been added to the feed in a timely fashion, and that no message has been censored and removed from the feed by the provider.
The problem with this model is that trusted third parties are security holes. Centralized social networks do modify things people have said. They do inject posts into people's timelines (ads and worse). They do change the order and timing of posts to suit their own goals. Perhaps worst of all, they do censor posts completely.
I won't get into the politics of censorship. Suffice it to say that censorship is fine and good right up until the point where your values differ from the entity doing the censoring. Values change, mysterious outside influences are myriad, and few people have complete alignment of values even with our noble corporate overlords in Silicon Valley.
The basic goal here is that if you've chosen to read somebody's timeline you can trust that the feed of posts you are reading from them is authentic and complete and as they intended.
Happily, we can use the BEP44 technique to route around the trusted-third-party centralized-social-network-provider security hole. BEP44 provides a way to receive an update and verify it cryptographically. It provides a way to know that this is the latest and complete version. It allows the verifier to do this using only the received data structure without requiring any kind of secret server side logic or password based authentiction like you would find in a centralized social network.
At the heart of the algorithm is a small datastructure which can be shared, updated by the sharer, and then shared again. Receivers can verify the updates are authentic. The idea is that you can share a small piece of authenticated data which points to a larger piece of immutable data. For instance, you can share an authenticated datastructure with the hash of a torrent containing all of your posts (an RSS feed for instance). Receivers then know that torrent is the current representation of your timeline of posts.
The datastructure has the following fields:
When a verifier receives a copy of this data structure they perform the following checks:
value field below the maximum size?value conform to the expected format / encoding?seq number higher than the last seq number I saw (if any)?cas is present does it match the previous seq number I have?In this final point they are checking that the structure has been digitally signed with the author's private key. This is accomplished by concatenating salt + seq + value and then checking that the signature is valid for that data and the given public key.
In this way verifiers can know that an update from a known keypair/identity is authentic and current.
Read more posts on the subject of cryptography & decentralized systems.
My mother, Sally Kathryn (Laing) McCormick, passed away on the 25th of April, 2019. This is the eulogy I gave at the celebration of her life on Monday.
"Your mum is the nicest person I've ever met."
I've heard these words from friends and strangers countless times during my life. They are how I slowly came to understand that Mum was somebody extraordinary and not to be taken for granted. I'm so glad to have been able to express my own gratitude to her in recent years.
In Henry the Fourth Shakespeare has a turn of phrase: "like bright metal on sullen ground." Mum was bright metal on sullen ground. She was solid gold.
Everyone here will have their own fond, and if you knew her well, sometimes frustrating memories of my mum. I want to focus today on those things I came to really admire in her.
We all knew that warm side of her because she gave it freely, to everyone, everywhere she went. No matter who you were you would get a sweet smile, warmth, and praise. Perhaps fewer people here will know of her strength, energy, and willpower, her humour, her infinite gratitude, and her fundamentally uncomplaining nature.
In recent years I've found inspiration in the philosophy of the stoics and it is hard not to notice strong parallels with the way mum carried herself, always ready to help, always grateful, always giving. Standing straight, or held straight, by sheer will. Until around 11 each night when the universe would gently say "ok, it's time to sleep now Sally", and she would nod off where she stood.
And then, as my father knows all too well, was woken every morning by her two alarms, one at 4:30, so that she had "time to think", and one at 5 so that she could actually wake up.
But Mum was not a stoic, she was a Christian. Even though she was tested by the Good Lord with three opinionated, and two atheist sons, her faith never waivered in the slightest, even at the very end. As with everything about her, it held fast and true and uncompromising throughout her life. I am not a Christian but I admire her persistence greatly.
I am going to try to be more like my mother.
I'm going to try to be as grateful as she was. To remember that every morning I wake, every cup of coffee, every lego spaceship built with my kids, is a gift of limited supply. To delight and give thanks as she did in the smallest of things, common or uncommon. Any time I said thank you to mum, she would brush it off. Clearly, she thought it her basic duty to this world and the people in it to give everything she had with gratitude.
The most valuable thing each of us has to give is our time. Mum always had time for people and most especially for us her sons. In this too I hope to emulate her and give more time to my own kids, and to you, my friends and family, and the other people in my life, just as she did.
She never ever complained. Never. Instead she did whatever was in her control to help other people, and sometimes even things which were well outside her control. She knew that complaining accomplishes nothing but to make you feel worse and to burden those around. Mum never burdened anybody. If I need help I will ask for it. If I can fix something I will simply fix it, just as she did.
Mum had incredible energy. It wasn't the kind of energy you get from an energy drink, or from eating high energy food, because we all know she did not do that. No, it was the energy that springs forth at 4am at the hospital after several sleepless nights when your sick kid needs to be held. Its the energy you didn't realise you had until you decided to try and overcome the fatigue with willpower. Mum made amazing use of that particular reserve, and whilst it would probably not be advisable to dip in to quite the same extent as she did, it's good to know that there is always more you can give when required.
Finally, I aspire to mum's strength. She powered joyfully 100% into everything that life handed to her, and even her last hugs were vice-like, and with a genuine smile. Again that is not because she was particularly physically strong, although she was, but because she was emotionally and spiritually strong. If a mouse like my mother can be so mighty, surely I can too. She made it apparent that it is a simple matter of choosing to be strong.
Mum and Dad's legacy speaks for itself. The life they made for themselves in this country. The lives they helped others to build. The success of their sons Dirk and Mike in building a good life for themselves, sometimes against extraordinary odds which most people will never face. Of course I also owe a huge debt to my parents for the wonderful life I enjoy today, a debt I intend to pay forward wherever I can, in my mum's spirit.
At times like this the universe can seem cold, and harsh, and unfair; but people like Mum show us that goodness and love abound in this universe. The goodness and love is in us. We get to defy the cold and the unfair. Like mum, each of us gets to choose to be strong, to love, and to make reality wonderful for each other.
Thank you Mum, for caring for me, and Dirk, and Mike, and Dad, and for Orson and Scout, and for always believing in me. Thank you for always laughing out loud at our stupid jokes. Thank you for sitting patiently and learning to code with me on our Apple IIe when I was 8. Thank you for showing me the way to be a good human. I promise I will try.
This post originally appeared on the David Walsh blog.
In this 15 minute tutorial we're going to build a simple decentralized chat application which runs entirely in a web browser.
All you will need is a text editor, a web browser, and a basic knowledge of how to save HTML files and open them in the browser.
We're going to use Bugout, a JavaScript library that takes care of the peer-to-peer networking and cryptography.
Read more posts on the subject of cryptography & decentralized systems.
As part of the TOPLAP 15th Birthday live-stream I live-coded some algorithmic rave music in Speccy:
Speccy is a browser based environment for live-coding 8-bit algorithmic rave music in ClojureScript.
You can watch the videos of everybody who participated here.