Heaps of Slime

The sorites paradox is a fancy name for a stupid-sounding problem. It’s a problem of meaning, of the kind software developers have to deal with all the time, and also of the kind software generates all the time. It’s pervasive, emergent property of formal and informal languages.

You have a heap of sand. One grain of sand is not a heap. You take away one grain of sand. One grain of sand makes little difference – so you still have a heap of sand.

You have a grain of sand. You add another grain. Two grains of sand are surely not a heap. You add another. Three grains of sand are not a heap.

If you add only a grain or take away only a grain of sand, since one grain of sand can hardly make a difference, how do you tell when you have a heap?

That’s the paradox. The Stanford Encyclopedia of Philosophy has a more comprehensive historical overview.


Slime Baking

To make software, you build a machine out of executable formal logic. Let’s call that code a model, including its libraries and compiler, but excluding the software machinic layers below that.

The model has different elements which we represent in programming language structures, usually with names corresponding to our understanding of the domain of the problem. These correspond to phenomena in two ways: parsing, and delegation to an analogue instrument. Parsing is the process of structuring information using formal rules. An analogue instrument from this perspective is a thermostat, a camera, a human user, a rabbit user, or possibly some statistical or computational processes with emergent effects, like Monte Carlo simulations or machine learning autoencoders.

You can imagine any particular software system as a free-floating machine, just taking in inputs and providing outputs over time. Think of a program where all names of classes, functions, variables, button labels, etc, are replaced with arbitrary identifiers like a1, a2, etc, (which does have some correspondence to the processing happening inside a compiler, or during zip compression). We tether this symbolic system to the world by replacing these arbitrary names with ones that have representational meaning in human language, so that users and programmers can navigate the use and internals of the system, and make new versions of it.

To make it easier to understand, navigate and change this system, we label its interface and internals with names that have meaning in whatever domain we are using it for. Dairy farm systems will have things named after cows and online bookstores will have data structures representing books.

We have then delegated the problem of representation to the user of the system – a human choosing from a dropdown box, on a web form, for example, does the work of identification for the user+software system. But we run slap bang into the problem of vagueness.

Most of the users of our dairy software will not be on quaint farms in the English countryside owning one cow named Britney, so it will be necessary to represent a herd. How many cows do you need to qualify as a herd? Well, in practice, a programmer will pick a useful bucket data structure, like a set or a list, and name that variable “herd”. Nowadays it would probably be a collection in a standard library, like java.util.HashSet. The concept of an empty collection is a familiar one to programmers, furthermore there is a specific object to point to called “herd” (the new variable), so a herd is defined to be a data structure with zero or more (whole) cows. Sorites paradox solved <dusts hands>. And unwittingly too.

herd = []
# I refute it thus!

The loose, informal, family resemblance definition of a concept (herd) gets forced into a symbolic structure, like an everyday Python variable, to treat it as an object in a software system. This identification of a concept with a specific software structure is called reification. In the case of a herd (or a heap of sand) the formalism is a fairly uncontroversial net win; after getting over the slightly weird idea of the empty herd, the language will may converge around this new more formal definition, at least in the context of the system. (Or it may not. It is interesting to note the continuing popularity of the shopping cart usability metaphor, a concrete physical container that can be empty, rather than say, a pile of books that is allowed to have zero books in it.) 

The sorites might be thought of as a limiting case of vagueness, due to the deliberate simplicity of the concept involved (one type of thing, one collection of it). There are much messier cases. Keith Braithwaite points out that software is built on a foundation of universal distinguished types, and it is a constant emphasis of training in science and engineering. People without that training tend to instead organize their thinking around representative examples, and categorize by what Wittgenstein called family resemblance, ie, sharing a number of roughly similar properties. Accordingly Braithwaite suggests foregrounding examples as a shared artifact for discussion between programmers and users, and using legible, executable examples, as in Behaviour Driven Development (BDD).

Example-driven reasoning is also a survival technique in an environment lacking clearly distinguishable universal rules. Training in physical sciences emphasizes the wonderful discovery of universal physical laws such as those for gravity or electrical charge. Biologists are more familiar with domains where simple universal laws do not have sufficient explanatory power, and additional, much more local rules, are the only navigational aids possible. Which is to say, non-scientific exemplary reasoning was likely rational in the context it evolved in, and additionally, there are many times in science and engineering when we can not solve problems using universal rules. William Wimsatt names these conditions of highly localized rules “ontological slime”, and the complex feedback mechanisms that accompany them “causal thickets”. He points out that even if you think an elegant theory of everything is somehow possible, we have to deal with the world today, where there definitely isn’t one to hand, but ontological slime everywhere.

Readers who have built software for organizations may see where this is going. It’s not that (fairly) universal rules are unknown to organizations, but that rules run the gamut from wide generality right down to ontological slime, with people in organizations usually navigating vagueness by rule-of-thumb and exemplar-based categories which don’t form distinguished types. Additionally, well-organized domains of knowledge often intersect in organizations in idiosyncratic ways. For example, a hospital has chemical, electrical and water systems, many different medical domains, radioactive and laser equipment, legal and regulatory codes, and financial constraints to work within. And so the work of software development proceeds, one day accidentally solving custom sorites paradoxes, the next breaking everything by squeezing a twenty-nine sided Escher tumbleweed peg into a square hole.



For software applications written for a domain, especially, software acts as a model to the world. This relation even holds for a great deal of “utility” software – software that other software is built on. An operating system needs to both use and provide functions dealing with time, for example, which has a lot more domain quirks than you might at first think.

Model is a specific jargon term in philosophy of science, and the use here is deliberate. For most software, the software : world relation is a close relative of the model : world relation in science. The image of code running without labels, untethered to the world, above, is an adaptation of an image from philosopher Chuang Liu: a map, showing only a selected structure, without labels or legend. We use natural language in all its power and ambiguity to attach labels to structures. This relation is organized according to a theory. Michael Weisberg calls the description, in the light of the theory, of how the world maps and doesn’t map to the model, a construal. Unlike scientific theories, the organizing theory for a software application is rarely carefully stated or specifically taught. So individual users and programmers build their own specific theory of the system as they work, and their own construals to go with them.

Software is not just a model: it’s also an instrument through which users act. The world it models is changed by its use, much more directly than for scientific models. Most observably, the world changes to be more like the model in software. Software also changes frequently. New versions chase changes in the world, including those conditioned by earlier versions of the software, in a feedback spiral. (Donald Mackenzie calls this “Barnesian performativity” when discussing economic models, the CCRU called it “hyperstition” when discussing fiction, and Brian Goetz and friends call it “an adventure in iterative specification discovery” when discussing programming.)

It is this feedback spiral which can eliminate ambiguity in terms by identifying them with exactly their use in software, therefore solving the sorites paradox in a stronger sense. It becomes meaningless to talk about an artifact outside its software context. We don’t argue about whether we have a pile of email, as it is obvious it is a container with one limit at Inbox Zero. This is one sense in which software can be said to be “eating the world”: by realigning the way a community sees and describes it.

There are other forms of software / language / world feedback, including ones that destroy meaning, dissolve formal definitions and create ambiguity. It’s often desirable, but perhaps not always, to collapse definitions into precise model-instrumented formality. Reifying an ambiguous concept by collapsing a sorites paradox into a concrete machine component is simply one process to be aware of when building software; an island of sediment in a river of slime.


Braithwaite – Things: how we think of them, what that means for building systems https://www.darkpeakconsulting.co.uk/blog/things-how-we-think-of-them-what-that-means-for-building-systems
Goetz et al – Java Concurrency In Practice
Hyde and Raffman – Sorites Paradox https://plato.stanford.edu/entries/sorites-paradox/
Liu – Fictionalism, Realism, and Empiricism on Scientific Models http://philsci-archive.pitt.edu/11162/
Mackenzie – An Engine, Not A Camera: How Financial Models Shape Markets
Visee – Falsehoods Programmers Believe About Time https://gist.github.com/timvisee/fcda9bbdff88d45cc9061606b4b923ca
Weisberg – Simulation and Similarity
Wimsatt – Re-Engineering Philosophy For Limited Beings


Bardo Birdsong

George Saunders has written a great sentimental inhumanist novel. The book comes at you time-sorted and many-voiced, like the chat room history of channel #civilwargraveyard. In Lincoln in the Bardo, messages come in slices, and names come uncapitalized, like a child’s signature, or a Twitter handle. Even the blocks of interspersed historical (or pretend-historical) text that ground the story have the feel of a link followed, or a long block quote shared as a photo, as those on bookish corners of the platform might recognize.

Amy Ireland has the best description, from 2016, of the sliced up, liminal design affordances of the birdsite, and so this novel:

Twitter is excellent. The botlife runs wild and free, swerving into sheer paranoia-inducing bizarreness at times (Weird Sun Twitter) and there are writers doing really innovative work that engages directly with the unique formal possibilities of the medium (Uel Aramchek’s ‘This Could Be Your Past’ is one of my favourite recent examples). It’s the Arcadia of human/bot collaboration.


Only here we have a scroll updated to capitalise on the possibilities of hypertextuality: it is effectively nonlinear yet accommodates series of interlinked tweets, its citation system harbours abyssal potential for embedded referencing, its search function and the public nature of its contents make for a vast and bizarre dataset […], and it forces the honing of expression to a compact 140 characters Per unit of information. […]

During its first exciting moments, Twitter appears as an open horizon for the accumulation of all sorts of gratifying information, […] Nevertheless, the illusion of accumulation inevitably breaks down and it does so in perfect correspondence with the intensity of one’s Twitter habit. Accumulation cycles pathologically into dispersion, and before you recognise what is occurring, the mesmeric infinity of the digital scroll has entirely voided your capacity to focus or reflect. There is nowhere to go but further into the abyss.

If one could allot a genre to the platform as a whole, Twitter would be horror. The interface manifests visually and cognitively as a series of incisions. What begins as a benign mode of textual organisation quickly becomes applicable to human concentration. Its twentieth century prototype can perhaps be found in the mechanical writing/torture machine of Kafka’s In the Penal Colony. Both oversee the virulent machining of the human through text, and both tend towards a similar outcome in which the relentless numerical insistence of machinic agency ultimately succeeds in eradicating the latter.

Poetry is Cosmic WarAJ Carruthers interview with Amy Ireland

Klee - Die Zwitscher Maschine (Twittering Machine)

Klee – Die Zwitscher Maschine (Twittering Machine)

The bardo is an intermediate state between death and rebirth in Tibetan Buddhism, a purgatorial place where we are separated from ties to mortal lives. So the Bardo Thodol, the Tibetan Book of the Dead, is more literally translated Liberation Through Hearing In The Intermediate State. Saunders populates the bardo with ghosts, imprisoned within the frame of a Washington graveyard, lost in scripts of their former lives, niggling at their traumas without accepting the central fact that they are dead.

The story follows the ghost of the boy Willie Lincoln, and the imagined aftermath of his sad death of typhoid at eleven years old. Eleven years old, that transitional age; “A sunny child, dear & direct, abundantly open to the charms of the world.” The talking, however, is largely done by more experienced graveyard spirits. There are quite a number – slave women and plantation owners, soldiers and farmwives – but with three men foregrounded: Hans Vollman, Roger Bevins III, and the Reverend Everly Thomas. Now with ghosts, Buddhism, death, presidents, Christianity, the US civil war, and what not, there’s a vast swathe of cultural allusions you could be drawing from.But I found myself most reminded of Journey To The West 西游记.

Delving into spoilery detail, three imprisoned spirits become disciples to a younger mortal, after a bit of ear-boxing encouragement at the start. Following his teachings and example, they protect him on his long journey, saving him from many demons intent on eating his flesh. Though they possess great magical power, when they get really stuck they need to call on Guanyin 观音, the bodhisattva goddess of mercy, to tip the scales a bit in their favour, and in the end they are released to positions of worth and enlightenment.  In this mapping, Willie Lincoln is the monk Xuanzang 玄奘 (Tripitaka), another real historical figure. The three disciples represent different virtues and sins. The Reverend Everly Thomas is the devout and overserious Sandy 沙悟净. Roger Bevins III is Pigsy Eight-sins 猪八戒, consumed by earthly gluttony and lusts, immersed in senses, always growing new eyes, ears and noses. Hans Vollman is the Monkey King 孙悟空, with a more than usually explicitly phallic giant red staff. ((You can even link the names – Vollman – Full Man – 悟空 – 无空 / Without Space, though I’m not sure if anyone really puns in three languages outside Hong Kong.))

Which makes Abraham Lincoln Guanyin. The One Who Perceives The Sounds of The World. Lincoln, in this mythic shape, is too large to fit onstage for long. We see his shaking grief through the eyes of the spirits, and then he leaves. He is the only character who re-enters and re-exits the graveyard. The Goddess of Mercy. The Great Emancipator.

Of course Lincoln was not just the rail splitter and the breaker of slavechains. He’s also the Doctor Frankenstein of the American body politic, stitching the dismembered states together for reanimation. Both George Saunders and Amy Ireland talk of writing as sampling and reassembling snippets from overwhelming torrents of data. Saunders describes it as curation: “I’d be in my room for six or seven hours, cutting up bits of paper with quotes and arranging them on the floor”, he tells Zadie Smith. Ireland notes that “the diminishment of human authorship plunges the human reader into a problematics of scale. … In response, less linear and sedentary methods of reading start to take precedence – techniques akin to scanning, scrolling, and – for the unashamedly hyperstimulated – spritzing.” In assembling his novel, Saunders does this for us across the corpus of civil war history, Lincoln biography, Sino-Tibetan Buddhism and his own imagination. Yet it still shows the zigzag path across that vast field more honestly and artfully than most novels. The omniscient narrator is replaced with the hyperstimulated archaeologist of the past-saturated present, asynchronously replayed by the reader at a rate just slow enough to allow understanding.

Lincoln’s mutated industrial union doesn’t fit in the novel’s timeline. The reader and the characters are severed from it by a bullet and the matterlightblooming phenomenon of a bound book’s last page. The sensory systems of the brain cut down, sample, pre-processes, and outright alter everything we see and hear. Our machines and our spirits do the same. There’s too much data for human consciousness to comprehend. Wasn’t there always?

The Road Not Taken, No Not That One, You Know, The Other One

1961. Robert McNamara, the recently installed Secretary of Defense in President John Fitzgerald Kennedy’s cabinet, sits in his Pentagon office at a nine-foot-long mahogany desk, polished to a mirrorlike shine. He is wearing a dark suit, his thick brown hair slicked back on his head and parted in the middle, old fashioned wire-rim spectacles framing his mirthless eyes, his jaw clenched tight, a severe expression on his face, looking very much the Presbyterian elder he is.

The door opens, and Edward Lansdale, career spy and counterinsurgency specialist, enters. He has handsome, movie star good looks and a neat moustache. A look of determination is on his face, with a hint of a maverick smile, and he is carrying a box of what seem to be weapons and camping gear, caked with mud and even blood.

MCNAMARA is making notes on some graph paper.

MCNAMARA: [Stiffly] Colonel Lansdale, good to see you.

LANSDALE: Good morning, Mr Secretary.

MCNAMARA: Lansdale, we’re doing a systems analysis on our policy in South Vietnam. I understand you’ve just returned from a trip there and I need your report on the situation. You have nine and a half minutes for this briefing.

LANSDALE strides over to the desk and upends the content of the box. Out spill handmade pistols and knives, old French rifles, and bamboo punji sticks, all over the desk with a clatter, with a few falling on the floor.

LANSDALE: The enemy in Vietnam uses these weapons – and they were using them just a little bit ago before I got them. Many of them are barefoot or wear sandals. They wear black pajamas, usually, with tatters or holes in them. I don’t think you’d recognize any of them as soldiers, but they think of themselves that way. The people that are fighting there, on our side, are being supplied with our weapons and uniforms and shoes and all of the best that we have; and we’re training them. Yet, the enemy is licking our side. Always keep in mind about Vietnam, that the struggle goes far beyond the material things of life. It doesn’t take weapons and uniforms and lots of food to win. It takes something else, ideas and ideals, and these guys are using that something else. Let’s at least learn that lesson.

MCNAMARA stares at his soiled desk, blinking.

MCNAMARA: I see. [Stands up.] Colonel Lansdale, you can’t substitute emotions for reason.

LANSDALE: [Chuckles] It substituted just fine when we made those Marxists on Luzon think their villages were attacked by vampires.

MCNAMARA, somewhat fussily, fishes out his graph paper and pencil from under the weapons and other junk now on his desk. He walks to a clearer part of the desk and places it down.

MCNAMARA: Lansdale. As I said before, we are performing an extensive systems analysis on the situation in South and North Vietnam, and very much need to capture all the factors at play. I’m not sure if you are familiar with systems analysis. This is a process we used when I was an executive at Ford Motors. I have a list of seventy three factors our staff have so far found, including food supply, ammunition, rice production, oil imports, and so on. We’ll crunch the numbers, and once the analysis is complete, the output of the model will give us a clear path to victory.

LANSDALE glances at the list.

LANSDALE: Mr Secretary, your list is incomplete. You’ve left out the most important factor of all.

MCNAMARA: What is it?

LANSDALE: Well, it’s the human factor. You can put it down as the X factor.

MCNAMARA writes down “X Factor” on the graph paper.

MCNAMARA: What does it consist of?

LANSDALE: What the people out on the battlefield really feel; which side they want to see win and which side they’re for at the moment. That’s the only way you’re going to ever have this war decided.

MCNAMARA: Ah. Good point, but we’ve got that actually. Over here, see: “Volunteer signups”, “Ho Chi Minh “uncle-ization” ratio”, “Negative reviews on Saigon embassy facebook page”, “Mao Zedong cat pun frequency”, “GI sales of Conrad short stories”.

LANSDALE: I see. Well, what about the V factor?

MCNAMARA writes down “V Factor” on the graph paper.

MCNAMARA: What does it consist of?

LANSDALE: Vampires.

MCNAMARA: Ok. Ah, we’ll make sure to give that, uh, the appropriate weighting.

LANSDALE: Mr Secretary, there’s no mathematical formula for the human spirit.

MCNAMARA: That’s true. Some of those smart IBM boys we seconded from Cambridge found that a fifteen dimension vector including poetry writing and fish sauce consumption was a passable proxy in the Indochinese context, though.

LANSDALE: Oh, ok. Well that sounds just dandy. Don’t forget there are three brands of fish sauce popular in the south though – you should really track the lot.

MCNAMARA: Interesting. I’ll put those IBM boys onto it.


MCNAMARA: Good. This has been really useful, but I’ve got something else to do now. Oh, by the way, the intern you put on the Saigon embassy Twitter account is going great. The analytics are through the roof!

LANSDALE: [Snaps fingers, points back to MCNAMARA and smiles winningly.] I’ll pass it on.

MCNAMARA: Thanks Ed.

LANSDALE: Thanks Bob.

Rather loosely adapted from Chapter 22 of Max Boot’s recent biography of Lansdale, The Road Not Taken.


Boot – The Road Not Taken: Edward Lansdale and the America Tragedy In Vietnam
Brecher, Ames – War Nerd Podcast Episode 39
McNamara – In Retrospect
Wintermute, Boot – Max Boot interview on The Road Not Taken

Kafka As Deterritorializing Stream Function

Sometimes he accompanied her on her errands in the city, where everything had to be carried out in the utmost hurry. Then she would almost run to the next subway station, Karl with her bag in his hand, the journey went by in a flash, as if the train were being carried away without any resistance, already they were getting off, clattering up the stairs instead of waiting for the elevator that was too slow for them, the large squares from which the streets flowed out in a starburst emerged and brought a tumult of streamed lines of traffic from all sides, but Karl and Therese hurried, tightly together, to the different offices, cleaners, warehouses and stores which weren’t easy to contact by telephone in order to make orders or complaints, generally trivial things.

– Kafka, Amerika, Hofman trans.

“No one is better than Kafka at differentiating the two axes of the assemblage and making them function together,” say Deleuze and Guattari, and though they are referring to the Czech writer, it applies quite well to the open source distributed message queue too, even though the quote was written thirty years before its invention. Kafka in either form effects both decoupling of components and a disintegration of content. Models, formats and codes are first broken down, then made available to be reassembled in other ways.

I admit that when I first heard of it, naming an information processing system after a writer famous for depictions of absurd, violent and impenetrable bureaucracy did strike me as bold. It evokes The Departure (Der Aufbruch) as router documentation, or An Imperial Message (Eine kaiserliche Botschaft) as a Service Level Agreement. Perhaps we can think of the system developers as finally addressing Franz’s eloquent, frustrated, bug reports. Jay Kreps, the namer of Apache Kafka and one of the co-creators (with Narkhede and Rao), simply explains that he wanted his high performance message queue to be good at writing, so he named it after a favourite prolific writer. Fortunately complete publication and reading doesn’t also require the process dying of tuberculosis, followed by a decades-long legal case. Even if Kreps did have a deeper correspondence in mind, if I were forced to explain the name all the time, I might just smile and point at my Franz fridge magnet too.

The D/G quote is from the Postulates of Linguistics chapter, concerned with the way meaning is imposed on communicating agents and their intertwining systems.

On a first, horizontal, axis, an assemblage comprises two segments, one of content, one of expression. On the one hand it is a machinic assemblage of bodies, of actions and passions, an intermingling of bodies reacting to one another; on the other hand it is a collective assemblage of enunciation, of acts and statements, of incorporeal transformations attributed to bodies. Then on a vertical axis, the assemblage has both territorial sides, or reterritorialized sides, which stabilize it, and cutting edges of deterritorialization, which carry it away. No one is better than Kafka at differentiating the two axes of the assemblage and making them function together.

– Deleuze and Guattari, November 30, 1923: Postulates of Linguistics, A Thousand Plateaus

Deleuze and Guattari are secret pomo management consultants at heart, and as their dutiful intern I have accordingly expressed the great men’s vision as an Ansoff Matrix slide for distribution to valued stakeholders.

Compare this Jay Kreps slide from Strange Loop 2015, itself entirely representative of a million whiteboard sketches accompanying middleware everywhere:

For middleware, what D/G call the cutting edges of deterritorialization, we might call a payload codec. The data structure used within the producing process is disassembled, scrambled into a bucket of bytes, then carried away along a line of flight – in this case a Kafka topic. A Kafka topic is a transactional log, a persistent multi-reader queue where the removal policy is decoupled from reader delivery, and retention is instead controlled by time or storage space windows. (Blockchains are public transaction logs optimised for distributed consensus and no retention limit. Hence their inherent parliamentary slowness.) The consumer then uses its own codec to reterritorialize the data – making it intelligible according to its own data model, and within its own process boundary. Though nowadays the class signature of the messages may match (say both sides use the JVM and import the definition from the same library), at a bare minimum the relationship of those messages to other objects and functions within the process differs.

In Anti-Oedipus, D/G call reading a text “productive use of the literary machine”, and it’s along those lines that the quote continues:

No one is better than Kafka at differentiating the two axes of the assemblage and making them function together. On the one hand, the ship-machine, the hotel-machine, the circus-machine, the castle-machine, the court-machine, each with its own intermingled pieces, gears, processes, and bodies contained in one another or bursting out of containment (see the head bursting through the roof). On the other hand, the regime of signs or of enunciation: each regime with its incorporeal transformation, acts, death sentences and judgements, proceedings, “law”. […] On the second axis, what is compared or combined of the two aspects, what always inserts one into the other, are the sequenced or conjugated degrees of deterritorialization, and the operations of reterritorialization that stabilize the aggregate at a given moment. K., the K.-function, designates the line of flight or deterritorialization that carries away all of the assemblages but also undergoes all kinds of reterritorializations and redundancies – redundancies of childhood, village life, bureaucracy, etc.

 – Deleuze and Guattari, ATP ibid.

Cataloguing these correspondences between D/G’s description of Kafka and the software that bears his name is not intended to ignore that their contact is a kind of iconographic car accident. Kafka is a famous, compelling writer, and frequent cultural reference point, after all. The collision of names reveals structural similarities that are usually hidden.

In the Kreps talk above, titled in Deleuzian fashion “Apache Kafka and the Next 700 Stream Processing Systems”, he also introduces a stream processing API to unify the treatment of streams and tables. The team saw this as crucial to Kafka’s identity as a streaming platform rather than just a queue, and delayed calling Kafka 1.0 for years, until this component was ready. Nomadic messages escape through the smooth stream space, before capture and transformation in striated tablespace as rows in data warehouses.

First version of Tetris.

This unification of batches and streams echoes a similar call in computational theory by Eberbach. Turing computation is built around batches. Data is available in complete form at input on the Turing machine tape, then the program runs, and if it terminates, a complete output is available on the same tape. The theory of computability and complexity are built around this same encapsulated box of space and time. Much of what computers do in 2018 is actually continual computation – the reacting to events or processing streams of data that have no semantically tied termination point. That is, though the process may terminate, that isn’t particularly relevant to any analysis of computational complexity or performance we want to do. When editing a document on a computer, you care about the responsiveness keystroke to keystroke, not the entire time editing the document as if it were one giant text batch.  

By contrast, modern computing systems process infinite streams of dynamically generated input requests. They are expected to continue computing indefinitely without halting. Finally, their behavior is history-dependent, with the output determined both by the current input and the system’s computation history.

 – Eberbach, Goldin and Wegner – Turing’s Ideas and Models of Computation

Streams are a computational model of continuation, and therefore infinity. In their wide-ranging 2004 paper, Eberbach and friends go on to argue for models of Super-Turing computation. This includes alternative theoretical models such as the π-calculus and the $-calculus, new programming languages, and hardware architectures.

We conjecture that a single computer based on a parallel architecture is the wrong approach altogether. The emerging field of network computing, where many autonomous small processes (or sensors) form a self-configured network that acts as a single distributed computing entity, has the promise of delivering what earlier supercomputers could not.

 – Eberbach et al, ibid.

These systems are now coming into existence. Through co-ordination with distributed registries (Zookeeper here), and with the improved deployment and configuration baseline devops techs have brought in, spinning up new nodes or failing over existing nodes is autonomously self-configured. Complete autonomy isn’t there, but it seems a high and not particularly desirable bar for many systems. Distributed systems and streams predate Apache Kafka, nor shall it be the last one. Yet is marks a moment where separate solutions for managing streams, tables and failover are concretized in a single technical object, a toolbox for the streaming infrastructure of infinity.


CCRU – Ccru Writings 1997-2003
Cremin – Exploring Videogames with Deleuze and Guattari: Towards an affective theory of form
Deleuze and Guattari – Anti-Oedipus
Deleuze and Guattari – A Thousand Plateaus
Eberbach, Goldin and Wegner – Turing’s Ideas and Models of Computation
Kafka, Hofman trans. – Amerika (The Man Who Disappeared)
Kreps, Narkhede and Rao – Kafka – A Distributed Messaging System For Log Processing
Kreps – Apache Kafka and the Next 700 Stream Processing Systems (talk)
Narkhede – Apache Kafka Goes 1.0 https://www.confluent.io/blog/apache-kafka-goes-1-0/
Pajitnov – Tetris (game)
Stopford – The Data Dichotomy: Rethinking the Way We Treat Data and Services https://www.confluent.io/blog/data-dichotomy-rethinking-the-way-we-treat-data-and-services/
Thereska – Unifying Stream Processing and Interactive Queries in Apache Kafka https://www.confluent.io/blog/unifying-stream-processing-and-interactive-queries-in-apache-kafka/

Pogromon Go

It’s not fair to blame it on Pokémon, really. They weren’t the worst of it by far, though the limited edition Pikajew did echo the unfortunate spirit of the times.

The Pure Tribe had their own apps, monsters and backends. Variations. Innovations. Memes and games catch on and evolve. The whole time they were annotating and mapping. Highlighting targets in bold colours: signs, shrines, grandmothers, foxes. Sharing high scores on the day. Trading in parts. People and drones had got in plenty of practice beforehand. “Gotta catch ‘em all!”

Fads come and go.