Reducing Abstraction in Code

Abstraction is something we are taught to value as programmers, and the process of finding common patterns across parts of a system is one programmers are usually good at. There is another equally important process of improving systems by collapsing redundancy and abstraction. Gilbert Simondon names this “concretization”.

Primitives

Concretization removes parts and makes machines more specific. A simple example is the abbreviation of clutter by replacing with clearer syntax. Say in Python

if available == True:
   reserve()

to

if available:
   reserve()

Or in JUnit:

assertTrue(false);

to

fail();

These are behaviour-preserving design improvements, or in other words, refactorings. They often turn up in novice programmer code or code written by people new to a language or toolset. Other primitive concretizing refactorings might be dead code removals, such as Remove Unused Parameter.

Another primitive concretization step is recognizing that a variable with a highly general type can be typed more precisely. A String, byte[] or a void* are highly general types, in that they can hold pretty much anything. Replacing with a more specific type usually relies on a precondition, either implicitly or explicitly.

int age = Integer.parseInt(ageStr);

In this case the potential throwing of NumberFormatException entails an implicit precondition. The concretizing step is the refactoring that introduces the typed variable.

Wait, but isn’t the problem with using Strings and primitive objects everywhere that they lack abstraction? Yes. They indicate that the code lacks an explicit model, or in other words, abstractions. They also indicate the code lacks concretizations – specifics from the problem domain that make it a well-focused machine. (Lacking both abstraction and concretization indicates ontological slime, a wonderful term from William Wimsatt, and perhaps the topic of another post.)

For a multi-line example of primitive concretization, consider this refactoring available when going from Java 1 to 5:

Iterator expenseIter = expenses.iterator();
while (expenseIter.hasNext()){
  Expense expense = (Expense)expenseIter.next();
  sum += expense.getExpenseValue();
}

to

for (Expense expense: expenses){
  sum += expense.getExpenseValue();
}

This mirrors the evolution of Java itself as a technical object and iteration as a technical concept. I’ve written about Simondon and the history of looping at more length elsewhere. Specialization and reduction are near-synonyms more frequently used in programming, but because of the clearer relationship to abstraction, and the connection to Simondon, I stick with concretization here, at the cost of a few more syllables. (Reification is a different concept, in my opinion.)

Interleaving Abstraction and Concretization

The adjunction of a supplementary structure is not a real progress for the technical object unless that structure is concretely incorporated into the ensemble of the dynamic system of its operation. – Simondon, Mode of Existence of Technical Objects, Mellamphy trans.

Design improvements often include both abstracting and concretizing steps. The feeling is of abstraction clearing space and concretization then making better use of it.

Michael Feathers’ use of characterization tests is an example of starting a design process with concretization.

    @Test
    public void removesTextBetweenAngleBracketPairs() {
        assertEquals("", Pattern.formatText(""));
    }

Characterization tests stabilize the function of the machine by pinning down very specific behaviors in the form of facts. This then allows a round of refactorings and rewrites. The immediate next step would often be abstracting refactorings such as Extract Method and Extract Class (naming a clump of things introduces an abstraction and an indirection).

Arlo Belshee’s Naming Is A Process also interleaves abstracting and concretizing steps.

Missing to Nonsense – Abstraction
Nonsense to Honest – Concretization
Honest to Honest and Complete – Concretization
Honest and Complete to Does the Right Thing – Abstraction
Does the Right Thing to Intent – Concretization
Intent to Domain Abstraction – Abstraction

A number of these steps, especially in the later half, themselves consist of interleaved abstracting and concretizing sub-steps. Eg in Honest and Complete:

1/ Use Introduce Parameter Object. Select just the one parameter you want to encapsulate. Name the class Foo and the parameter self. (Abstraction)
2/ Use Convert To Instance Method on the static. Select the parameter you just introduced. (Abstraction)
3/ Improve the class name (Foo) to at least the Honest level. (Concretization)
4/ Go to the caller of the method. Select the creation of the new type. Introduce parameter to push it up to the caller’s caller. (Abstraction)
5/ Convert any other uses of the parameter you are encapsulating to use the field off the new class. (Concretization)

Belshee’s process, using names as the signposts for improving code, is a wonderful combination of practical walkthrough and a theory of programming. It even seems to put living flesh on my skeletal wish for Name Oriented Software Development, though, eg, stronger tool and language support for consistent dictionaries are needed to realize the full vision.

Executable Theory

This kind of divergence of functional aims is a residue of abstract design in the technical object, and the progress of a technical object is definable in terms of the progressive reduction of this margin between functions in plurivalent structures. – Simondon, ibid

Every abstraction, even one as small as an extracted method, is also a theory. These little theories then need to be applied and refined to ensure a coherent system. What Simondon saw in the evolution of mechanical engines and other industrial era machines, we can observe at smaller scale and higher frequency when engineering in our more plastic computational material.

Simondon describes machines as becoming more concrete over time, finally reaching a highly focused state where each part cleanly supports the functions of others in an overall system. He also states that the introduction of a new theory is the invention of a new machine. So perhaps he would disagree that the process is cyclical.

We can, perhaps, reconcile this if we think of each software function or class as a small widget in a larger system. In this sense of the widget = machine = function, every new method is a new Simondonian machine. This also suggests that software rarely progresses to the refined machines he describes, but is more usually an assembly of semi-refined widgets. Which sounds about right.

Once you realise abstraction and concretization are complementary, anti-parallel processes, you start noticing it everywhere. I suspect casual design phrases like “nice abstraction” are actually misleading. Ohm’s Law is a nice abstraction; modern chips that rely on parasitic capacitance in a material context of silicon are well-built machines. In working software, a nice abstraction is also a nice concretization: a well-formed widget within a coherent machine.

All problems in computer science can be solved by another level of indirection, except of course for the problem of too many indirections. – David Wheeler

Lanfang as Frontier

The Lanfang Republic 兰芳公司 illustrates non-European, early modern, self-organizing, frontier settlement. Many of the self-reinforcing dynamics of the American West described by Frederick Turner can be sketched at work there, as can other geographic and historical contingencies that ultimately limited its expansion and led to its fall.

The Elegant Republic

The Lanfang Republic was a Chinese settlement in Borneo from 1777 to 1884. As described in Yuan Bingling’s study, Chinese settlers were originally invited by the Malay sultans of Borneo for their mining expertise. The multiple settlements founded in Borneo, of which Lanfang is the best documented, were organized politically around kongsi 公司, literally “common management”, today used as the Chinese word for corporation. The historical usage was broader and this usage survives in, eg, the organization for temple societies in Taiwan and clan associations throughout SE Asia (eg Khoo Kongsi in Penang). The translation of “republic” comes from 19th century Dutch orientalists who visited and studied these specific kongsi in Borneo.

The Chinese settlers in Borneo innovated on these existing civic institutions to construct the state infrastructure they required, around the council of the zongting 总厅.

The zongting had their own courts of law, their own financial systems, minted their own money, levied their own taxes, and maintained a number of treaties with the neighbouring Malay sultanates and Dayak tribes.
— Yuan Bingling, Chinese Democracies: A Study of the Kongsis of West Borneo

Lanfang acknowledged and paid tribute to the Qing emperor as part of the foreign policy conventions of the region, but did not depend on the Qing state in a colonial relationship. In this sense they arguably exercised more sovereignty than early American or Australian colonies.

Much of the immigration was Hakka or Hokkien, more historically mobile ethno-linguistic groups within the Han Chinese ethnicity. Hakka 客家 literally means “guest families”. Chinese communities as part of existing polities in SE Asia go back at least to the 13th century Yuan dynasty and trade links between SE Asia and China are a historic feature of the region, waxing and waning over time. 18th century immigration built on these existing trade links.

Turner, in The Significance of the Frontier in American History, describes the trader as “the pathfinder of civilization”, though in SE Asia it is the geographic interface of multiple civilizations as well as indigenous and non-state tribes. The traders were then followed by “sudden tides of adventurous miners” (reapplying Turner), followed finally by farmers. Turner is criticized as culturally imperialistic and nation-centric, and he tends to be a cheerleader for the advancing state. In this sketch we take “civilization” as just a synonym for “state-connected people”, without particularly seeing state or non-state non-state societies as better, hopefully without interfering particularly with the rest of the comparison.

James C Scott describes that in pre-modern SE Asian states, “oceans connect, mountains divide”. The frontier was not so much the ocean itself, as the more ambiguous barrier between accessible coastal settlement and mountain or jungle interiors. As in Turner’s America, “the frontier […] is not the European frontier — a fortified boundary line running through dense populations.” Scholars like Anthony Reid and William Skinner describe SE Asia generally as a place of creolization and hybridization. This creation of a new mixed culture in the new environment is a focus of Turner, for whom “the frontier is the line of most rapid and effective Americanization”. In Lanfang specifically there was linguistic creolization and intermarriage between Chinese settlers and Dayak tribes.

Stack Trace

In Lanfang and the other kongsi of Borneo, you have a Chinese maritime culture, self-reinforcing, individually-directed settler dynamics with autonomous government and innovation in civic institutions. These all parallel the American frontier experience. It is also history that the Borneo kongsi had less extent in space and time than the American frontier experience and the subsequent American nation states.

This note doesn’t rigorously address why those differences exist, but we can sketch possible answers within the framework of recurrent reinforcing processes. The arguments fall roughly into whether the settlement process lacked strength in each iteration, or whether the environment was less conducive to the process. In the Lanfang Kongsi case, the process specifically stopped when the Dutch colonial state chose to destroy the kongsi militarily, as threats to their regional interests.

Borneo is the world’s third largest island, bigger than Great Britain and Honshu combined, but it is still much smaller than continental North America. The frontier of state settlement in SE Asia generally was not exhausted in the 19th century – arguably it exists in Myanmar and Laos today – but the opportunities were not so abundant and geographically contiguous as in the enormous American Midwest. The frontier process was not so extensive in time, lasting a century instead of several centuries, so there was less recurrence, and less reinforcing feedback.

The archipelagos of SE Asia made the naval power projection of European empires more crucial. Simply being attached to a power was hardly enough to ensure success, and other centres like Melaka flipped control between Malay sultans and various European empires. It is still notable that the kongsis existed outside support by great or regional powers. They didn’t benefit from nearby similar state-sponsored immigration or colonies, the way, eg, independently founded Plymouth did from state-founded Jamestown being in the same region. The kongsis also couldn’t benefit from imperial military support, either active or reluctant entanglement. After the state sponsored voyages of Zheng He 郑和, a kind of Ming Dynasty Apollo Program of superior technological achievement for reasons of political prestige, the Chinese state gave up on naval power, and the embattled 19th century Qing was in no condition to help mining colonies on the other side of the South China Sea.

Turner describes attempts by established states to establish a defined border being undermined by the self-settling frontier process, and those empires being drawn reluctantly forward, into yet further wars, by yet further waves of pioneers. Comparing with America, the missing figure in Chinese maritime settlement in SE Asia is perhaps not the settler but the privateer.


Louise Levathes – When China Ruled the Seas: The Treasure Fleet of the Dragon Throne, 1405-1433
Anthony Reid – Hybrid Identities in the 15th-century Straits in Southeast Asia in the Fifteenth Century: The China Factor
G William Skinner – Creolized Chinese Societies in Southeast Asia in Sojourners and Settlers: Histories of Southeast Asia and the Chinese
James C Scott – The Art of Not Being Governed: An anarchist history of upland Southeast Asia
Frederick Jackson Turner – The Significance of the Frontier in American History
Yuan Bingling – Chinese Democracies: A Study of the Kongsis of West Borneo

Refactoring the Argo

Le vaisseau Argo ~ The ship Argo

A frequent image: that of the ship Argo (luminous and white), each piece of which the Argonauts gradually replaced, so that they ended with an entirely new ship, without having to alter either its name or its form. This ship Argo is highly useful: it affords the allegory of an eminently structural object, created not by genius, inspiration, determination, evolution, but by two modest actions (which cannot be caught up in any mystique of creation): substitution (one part replaces another, as in a paradigm) and nomination (the name is in no way linked to the stability of the parts): by dint of combinations made within one and the same name, nothing is left of the origin: Argo is an object with no other cause than its name, with no other identity than its form.

Another Argo: I have two work spaces, one in Paris, the other in the country. Between them there is no common object, for nothing is ever carried back and forth. Yet these sites are identical. Why ? Because the arrangement of tools (paper, pens, desks, clocks, calendars) is the same: it is the structure of the space which constitutes its identity. This private phenomenon would suffice to shed some light on structuralism: the system prevails over the very being of objects.

— Roland Barthes, Roland Barthes

The Argo, Constantine Volanakis

The Argo (luminous and white) is a software system and the Argonauts its human components. They trim its sails and move its oars; its rudder steers by the use of a helmsman.

When each piece comes in turn to be replaced, it is substituted for a better part; newer and hence luminous, but in keeping with the shape of the old. The system is rebuilt anew without an act of raw greenfield creation.

The Argonauts are skilled and far from friendly ports: they repair the ship as they sail it. This too is why one traveller will tell of the grand trireme Argo, and others of a swift catamaran, an Argo with a glorious triangular sail, growing ever swifter, ever smoother as time passes. But most will tell of strange hybrid ships, nautical chimera where names and shapes are stretched before losing now redundant pieces entirely, and reforming around a new coherence.

Some stories tell of another Argo, also sailing and perpetually repaired. When each part eventually decays and breaks, it is replaced with the most perfect imitation, in shape, strength, flex, texture and colour. Some say this Argo has a beautiful unity lacking in the first, where the bastard styles and material of past and future ships are always found together. This Argo is easier to find than the others, for though its Argonauts are strong and brave in battle, they never sail more than a few weeks from Thessaly. It is there, in the forest on the outskirts of Iolcus, that a copse of tall trees grow fast and strong. They are excellent for building triremes, and the Argonauts only source of timber. Jason has long ago died, and been replaced with other princes, with their own usurped claims. None of them have held the Golden Fleece.


(I learnt of this Barthesian image from Jenny Turner’s review of Maggie Nelson’s The Argonauts.)

Stasis Request Form

MegaCorp Industries, Technology Process Technology Department (TPT)

Name of system ______________________
MegaCorp Technology System Identifier (MCTSID) ______________________
Stasis owner name and employee ID
_
Dates requested (Note stasis periods longer than one calendar week require separate forms for each week) ______________________
System criticality rating ______________________
User population ______________________
Justification for stasis exemption (Eg natural catastrophe, medical emergency impacting > 60% of team, multi-site build chain outage)
_
_
_
Evidence supporting (please link or attach)
_
_
_
Mitigations for risks accumulated during freeze period
_
_
_
Emergency roll forward procedure
_
_
_
Size of development-support teams for system ______________________
Plan for remediation and resumption of normal releases (Please link or attach if space insufficient)
_
_
_
Line of business management approval
_
_
_
Diamond band manager approval
_
_
_
Other notes or comments
_
_
_

Stasis control board approval meetings are scheduled twice weekly. Stasis requests not represented in the meeting will be automatically rejected. A separate form is required for each request.


This was written as

  • A sarcastic reaction to experiences with established change procedures in large bureaucratic organisations.
  • A tool for framing the costs and risks of not releasing software frequently, instead accumulating large changesets so every release is inherently major and complex. Many change release procedures privilege the cost of change, rather than the cost of stasis (for instance, missed security patches). The actual work of filling out and reviewing a change request form is Risk Management Theatre, the point of which is not the content but the time tax imposed on release frequency by filling out the form, in a belief frequent releases are necessarily rushed and unsafe.
  • A document from a dromological future where continuous delivery techniques such as those used by Google and other firms are widespread through the industry and therefore institutionalized. Consider Facebook changing the way privacy settings work for the seventh time in a year, or a university administration sending one of these forms to Donald Knuth for TeX.

The Bureaucracy of Automatons

An introduction to the notes on Confucian Software.

Software and the Sage

Among the many dissimilarities between software and gentlemen of the classical Chinese Spring and Autumn Period, two in particular stand out. One existed in a pre-scientific feudal society on an agricultural technological and economic base, and the other presupposes the scientific method and a modern (or post-modern) industrial base. Secondly, the concept of virtue or potency (德) is central to The Analects, but software artifacts are, in our day and age, non-sentient. Morality requires some degree of self-awareness – of consciousness – and so software does not itself practice virtue any more than a spoon or a lawnmower.

The immediate relevance, for a developer, of the Analects, are the two other grand concerns of Confucius, which are existential fundaments of software. These are names (名), and the rites (礼).

Continue reading