Lanfang as Frontier

The Lanfang Republic 兰芳公司 illustrates non-European, early modern, self-organizing, frontier settlement. Many of the self-reinforcing dynamics of the American West described by Frederick Turner can be sketched at work there, as can other geographic and historical contingencies that ultimately limited its expansion and led to its fall.

The Elegant Republic

The Lanfang Republic was a Chinese settlement in Borneo from 1777 to 1884. As described in Yuan Bingling’s study, Chinese settlers were originally invited by the Malay sultans of Borneo for their mining expertise. The multiple settlements founded in Borneo, of which Lanfang is the best documented, were organized politically around kongsi 公司, literally “common management”, today used as the Chinese word for corporation. The historical usage was broader and this usage survives in, eg, the organization for temple societies in Taiwan and clan associations throughout SE Asia (eg Khoo Kongsi in Penang). The translation of “republic” comes from 19th century Dutch orientalists who visited and studied these specific kongsi in Borneo.

The Chinese settlers in Borneo innovated on these existing civic institutions to construct the state infrastructure they required, around the council of the zongting 总厅.

The zongting had their own courts of law, their own financial systems, minted their own money, levied their own taxes, and maintained a number of treaties with the neighbouring Malay sultanates and Dayak tribes.
— Yuan Bingling, Chinese Democracies: A Study of the Kongsis of West Borneo

Lanfang acknowledged and paid tribute to the Qing emperor as part of the foreign policy conventions of the region, but did not depend on the Qing state in a colonial relationship. In this sense they arguably exercised more sovereignty than early American or Australian colonies.

Much of the immigration was Hakka or Hokkien, more historically mobile ethno-linguistic groups within the Han Chinese ethnicity. Hakka 客家 literally means “guest families”. Chinese communities as part of existing polities in SE Asia go back at least to the 13th century Yuan dynasty and trade links between SE Asia and China are a historic feature of the region, waxing and waning over time. 18th century immigration built on these existing trade links.

Turner, in The Significance of the Frontier in American History, describes the trader as “the pathfinder of civilization”, though in SE Asia it is the geographic interface of multiple civilizations as well as indigenous and non-state tribes. The traders were then followed by “sudden tides of adventurous miners” (reapplying Turner), followed finally by farmers. Turner is criticized as culturally imperialistic and nation-centric, and he tends to be a cheerleader for the advancing state. In this sketch we take “civilization” as just a synonym for “state-connected people”, without particularly seeing state or non-state non-state societies as better, hopefully without interfering particularly with the rest of the comparison.

James C Scott describes that in pre-modern SE Asian states, “oceans connect, mountains divide”. The frontier was not so much the ocean itself, as the more ambiguous barrier between accessible coastal settlement and mountain or jungle interiors. As in Turner’s America, “the frontier […] is not the European frontier — a fortified boundary line running through dense populations.” Scholars like Anthony Reid and William Skinner describe SE Asia generally as a place of creolization and hybridization. This creation of a new mixed culture in the new environment is a focus of Turner, for whom “the frontier is the line of most rapid and effective Americanization”. In Lanfang specifically there was linguistic creolization and intermarriage between Chinese settlers and Dayak tribes.

Stack Trace

In Lanfang and the other kongsi of Borneo, you have a Chinese maritime culture, self-reinforcing, individually-directed settler dynamics with autonomous government and innovation in civic institutions. These all parallel the American frontier experience. It is also history that the Borneo kongsi had less extent in space and time than the American frontier experience and the subsequent American nation states.

This note doesn’t rigorously address why those differences exist, but we can sketch possible answers within the framework of recurrent reinforcing processes. The arguments fall roughly into whether the settlement process lacked strength in each iteration, or whether the environment was less conducive to the process. In the Lanfang Kongsi case, the process specifically stopped when the Dutch colonial state chose to destroy the kongsi militarily, as threats to their regional interests.

Borneo is the world’s third largest island, bigger than Great Britain and Honshu combined, but it is still much smaller than continental North America. The frontier of state settlement in SE Asia generally was not exhausted in the 19th century – arguably it exists in Myanmar and Laos today – but the opportunities were not so abundant and geographically contiguous as in the enormous American Midwest. The frontier process was not so extensive in time, lasting a century instead of several centuries, so there was less recurrence, and less reinforcing feedback.

The archipelagos of SE Asia made the naval power projection of European empires more crucial. Simply being attached to a power was hardly enough to ensure success, and other centres like Melaka flipped control between Malay sultans and various European empires. It is still notable that the kongsis existed outside support by great or regional powers. They didn’t benefit from nearby similar state-sponsored immigration or colonies, the way, eg, independently founded Plymouth did from state-founded Jamestown being in the same region. The kongsis also couldn’t benefit from imperial military support, either active or reluctant entanglement. After the state sponsored voyages of Zheng He 郑和, a kind of Ming Dynasty Apollo Program of superior technological achievement for reasons of political prestige, the Chinese state gave up on naval power, and the embattled 19th century Qing was in no condition to help mining colonies on the other side of the South China Sea.

Turner describes attempts by established states to establish a defined border being undermined by the self-settling frontier process, and those empires being drawn reluctantly forward, into yet further wars, by yet further waves of pioneers. Comparing with America, the missing figure in Chinese maritime settlement in SE Asia is perhaps not the settler but the privateer.

Louise Levathes – When China Ruled the Seas: The Treasure Fleet of the Dragon Throne, 1405-1433
Anthony Reid – Hybrid Identities in the 15th-century Straits in Southeast Asia in the Fifteenth Century: The China Factor
G William Skinner – Creolized Chinese Societies in Southeast Asia in Sojourners and Settlers: Histories of Southeast Asia and the Chinese
James C Scott – The Art of Not Being Governed: An anarchist history of upland Southeast Asia
Frederick Jackson Turner – The Significance of the Frontier in American History
Yuan Bingling – Chinese Democracies: A Study of the Kongsis of West Borneo

Refactoring the Argo

Le vaisseau Argo ~ The ship Argo

A frequent image: that of the ship Argo (luminous and white), each piece of which the Argonauts gradually replaced, so that they ended with an entirely new ship, without having to alter either its name or its form. This ship Argo is highly useful: it affords the allegory of an eminently structural object, created not by genius, inspiration, determination, evolution, but by two modest actions (which cannot be caught up in any mystique of creation): substitution (one part replaces another, as in a paradigm) and nomination (the name is in no way linked to the stability of the parts): by dint of combinations made within one and the same name, nothing is left of the origin: Argo is an object with no other cause than its name, with no other identity than its form.

Another Argo: I have two work spaces, one in Paris, the other in the country. Between them there is no common object, for nothing is ever carried back and forth. Yet these sites are identical. Why ? Because the arrangement of tools (paper, pens, desks, clocks, calendars) is the same: it is the structure of the space which constitutes its identity. This private phenomenon would suffice to shed some light on structuralism: the system prevails over the very being of objects.

— Roland Barthes, Roland Barthes

The Argo, Constantine Volanakis

The Argo (luminous and white) is a software system and the Argonauts its human components. They trim its sails and move its oars; its rudder steers by the use of a helmsman.

When each piece comes in turn to be replaced, it is substituted for a better part; newer and hence luminous, but in keeping with the shape of the old. The system is rebuilt anew without an act of raw greenfield creation.

The Argonauts are skilled and far from friendly ports: they repair the ship as they sail it. This too is why one traveller will tell of the grand trireme Argo, and others of a swift catamaran, an Argo with a glorious triangular sail, growing ever swifter, ever smoother as time passes. But most will tell of strange hybrid ships, nautical chimera where names and shapes are stretched before losing now redundant pieces entirely, and reforming around a new coherence.

Some stories tell of another Argo, also sailing and perpetually repaired. When each part eventually decays and breaks, it is replaced with the most perfect imitation, in shape, strength, flex, texture and colour. Some say this Argo has a beautiful unity lacking in the first, where the bastard styles and material of past and future ships are always found together. This Argo is easier to find than the others, for though its Argonauts are strong and brave in battle, they never sail more than a few weeks from Thessaly. It is there, in the forest on the outskirts of Iolcus, that a copse of tall trees grow fast and strong. They are excellent for building triremes, and the Argonauts only source of timber. Jason has long ago died, and been replaced with other princes, with their own usurped claims. None of them have held the Golden Fleece.

(I learnt of this Barthesian image from Jenny Turner’s review of Maggie Nelson’s The Argonauts.)

Stasis Request Form

MegaCorp Industries, Technology Process Technology Department (TPT)

Name of system ______________________
MegaCorp Technology System Identifier (MCTSID) ______________________
Stasis owner name and employee ID
Dates requested (Note stasis periods longer than one calendar week require separate forms for each week) ______________________
System criticality rating ______________________
User population ______________________
Justification for stasis exemption (Eg natural catastrophe, medical emergency impacting > 60% of team, multi-site build chain outage)
Evidence supporting (please link or attach)
Mitigations for risks accumulated during freeze period
Emergency roll forward procedure
Size of development-support teams for system ______________________
Plan for remediation and resumption of normal releases (Please link or attach if space insufficient)
Line of business management approval
Diamond band manager approval
Other notes or comments

Stasis control board approval meetings are scheduled twice weekly. Stasis requests not represented in the meeting will be automatically rejected. A separate form is required for each request.

This was written as

  • A sarcastic reaction to experiences with established change procedures in large bureaucratic organisations.
  • A tool for framing the costs and risks of not releasing software frequently, instead accumulating large changesets so every release is inherently major and complex. Many change release procedures privilege the cost of change, rather than the cost of stasis (for instance, missed security patches). The actual work of filling out and reviewing a change request form is Risk Management Theatre, the point of which is not the content but the time tax imposed on release frequency by filling out the form, in a belief frequent releases are necessarily rushed and unsafe.
  • A document from a dromological future where continuous delivery techniques such as those used by Google and other firms are widespread through the industry and therefore institutionalized. Consider Facebook changing the way privacy settings work for the seventh time in a year, or a university administration sending one of these forms to Donald Knuth for TeX.

The Bureaucracy of Automatons

An introduction to the notes on Confucian Software.

Software and the Sage

Among the many dissimilarities between software and gentlemen of the classical Chinese Spring and Autumn Period, two in particular stand out. One existed in a pre-scientific feudal society on an agricultural technological and economic base, and the other presupposes the scientific method and a modern (or post-modern) industrial base. Secondly, the concept of virtue or potency (德) is central to The Analects, but software artifacts are, in our day and age, non-sentient. Morality requires some degree of self-awareness – of consciousness – and so software does not itself practice virtue any more than a spoon or a lawnmower.

The immediate relevance, for a developer, of the Analects, are the two other grand concerns of Confucius, which are existential fundaments of software. These are names (名), and the rites (礼).

Continue reading

Early Thoughts On Large Scale Scrum

Some people lose their sense of perspective. I’ve lost my sense of scale. – Will Self, Scale

We learn, in computing, that choice of data structure determines the algorithms one can run, their complexity, and the corresponding performance characteristics of an application. To talk of an algorithm without a data structure is to risk meaninglessness, for the two exist in a system, like the hub and spoke of a bicycle wheel. Likewise, Scrum and Scrum-like techniques are software processes, but they are just as fundamentally about changing team structure as about the various meetings and practices then enabled. Waterfall processes are pipelines of handoff between pinned threads organized by a task allocator-manager. Scrum is a common thread pool pulling work from a priority queue, then creating changes to a shared blackboard of state, which a product manager thread reacts to by adding or editing tasks on the queue. It’s model-view-controller: the code is the model, the production system is the view, and the product owner is the controller.

When organizations find success with agile, or Scrum, or something they call that which improves their delivery, they might reasonably want to expand on that success, simply because many organizations are larger than a team of seven people. Hence the demand for agile processes at scale. The demand, or criticism, of agile techniques ignoring large teams has been there for pretty much the whole history of the movement, but fifteen years in, we now have some road tested processes that try to address it directly. One of them is Large Scale Scrum (LeSS), and I had the good fortune this year to be introduced to it by one of its creators, Bas Vodde. That three day course and a bit of reading is the main source for these notes. There’s a lot of good material on the LeSS site.

Vodde and Larman say that Large Scale Scrum is Scrum, with some justification. There are various ways to combine small teams in delivering solutions for a client; you can federate, you can have cross cutting program managers and architects of some kind, create product governance committees, and so on. LeSS chooses to scale beyond one team working on a product by keeping the single product work queue pattern, and increasing the thread pool. The worker thread that pulls issues is now a team instead of a developer. That’s it. There is still only one product owner. Everyone in the organization works in the same sprint.

As in Scrum, everything else about the process is a support activity to facilitate humans maintaining that queue data structure. For scrum masters this is usually a combination of presiding over Scrum rituals and malediction of alternative work channels born of earlier software traditions – destruction of Old Ways, like long specification documents or individual users pushing their own priorities on particular developers. Other functions such as managers and sales exist not to direct development tasks but to secure resources from the larger organizational and commercial milieu.

This focus on the product at the heart of the software organization gives a rather existential question great importance: what exactly is the product? Why, exactly, is your organization here? What is the meaning of work life? Vodde and Larman don’t put the questions quite that way, but they do make product definition a key part of LeSS adoption, and strongly favour more expansive definitions based on actual user experience. This may seem obvious on the surface, but is the sort of obvious that is easily lost in the complex interactions of a large established organization. If your business provides sports news to its users, your real product is probably not a statistics vending web service with three “internal clients”. Or at least if you define your team’s product that way, you won’t be as effective at helping your users do something; it will at best be a local optimization. Part of the training was trying to define this product boundary for real systems we worked on, and real organizations. Often the best product definition cuts across senior managerial hierarchies, and this may limit what is the practical product definition can be in the near term. Crossing an organizational boundary usually requires coordination overhead that splits the thread pool of development teams, or introduces multiple product owners, both of which muddle the pattern. LeSS then uses the definition of done to ratchet the team closer to a product delivery mindset in every sprint. You are done only when the user experiences a change in the product.

LeSS mandates feature teams, which are fashionable at the moment, for mostly good reasons. This is again to support the work queue data structure and reinforce a whole-product view across the development teams. The argument is that programmers can learn, and the cost of context switches is worth it for product knowledge, reduced handoff queuing and integration risk, and flexibility of always having developers working on the most valuable features. Bas in particular emphasized learning as at the heart of software development. “When you switch components, velocity will go down, and that’s a good thing!” he said at multiple points during the three days. “It means you’re learning.”

Large Scale Scrum owes much of its success to Bas Vodde’s worldview of constructive organizational nihilism. “You have to give up the idea that things happen for a reason,” he says. “Or because someone decides them.” When I described it this way on morning three of the course Bas laughed, and tilted his head; then someone else asked what nihilism was. Study of nineteenth century philosophy isn’t a focus of a typical software engineering education, though Bas as it happens has squeezed some into his schedule around coding, becoming fluent in Chinese, raising a young family and keeping a day job of ripping internal corporate bureaucracies to pieces in the cause of building better software. He hadn’t seen Scrum and nihilism as linked.

Nihilism is the position that things have no inherent meaning, or, for Bas Vodde, that organizations have no inherent meaning. I call it constructive as I think he likes building useful things, and that’s what the most powerful parts of LeSS are about too. LeSS draws on Lean Engineering in its move away from an idea of best practice to one of continuous learning. In Taylorist scientific management, specialist managers discover a best practice and then encode it as a procedure to be followed by the organization. Nietzsche famously said that God was dead, that we had killed him, and set up Truth in his place, as another lie. Taylorism has the same tendency: it sets up procedures, and a hierarchical organizational tree, as an organizational singular Truth which is received by workers and mediated by a priesthood of managers. LeSS is pretty explicit about wanting to tear that down.

Hello? Is it Truth You're Looking For?

Bas himself, responding to an earlier version of this post, says his position is a little more moderate: that organizations usually have a purpose, but the things they actually do don’t achieve it, or often have any purpose at all, if you look for them.

The LeSS training explicitly frames itself against Ford and Taylor – sections contrasting Taylor with the Toyota way were part of the first day – but the argument is strictly pragmatic. Taylor’s techniques suit a world where large numbers of lightly schooled agricultural workers are coming to work in a factory, in industrial capitalism. It’s not well suited for knowledge workers building software under cognitive capitalism. You don’t need to discard God or Truth in general, both of which engineers can be fond of, to recognize that they aren’t a good model for software development, or other processes of design invention. An idea of a negotiated social truth, the artifacts and understanding of which mutates over time and depend on context, is rather more recognizable. That’s why the agile manifesto says it favours what it does; it is all about rejecting attempts to fixate on a flakily specified organizational high modernist Truth in favour of grappling with messy user reality. (Since the training I found Paulo Virno and the operaismo thinkers already made this link between nihilism and just-in-time production; eg quote and interview.)

Vodde and Larman tear down the idea of a single managerial Truth, then, but they also erect a replacement: the product.

Scrum is a strict process, but productive even in partial use. Ron Jeffries describes this himself when he describes many organizations as not really doing Scrum, and this is echoed by plenty of agile coaches, though they by their role tend to see this as a gap rather than positive variation. Let’s call this use of only some tools from the Scrum toolbox following a Scrum-like process. Scrum isn’t designed to work like this: it’s not a rational unified monstrosity where you are theoretically supposed to start by editing 90% of it out. You may have also added extra guff. Many teams running Scrum-likes may be doing it based on misconceptions, they may be Scrum-but, and they may well benefit from coaching. My guess, from the teams I’ve seen, is that most get some benefit from a short defined iteration (ie a sprint, though an over in Test match cricket might have been a better metaphor). They also benefit from a retrospective and a short daily standup meeting, even if the rest is a bit of a muddle. Crucially, Scrum-like teams also accommodate experiments and localization while keeping a common jargon.

Large Scale Scrum seems to have less room for this variation, because LeSS is Scrum, but scaled up, so each ritual and artifact bears more process weight. It has to do this because much variation will come from organizational inertia and dysfunction that will destroy the queue-worker thread structure at the heart of Scrum.

Scrum, with its focus on a single product owner and a single team, could conceal its essentially political nature pretty well. It was very directly about work that individuals in a small team do, so it could be introduced organically, at a team level. LeSS is about rebuilding your organization around a product, so it’s too big to hide. It is explicitly revolutionary and constitutional in its impact on office politics. For implementations smaller than nine teams, it even prescribes transitioning in a single day, starting a new organizational epoch with a dramatic software process Year Zero. This new Dictatorship of the Product has a politics which are collaborative and decentralizing, but shouldn’t be mistaken for something democratic. The rule of the product turns on the fulcrum of the product owner’s vision. Feature team programmer übermensch escape the shackles of component specialization, spreading their influence across technologies and languages. And of course the waterfall hierarchies of Taylorism were even less democratic and collaborative. LeSS might instead, when working well, have the character of an austere republicanism. There is a separation of powers, there is an executive product owner, and the duty to the Product takes the place of duty to the State (or The People).

Scrum, and Agile before it, build in the idea that the business and technology organizations are separate, and that one serves the other. In many industries that’s simply a recognition of organizational reality. The Scrum community aren’t deaf to it, and they tend to translate it into advice on hunting down, catching and breaking in a product owner. It’s pretty decent advice on the whole, if that’s what you want. In something like a startup, that may not make sense. Perhaps the lead developer has the best idea of the product, so role separation is meaningless. If, like Giles Bowkett, you want Scrum to die in a fire, I’d guess you’d want Large Scale Scrum to die in a fire too; maybe a larger one.

The most passionate argument in the three days of the course – and there were a few good ones – wasn’t about turning org charts upside down, or sidelining managers, or because I called Bas a nihilist, or what a product really is. It was about Jira. Bas recommends using a simple tool for the feature backlog, preferably a spreadsheet, and almost certainly not Jira. The argument, which has something to it, is that Jira is lightweight enough to be used, but heavyweight enough to destroy team ownership and collaborative planning. I have seen this too: it gets worse the more you type in meetings and the more workflow you add.

Say what you will about the tenets of National Socialism, Dude, at least it has traceability.

Say what you will about the tenets of National Socialism, Dude, at least it has traceability.

The Jira users were ready to circle the wagons and have it prised from their cold dead hands. I had forgotten how important the immediate tactility of sticky notes on boards is to a certain type of agilist. “Our ‘process’ is mainly github”, Bowkett says. The toolset for remote collaboration is much richer now, and the assumed developer toolset is a much stronger foundation. (Are you old enough to have had a serious argument about whether to use source control in your career? How about unit tests? With a senior developer?) Both Vodde and Bowkett are right to say tools shape processes. We default to their defaults. It made me realize we had been conflating the feature backlog and the sprint tasks for years, because that’s what’s easy in Jira. We also use Jira to coordinate across multiple cities, sometimes within teams – but I already knew that wasn’t Scrum.

One of the strengths of Scrum is its clarity of definition: you can talk clearly about whether or not you are following it, it doesn’t pretend to fit existing teams without change, and the same is true of LeSS. Vodde and Larman’s insight that making better software in large organizations requires refactoring the organization is spot on. LeSS is an organizational pattern for the product development interface. It’s not the only such pattern. MVC isn’t the only user interface pattern either, but its variants dominate the design space for a reason. If you don’t have the problems of product definition, flexibility and responsiveness in large organizations LeSS was designed to address, and you ship useful features at high quality every week already, don’t use it. For myself, and I suspect many software development organizations, those problems are of daily relevance. Software politics is the art of the possible. A software process coup d’état isn’t possible over here tonight; I’ll probably mangle and cajole a while instead.