Software's essential form, and visual programming

NOTE HXA7241 2014-03-30T11:14Z

Text fits software's essential form. And any kind of visual programming will still have a similar kind of structure. We can do better than text, but it seems not a lot better.

Consider (again):

That is actually a reasonable pondering and aligns with the conclusion of this note, but let us step back and look a little more generally.

Is this not related to visual programming – based on icons, or diagrams, or other graphical things? What possible visual ways are there? Are there completely new ways to program?

No: any possible ideas of visual programming have certain firm limits. Here is why.


Start with this question: how would you input such graphical code? What would be the UI?

For a small language, of 20-ish elements, you can have a menu at the side and pick which to use. But what about including the basic library, of 200 items? Or the main library of 20000 items? A menu to pick visual elements will not work.

The obvious answer is: you type the name of the wanted element – i.e. text! Or a first thought of an alternative: classifying into groups and subgroups, and navigating down the tree. But the path to any such leaf item is just a sequence of ‘keys’ each from a small set: really, just like a name composed of chars, which would be quicker to use – text again.

Maybe the purpose is more about using richer gestures and representations to input and output code. But the problem is still there: how can those be referred to? How could you describe a sequence needed to perform some task? Probably by naming them – which is back to text.

So alternatives to text seem just, well:

“Any sufficiently complicated non-text programming system contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of ordinary text.”

– HXA's 10th law (adapted from Greenspun's).

It is not as bad as that, but the point is there. Purely visual/graphical systems do not seem workable, and there is always something pulling us back to text.


But is this merely long conditioned text-culture making everything seem like text? (like money) To avoid that we can ask this question: is it possible to break down and understand the matter in terms that are independent and primitive?

What is the structure, the form, the shape of software? It seems so ethereal and ungraspable, what is there to get hold of?

The form of software is what is done with it. Software has the features it has because of the actions performed on it. And what we do with it is build: assemble smaller pieces into larger ones, and also as part of that process, analyse larger pieces into smaller ones. There are two basic aspects: connecting and nesting – connecting in a kind of adjacency, and nesting as grouping mulitiple into one. Software is constructive. The form of software reflects the engineering design process.

And any sophisticated and powerful way of making things must be like this – built-from-pieces. If every new artifact were made whole and indivisible, there could be no reuse. And we would have no knowledge about what we make – how it would work/behave – because it would contain nothing used, hence known, before.

With material things, their shape and features determines what we can do with them. But with software it is the other way around: we can make any kind of appearance we want, but the activity is pre-set. We have a particular purpose and actions we want do with software, and so we have to make it ‘look’ appropriate for that – or we are best off if we do.

It is this constructive form of software which makes a text-like system always make a lot of sense. Text – ‘built’ of chars, and words, and lines – nicely reflects that form. Text makes for a good expression of the essential structure of software.

So we need some constructive symbol system (of building symbols from smaller ones) to grasp software, and text is probably a well evolved, if not simply a well established standard for that.


Also try thinking from the other direction, from picture to software:

You may look at other visual forms – the many varieties of diagrams, illustrations, pictures – and feel they are richer and more communicative than mere text, but this vague feeling is misleading. Stop and ask: what do these pictures mean? what information do they present? what structure do they have?

If you had to draw a picture of software, it cannot be that just anything would do. Just because they might be colourful and detailed does not mean such pictures can express software, that they can fit software. Any kind of picture cannot represent any kind of structure – otherwise ‘representation’ would not mean anything.

Most of the richness you might see is meaningless for software. The main requirement for a picture of software is to express the structure of software. Software has a particular objective form: pieces connecting and nesting. Any picture of it must be centered on expressing that.

Code as text is visual programming: it is just a picture made with a quite simple kind of drawing tool.


So visual programming ought not to be an ignis fatuus: it seems not to offer any fundamental breakthroughs. It probably cannot even be about eliminating text, or only in a superficial way. Instead, it must be about expanding text or augmenting it or making new variants of it.

This includes bringing forward various manipulations. Text-editing puts char-shuffling in the foreground, but we ought to emphasise the larger software structure manipulations (starting with the basics of connecting and nesting), that are currently rather in the background.