In this article, we are revisiting the original ideas about the pleasures of programming put forth by Fred Brooks in one of the most influential and timeless books on software project management known as The Mythical Man-Month.
Why is programming fun? What delights may its practitioner expect as his reward?
1. The Sheer joy of building things
First is the sheer joy of making things. As the child delights in his mud pie, so the adult enjoys building things, especially things of his own design. This delight must be an image of God’s delight in making things, a delight shown in the individuality and novelty of each leaf and snowflake.
“The characteristic of great innovators and great companies is they see a space that others do not. They don’t just listen to what people tell them; they actually invent something new, something that you didn’t know you needed, but the moment you see it, you say, ‘I must have it.’”
– Eric Schmidt, Google
Innovation distinguishes between a leader and a follower.
– Steve Jobs
2. The Pleasure of making things useful to others
Second is the pleasure of making things that are useful to other people. Deep within, we want others to use our work and to find it helpful. In this respect the programming system is not essentially different from the child’s first clay pencil holder “for Daddy’s office.”
The purpose of life is not to be happy. It is to be useful, to be honorable, to be compassionate, to have it make some difference that you have lived and lived well.
– Ralph Waldo Emerson
3. The Fascination of forming complex puzzles
Third is the fascination of fashioning complex puzzle-like objects of interlocking moving parts and watching them work in subtle cycles, playing out the consequences of principles built in from the beginning. The programmed computer has all the fascination of the pinball machine or the jukebox mechanism, carried to the ultimate.
“Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. And to make matters worse: complexity sells better.”
― Edsger W. Dijkstra
4. The Joy of Perennial Learning experience
Fourth is the joy of always learning, which springs from the non-repeating nature of the task. In one way or another the problem is ever new, and its solver learns something: sometimes practical, sometimes theoretical, and sometimes both.
“Live as if you were to die tomorrow. Learn as if you were to live forever.”
― Mahatma Gandhi
5. The Delight of working in a tractable medium
Finally, there is the delight of working in such a tractable medium. The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination. Few media of creation are so flexible, so easy to polish and rework, so readily capable of realizing grand conceptual structures.
Yet the program construct, unlike the poet’s words, is real in the sense that it moves and works, producing visible outputs separate from the construct itself. It prints results, draws pictures, produces sounds, moves arms. The magic of myth and legend has come true in our time. One types the correct incantation on a keyboard, and a display screen comes to life, showing things that never were nor could be.
Programming is fun because it gratifies creative longings built deep within us and delights sensibilities we have in common with all men.
Hope, you enjoyed this article and please feel free to share your passion and interests for the craft in the comments section.
Conceptual integrity is the principle that anywhere you look in your system, you can tell that the design is part of the same overall design. This includes low-level issues such as formatting and identifier naming, but also issues such as how modules and classes are designed, etc. This is vitally important, because inevitably, unanticipated issues come up that must be resolved quickly. If there is a coherent design to the system, it can be much easier to resolve these issues by determining what would be consistent with the overall design, especially if the other designers are not present.
Conceptual Integrity is a required ingredient for achieving the principle that a system must have a powerful metaphor that is uniformly applied throughout a system.
Conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas.
Conceptual integrity does not mean one shouldn’t include many minds (or even the entire team for that matter) in the Analysis & Design process. This is a very important detail that shouldn’t be discounted by those who wish to do away with the role of architect. Team input in Analysis and Design is absolutely essential for:
(1) establishing Team Gel,
(2) ensuring the soundness and quality of the analysis, and
(3) refactoring the design into something more polished.
In fact, the earlier the architect or design-team can include the entire team (or domain-team leads for very large teams), the higher quality the design will be. The Design Team must be open to and accept criticism, the architect(s) must be egoless (see Egoless Programming).
In this article, we will examine the consequences of this theme for programming system design:
• How is conceptual integrity to be achieved?
• Does not this argument imply an elite, or high society of architects, and a group of ordinary implementers whose creative talents and ideas are suppressed?
• How does one keep the architects from drifting off into the blue with unimplementable or costly specifications?
• How does one ensure that every trifling detail of an architectural specification gets communicated to the implementer, properly understood by him, and accurately incorporated into the product?
Deciphering Conceptual Integrity
To appreciate the importance of conceptual integrity, we need to take a closer look at its meaning. The word conceptual is associated with the cognitive process of concept forming that involves the conscious recognition and identification of elements of our experience; and the word integrity is associated with the idea of being integrated or being one. The idea of a coherent whole is reflected in the way that having integrity is used to describe something that always conforms to one’s expectations – there is an implicit reference to future events. We describe something as having conceptual integrity, if the concepts formed from our experience of the thing can reliably determine the future events which are associated with the thing (there is no surprise).
In obtaining conceptual integrity, we are concerned with the emergence of concepts (representing the rational world) from experience (representing the empirical world). As the American philosopher, William James, points out in his Essays on Radical Empiricism, first published in 1902:
“Experiences come on an enormous scale, and if we take them all together, they come in a chaos of incommensurable relations that we cannot straighten out. We have to abstract different groups of them and handle these separately if we are to talk of them at all.”
Raw experiences are potentially confusing or incoherent as “they come in a chaos of incommensurable relations”. To make sense of them, we need to abstract groups of relations between them and study them separately. Abstracting groups of relations from experiences is the embryonic concept-forming activity. The concepts formed may conflict each other. Obtaining conceptual integrity is a process of removing conflict. To remove conflict, we need to access the raw experiences that originally informed our concepts – to explore them, to understand, to compare and to analyze. This idea is depicted in the following figure. At the left-hand-side of the figure, two groups of concepts are formed by abstracting relations from raw experiences. To obtain conceptual integrity, we need to combine the concepts into one group of unified concepts. This involves the resolution of potential conflict between concepts so that they acquire coherence.
Significant features of the above observations that relate to the nature of conceptual integrity include:
Conceptual integrity is something appreciated by an observer. This is because we cannot talk about experience without reference to the observer.
To obtain conceptual integrity, we need to resolve conflicts between concepts. This involves access to the raw experiences that inform these concepts.
A central issue in obtaining conceptual integrity is the potential incoherence of raw experiences.
Achieving Conceptual Integrity
The purpose of a programming system is to make a computer easy to use.
To do this, it furnishes languages and various facilities that are in fact programs invoked and controlled by language features. But these facilities are bought at a price: the external description of a programming system is ten to twenty times as large as the external description of the computer system itself. The user finds it far easier to specify any particular function, but there are far more to choose from, and far more options and formats to remember.
Ease of use is enhanced only if the time gained in functional specification exceeds the time lost in learning, remembering, and searching manuals. With modern programming systems this gain does exceed the cost, but in recent years the ratio of gain to cost seems to have fallen as more and more complex functions have been added.
Because ease of use is the purpose, this ratio of function to conceptual complexity is the ultimate test of system design.
Neither function alone nor simplicity alone defines a good design.
This point is widely misunderstood.
For a given level of function, however, that system is best in which one can specify things with the most simplicity aid straightforwardness. Simplicity is not enough.
The expression of the things one wants to do often requires involuted and unexpected combinations of the basic facilities. It is not enough to learn the elements and rules of combination; one must also learn the idiomatic usage, a whole lore of how the elements are combined in practice. Simplicity and straightforwardness proceed from conceptual integrity. Every part must reflect the same philosophies and the same balancing of desiderata. Every part must even use the same techniques in syntax and analogous notions in semantics. Ease of use, then, dictates unity of design, conceptual integrity.
Aristocracy and Democracy
Conceptual integrity in turn dictates that the design must proceed from one mind, or from a very small number of agreeing resonant minds.
Schedule pressures, however, dictate that system building needs many hands. Two techniques are available for resolving this dilemma. The first is a careful division of labor between architecture and implementation. The second is the new way of structuring programming implementation teams.
The separation of architectural effort from implementation is a very powerful way of getting conceptual integrity on very large projects.
By the architecture of a system, I mean the complete and detailed specification of the user interface. For a computer this is the programming manual. For a compiler it is the language manual. For a control program it is the manuals for the language or languages used to invoke its functions. For the entire system it is the union of the manuals the user must consult to do his entire job.
The architect of a system, like the architect of a building, is the user’s agent. It is his job to bring professional and technical knowledge to bear in the unalloyed interest of the user, as opposed to the interests of the salesman, the fabricator, etc.
Architecture must be carefully distinguished from implementation.
“Where architecture tells what happens, implementation tells how it is made to happen.”
Let’s take a simple example a clock, whose architecture consists of the face, the hands, and the winding knob. When a child has learned this architecture, he can tell time as easily from a wristwatch as from a church tower. The implementation, however, and its realization, describe what goes on inside the case—powering by any of many mechanisms and accuracy control by any of many.
Now we can deal with the deeply emotional question of aristocracy versus democracy. Are not the architects a new aristocracy, an intellectual elite, set up to tell the poor dumb implementers what to do? Has not all the creative work been sequestered for this elite, leaving the implementers as cogs in the machine? Won’t one get a better product by getting the good ideas from all the team, following a democratic philosophy, rather than by restricting the development of specifications to a few?
As to the last question, it is the easiest. Only the architects seldom have good architectural ideas. Often the fresh concept does come from an implementer or from a user. However, all my own experience convinces me, and I have tried to show, that the conceptual integrity of a system determines its ease of use. Good features and ideas that do not integrate with a system’s basic concepts are best left out. If there appear many such important but incompatible ideas, one scraps the whole system and starts again on an integrated system with different basic concepts.
As to the aristocracy charge, the answer must be yes and no. Yes, in the sense that there must be few architects, their product must endure longer than that of an implementer, and the architect sits at the focus of forces which he must ultimately resolve in the user’s interest. If a system is to have conceptual integrity, someone must control the concepts. That is an aristocracy that needs no apology.
No, because the setting of external specifications is not more creative work than the designing of implementations. It is just different creative work. The design of an implementation, given an architecture, requires and allows as much design creativity, as many new ideas, and as much technical brilliance as the design of the external specifications. Indeed, the cost-performance ratio of the product will depend most heavily on the implementer, just as ease of use depends most heavily on the architect.
There are many examples from other arts and crafts that lead one to believe that discipline is good for art. Indeed, an artist’s aphorism asserts, “Form is liberating.” The worst buildings are those whose budget was too great for the purposes to be served.
The external provision of an architecture enhances, not cramps, the creative style of an implementing group.
They focus at once on the part of the problem no one has addressed, and inventions begin to flow. In an unconstrained implementing group, most thought and debate goes into architectural decisions, and implementation proper gets short shrift.
However, if there is no final word, no one-mind fighting off the democratic compromises that can reduce a vision to its lowest common denominator, then it will be difficult to achieve conceptual integrity and the system may run the risk of becoming an amorphous blob of human insensitivity. It is important to realize that you can be inclusive (or team-oriented) without being everyone-designs or anti-architect. Said another way, it is possible to have an architect and have team collaboration on a design at the same time.
It is also important to note that on a small team, the design-team may in fact be the whole product team. Another approach would have the architect role and the coach or technical/team lead role (i.e. the final say or tie-breaker) be filled by a single individual in order to ensure conceptual integrity. As is mentioned in much of the Rational Unified Process literature, there is no requirement for a 1 to 1 or even a 1 to N cardinality between roles and people. A single person could hold many roles just as a single role could be held by many people.
What Does the Implementer Do While Waiting?
The lack of conceptual integrity makes the system far more costly to build and change.
When it is proposed that a small architecture team in fact write all the external specifications for a computer or a programming system, the implementers raise three objections:
• The specifications will be too rich in function and will not reflect practical cost considerations.
• The architects will get all the creative fun and shut out the inventiveness of the implementers.
• The many implementers will have to sit idly by while the specifications come through the narrow funnel that is the architecture team.
The first of these is a real danger, and it will be treated in the next chapter. The other two are illusions, pure and simple. As we have seen above, implementation is also a creative activity of the first order. The opportunity to be creative and inventive in implementation is not significantly diminished by working within a given external specification, and the order of creativity may even be enhanced by that discipline. The total product will surely be.
The last objection is one of timing and phasing. A quick answer is to refrain from hiring implementers until the specifications are complete. This is what is done when a building is constructed. In the computer systems business, however, the pace is quicker, and one wants to compress the schedule as much as possible. How much can specification and building be overlapped? As Blaauw points out, the total creative effort involves three distinct phases: architecture, implementation, and realization. It turns out that these can in fact be begun in parallel and proceed simultaneously.
In computer design, for example, the implementer can start as soon as he has relatively vague assumptions about the manual, somewhat clearer ideas about the technology, and well-defined cost and performance objectives. He can begin designing data flows, control sequences, gross packaging concepts, and so on. He devises or adapts the tools he will need, especially the recordkeeping system, including the design automation system.
Meanwhile, at the realization level, circuits, cards, cables, frames, power supplies, and memories must each be designed, refined, and documented. This work proceeds in parallel with architecture and implementation.
The same thing is true in programming system design. Long before the external specifications are complete, the implementer has plenty to do. Given some rough approximations as to the function of the system that will be ultimately embodied in the external specifications, he can proceed. He must have well-defined space and time objectives. He must know the system configuration on which his product must run. Then he can begin designing module boundaries, table structures, pass or phase breakdowns, algorithms, and all kinds of tools. Some time, too, must be spent in communicating with the architect.
Meanwhile, on the realization level there is much to be done also. Programming has a technology, too. If the machine is a new one, much work must be done on subroutine conventions, supervisory techniques, searching and sorting algorithms.
Conceptual integrity does require that a system reflect a single philosophy and that the specification as seen by the user flow from a few minds.
Because of the real division of labor into architecture, implementation, and realization, however, this does not imply that a system so designed will take longer to build. Experience shows the opposite, that the integral system goes together faster and takes less time to test. In effect, a widespread horizontal division of labor has been sharply reduced by a vertical division of labor, and the result is radically simplified communications and improved conceptual integrity.
Conceptual integrity is central to product quality. Having a system architect is the most important single step toward conceptual integrity.
The following are some of the well-known examples of Conceptual Integrity:
Unix (based on the notion of a “file” (e.g. directories, devices, filesystems, named pipes and sockets are all sort-of files)
Smalltalk (“everything is an object”, and the small set of other accompanying principles)
SQL (“all data is in tables”, with keys and constraints)
Lisp (“everything is a list”)
If you feel you can quote any more examples of conceptual integrity, please share your views in the comments.
In the field of psychology, cognitive dissonance is the mental discomfort (psychological stress) experienced by a person who simultaneously holds two or more contradictory beliefs, ideas, or values. The occurrence of cognitive dissonance is a consequence of a person performing an action that contradicts personal beliefs, ideals, and values; and also occurs when confronted with new information that contradicts said beliefs, ideals, and values.
In the fable of The Fox and the Grapes, by Aesop, on failing to reach the desired bunch of grapes, the fox then decides he does not truly want the fruit because it is sour. The fox’s act of rationalization (justification) reduced his anxiety about the cognitive dissonance occurred because of a desire he cannot realize.
Programming – perhaps more than any other profession – is an individual activity, depending on the abilities of the programmer himself, and not upon others. What difference can it make how many other programmers you run into during the day? If asked, most programmers would probably say they preferred to work alone in a place where they wouldn’t be disturbed by other people.
The ideas expressed in the preceding paragraph are possibly the most formidable barrier to improved programming that we shall encounter. First of all, if this is indeed the image generally held of the programming profession, then people will be attracted to, or repelled from, entering the profession according to their preference for working alone or working with others. Social psychologists tell us that there are different personality types – something we all knew, but which is nice to have stamped with authority. Among the general personality traits is one which is measured along three “dimensions” – whether a person is compliant, aggressive or detached. The compliant type is characterized by the attitude of liking to work with people and be helpful. The aggressive type wants to earn money and prestige, and the detached type wants to be left to myself to be creative.
Now, every person contains a mixture of these attitudes, but most people lean more heavily in one direction than the others. There is no doubt that the majority of people in programming today lean in the “detached” direction, both by personal choice and because hiring policies for programmers are often directed toward finding such people. And to a great extent, this is a good choice, because a great deal of programming work is alone and creative.
Like most good things, however, the detachment of programmers is often overdeveloped. Although they are detached from people, they are attached to their programs. Indeed, their programs often become extensions of themselves – a fact which is verified in the abominable practice of attaching one’s name to the program itself. But even when the program is not officially blessed with the name of its creator, programmers know whose program it is.
Well, what is wrong with owning programs? Artists own paintings; authors own books; architects own buildings. Don’t these attributions lead to admiration and emulation of good workers by lesser ones? Isn’t it useful to have an author’s name on a book so we have a better idea of what to expect when we read it? And wouldn’t the same apply to programs? Perhaps it would – if people read programs, but we know they do not. Thus, the admiration of individual programmers cannot lead to an emulation of their work, but only to an affectation of their mannerisms. This is the same phenomenon we see in art colonies, where everyone knows how to look like an artist, but few, if any, know how to paint like one.
The real difficulty with property-oriented programming arises from another source. When we think a painting or a novel or a building is inferior, that is a matter of taste. When we think a program is inferior – in spite of the difficulties we know lurk behind the question of good programming – that is a matter at least potentially susceptible to object proof or hypothesis. At the very least, we can put the program on the machine and see what comes out.
An artist can dismiss the opinions of a critic if they do not please him, but can a programmer dismiss the judgment of the computer?
On the surface, it would seem that the judgment of the computer is indisputable, and if this were truly so, the attachment of a programmer to his programs would have serious consequences for his self-image. When the computer revealed a bug in his program, the programmer would have to reason something like this:
This program is defective. This program is part of me, an extension of myself, even carrying my name. I am defective.
But the very harshness of this self-judgment means that it is seldom carried out.
Starting with the work of the social psychologist Leon Festinger, a number of interesting experiments have been performed to establish the reality of a psychological phenomenon called “cognitive dissonance”. A classical experiment in cognitive dissonance goes something like this:
Writing an essay
Two groups of subjects are asked to write an essay arguing in favor of some point with which they feel strong disagreement. One group is paid $1 apiece to write this argument against their own opinions, the other is paid $20 apiece. At the end of the experiment, the subjects are re-tested on their opinions of the matter. Whereas common sense would say that the $20 subjects – having been paid more to change their minds – would be more likely to change their opinions. Cognitive dissonance theory predicts that it will be the other group which will change the most. Dozens of experiments have confirmed the predictions of the theory.
The argument behind cognitive dissonance theory is quite simple. In the experiment just outlined, both groups of subjects have had to perform an act – writing an essay against their own opinions which they would not like to do under normal circumstances.
Arguing for what one does not believe is classed as insincerity or hypocrisy neither of which is highly valued in our society. Therefore, a dissonance situation is created.
The subject’s self-image as a sincere person is challenged by the objective fact of his having written the essay.
Dissonance, according to the theory, is an uncomfortable and unstable state for human beings, and must therefore be quickly resolved in one way or another. To resolve a dissonance, one factor or another contributing to it must be made to yield. Which factor depends on the situation, but, generally speaking, it will not be the person’s self-image.
That manages to be preserved through the most miraculous arguments. Now, in the experiments cited, the $20 subjects have an easy resolution of their dissonance:
Of course, I didn’t really believe those arguments. I just did it for the money.
Although taking money to make such arguments is not altogether the most admirable trait, it is much better than actually holding the beliefs in question.
But look at the difficult situation of the dollar group. Even for poor college students – and subjects in psychological experiments are almost always poor college students – one dollar is not a significant amount of money. Thus, the argument of the other group does not carry the ring of conviction for them, and the dissonance must be resolved elsewhere. For many, at least, the easiest resolution is to come to admit that there is really something to the other side of the argument after all, so that writing the essay was not hypocrisy, but simply an exercise in developing a fair and honest mind, one which is capable of seeing both sides of a question.
Another application of the theory of cognitive dissonance predicts what will happen when people have made some large commitment, such as the purchase of a car. If a man who has just purchased a Ford is given a bunch of auto advertisements to read, he spends the majority of his time reading about Fords. It was a Chevrolet he purchased, then the Chevrolet ads capture his attention. This is an example of anticipating the possibility of dissonance and avoiding information that might create it. For if he has just purchased a Ford, he doesn’t want to find out that Chevrolet is the better car, and the best way to do that is to avoid reading the Chevrolet ads. In the Ford ads, he is not likely to find anything that will convince him that he is anything but the wisest of consumers.
Now, what cognitive dissonance has to do with our programming conflict should be vividly clear. A programmer who truly sees his program as an extension of his own ego is not going to be trying to find all the errors in that program. On the contrary, he is going to be trying to prove that the program is correct – even if this means the oversight of errors which are monstrous to another eye. All programmers are familiar with the symptoms of this dissonance resolution – in others, of course. The programmer comes down the hall with his output listing and it is very thin. If he is unable to conceal the failure of his run, he makes some remark such as:
It must be a hardware problem.
There must be something strange in your data.
I haven’t touched that code in weeks.
There are thousands of variations to these objections, if you are interested in finding more, check out devexcuses or programmingexcuses. But the one thing we never seem to here is a simple
I goofed again
Of course, where the error is more subtle than a complete failure to get output – which can hardly be ignored – the resolution of the dissonance can be made even simpler by merely failing to see that there is an error. And let there be no mistake about it: the human eye has an almost infinite capacity for not seeing what it does not want to see. People who have specialized in debugging other people’s programs can verify this assertion with literally thousands of cases.
The human eye has an almost infinite capacity for not seeing what it does not want to see.
Programmers, if left to their own devices, will ignore the most glaring errors in their output – errors that anyone else can see in an instant. Thus, if we are going to attack the problem of making good programs, and if we are going to start at the fundamental level of meeting specifications, we are going to have to do something about the perfectly normal human tendency to believe that ones “own” program is correct in the face of hard physical evidence to the contrary.
Software project management is an art and science of planning and leading software projects. It is a sub-discipline of project management in which software projects are planned, implemented, monitored and controlled.
These words of wisdom are the outcome of multi-role viewpoint on software development. There is no ranking for better practices. But what really matters is to apply these practices where they make sense and where the users can benefit the most.
The structure of a system reflects the structure of the organization that built it.
– Richard E. Fairley, Wang Institute
Don’t keep doing what doesn’t work.
[Rule of Credibility]
The first 90% of the code accounts for the first 90% of the development time. The remaining 10% of the code accounts for the other 90% of the development time.
– Tom Cargill, Bell Labs
Less than 10% of the code has to do with the ostensible purpose of the system; the rest deals with input-output, data validation, data structure maintenance, and other housekeeping.
– Mary Shaw, Carnegie-Mellon University
Good judgement comes from experience, and experience comes from bad judgement.
– Fred Brooks, University of North Carolina
Don’t write a new program if one already does more or less what you want. And if you must write a program, use existing code to do as much of the work possible.
– Richard Hill, Hewlett-Packard S.A. , Geneva, Switzerland
Whenever possible, steal code.
– Tom Duff, Bell Labs
Good customer relations double productivity.
– Larry Bernstein, Bell Communications Research
Translating a working program to a new language or system takes ten percent of the original development time or manpower or cost.
– Douglas W. Jones, University of Iowa
Don’t use the computer to do things that can be done efficiently by hand.
– Richard Hill, Hewlett-Packard S.A. , Geneva, Switzerland
Don’t use hands to do things that can be done efficiently by the computer.
– Tom Duff, Bell Labs
I’d rather write programs to write programs than write programs.
– Dick Sites, Digital Equipment Configuration
[Brook’s Law of Prototypes]
Plan to throw one away, you will anyhow.
– Fred Brooks, University of North Carolina
If you plan to throw one away, you will throw away two.
– Craig Zerouni, Computer FX Ltd. London
Prototyping cuts the work to produce a system by 40%.
– Larry Bernstein, Bell Communications Research
[Thompson’s Rule for First-Time Telescope Makers.]
It is faster to make a 4-inch mirror then a 6-inch mirror than to make a 6-inch mirror.
– Bill McKeeman, Wang Institute
Furious activity is no substitute for understanding.
– H.H. Williams, Oakland, California
Always do the hard part first. If the hard part is impossible, why waste your time on the easy part? Once the hard part is done, you’re are home free.
Always do the easy part first. What you think at first is the easy part often turns out to be the hard part. Once the easy part is done, you can concentrate all your efforts on the hard part.