Over the past week I've been reading Thomas J. Allen's Managing the Flow of Technology, which summarizes about a decade of MIT Sloan research into how R&D organizations acquire and transmit knowledge. A number of passages have jumped out to me, and I wanted to comment on them here. Emphasis is mine throughout.
The distinction between science and engineering is key to this book. On page 3:
The scientist's principal goal is a published paper. The technologist's goal is to produce some physical change in the world. This difference in orientation, and the subsequent difference in the nature of the products of the two, has profound implications for those concerned with supplying information to either of the two activities.
And on page 5:
...whereas the provision of information in science involves the gathering, organizing, and distribution of publications, the situation in technology is very different. The technologist must obtain his information either through the very difficult task of decoding and translating physically encoded information or by relying upon direct personal contact and communication with other technologists. His reliance upon the written word will be much less than that of the scientist.
Starting on page 39:
THE NATURE OF TECHNOLOGY
The differences between science and technology lie not only in the kinds of people who are attracted to them; they are basic to the nature of the activities themselves. Both science and technology develop in a cumulative manner, with each new advance building upon and being a product of vast quantities of work that have gone before. In science all of the work up to any point can be found permanently recorded in literature, which serves as a repository for all scientific knowledge. The cumulative nature of science can be demonstrated quite clearly (Price, 1965a, 1970) by the way in which citations among scientific journal articles cluster and form a regular pattern of development over time.
A journal system has been developed in most technologies that in many ways emulates the system originally developed by scientists; yet the literature published in the majority of these journals lack, as Price (1965a, 1970) has shown, one of the fundamental characteristics of the scientific literature: it does not cumulate or build upon itself as does the scientific literature. Citations to previous papers or patents are fewer and are most often to the author's own work. Publication occupies a position of less importance than it does in science where it serves to document the end product and establish priority. Because published information is at best secondary to the actual utilization of the technical innovation, this archival is not as essential to ensure the technologist that he is properly credited by future generations. The names of Wilbur and Orville Wright are not remembered because they published papers. As pointed out in chapter 1, the technologist's principal legacy to posterity is encoded in physical, not verbal, structure. Consequently the technologist publishes less and devotes less time to reading than do scientists.
Information is transferred in technology primarily through personal contact. Even in this, however, the technologist differs markedly from the scientist. Scientists working at the frontier of a particular specialty know each other and associate together in what Derek Price has called "invisible colleges." They keep track of one another's work through visits, seminars, and small invitational conferences, supplemented by an informal exchange of written material long before it reaches archival publication. Technologists, on the other hand, keep abreast of their field by close association with co-workers in their own organization. They are limited in forming invisible colleges by the imposition of organizational barriers.
I'll pause here to note that this bothers me somewhat. I enjoy few things more than learning from other people, especially if they inhabit different worlds than I do. Allen continues:
BUREAUCRATIC ORGANIZATION
Unlike scientists, the vast majority of technologists are employed by organization with a well-defined mission (profit, national defense, space exploration, pollution abatement, and so forth). Mission-oriented organizations necessarily demand of their technologists a degree of identification unknown in most scientific circles. This organizational identification works in two ways to exclude the technologist from informal communication channels outside his organization. First, he is inhibited by the requirements that he work only on problems that are of interest to his employer, and second, he must refrain from early disclosure of the results of his research in order to maintain his employer's advantage over competitors. Both of these constraints violate the rather strong scientific norms that underlie and form the basis of the invisible college. The first of these norms demands that science be free to choose its own problems and that the community of colleagues be the only judges of the relative importance of possible areas of investigation, and the second is that the substantive findings of research are to be fully assigned and communicated to the entire research community. The industrial organization, by preventing its employers from adhering to these two norms, impedes the formation by technologists of anything resembling an invisible college.
Incidentally, I believe that companies lose more by inhibiting cross pollination than they gain by protecting their competitive position. It would appear that Allen would agree, at least to an extent. On page 42:
The Effect of Turnover
It is this author's suspicion that much of the proprietary protectionism in industry is far overplayed. Despite all of the organizational efforts to prevent it, the state of the art in technology propagates quite rapidly. Either there are too many martinis consumed at engineering conventions or some other mechanism is at work. This other mechanism may well be the itinerant engineer, who passes through quite a number of organizations over the course of a career...
Each time that an engineer leaves an employer, voluntarily or otherwise, he carries some knowledge of the employer's operations, experience, and current technology with him. We are gradually coming to realize that human beings are the most effective carriers of information and that the best way to transfer information between organizations or social systems is to physically transfer a human carrier. Roberts' studies (Roberts and Wainer, 1967) marshal impressive evidence for the effective transfer of space technology from quasi-academic institutions to the industrial sector and eventually to commercial applications in those instances in which technologists left university laboratories to establish their own businesses. This finding is especially impressive in view of the general failure to find evidence of successful transfer of space technology by any other mechanism, despite the fact that many techniques have been tried and a substantial amount of money has been invested in promoting the transfer.
This certainly makes sense. Ideas have no real existence outside of the minds of men. Ideas can be represented in verbal or graphic form, but such representation is necessarily incomplete and cannot be easily structured to fit new situations. The human brain has a capacity for flexibly restructuring information in a manner that has never been approached by even the most sophisticated computer programs. [Just jumping in here to say bravo. -SW] For truly effective transfer of technical information, we must make use of this human ability to recode and restructure information so that it fits into new contexts and situations. Consequently, the best way to transfer technical information is to move a human carrier. The high turnover among engineers results in a heavy migration from organization to organization and is therefore a very effective mechanism for disseminating technology throughout an industry and often to other industries. Every time an engineer changes jobs he brings with him a record of his experiences on the former job and a great amount of what his former organization considers "proprietary" information. Now, of course, the information is usually quite perishable, and its value decays rapidly with time. But a continual flow of engineers among the firms of an industry ensures that no single firm is very far behind in knowledge of what its competitors are doing. So the mere existence of high turnover among R&D personnel vitiates much of the protectionism accorded proprietary information.
As for turnover itself, it is well known that most organizations attempt to minimize it. If all of the above is even partially true, a low level of turnover could be seriously damaging to the interests of the organization. Actually, however, quite the opposite is true. A certain amount of turnover may be not only desirable but absolutely essential to the survival of a technical organization, although just what the optimum turnover level is for an organization is a question that remains to be answered. It will vary from one situation to the next and is highly dependent upon the rate at which the organization's technical staff is growing. After all, it is the influx of new engineers that is most beneficial to the organization, not the exodus of old ones. When growth rate is high, turnover can be low. An organization that is not growing should welcome or encourage turnover. The Engineers' Joint Council figure of 12 percent may even be below the optimum for some organizations. Despite the costs of hiring and processing new personnel, an organization might desire an even higher level of turnover. Although it is impossible to place a price tag on the new state-of-the-art information that is brought in by new employees, it may very well more than counterbalance the costs of hiring. This would be true at least to the point where turnover becomes disruptive to the morale and functioning of the organization.
Allen also discusses the degree two which academia influences technology development. On page 51:
Project Hindsight was the first of a series of attempts to trace technological advances back to their scientific origins. Within the twenty-year horizon of its backward search, Hindsight was able to find very little contribution from basic science (Sherwin and Isenson, 1967). In most cases, the trail ran cold before reaching any activity that could be considered basic research. In Isenson's words, "It would appear that most advances in the technological state of the art are based on no more recent advances than Ohm's Law or Maxwell's equations."
On page 52:
In yet another recent study, Langrish found little support for a strong science-technology interaction. Langrish wisely avoided the problem of differentiating science from technology. He categorized research by the type of institution in which it was conducted - industry, university, or government establishment. In tracing eighty-four award-winning innovations to their origins, he found that "the role of university as a source of ideas for [industrial] innovation is fairly small" (Langrish, 1971) and that "university science and industrial technology are two quite separate activities which occasionally come into contact with each other" (Langrish, 1969). He argued very strongly that most university basic research is totally irrelevant to societal needs and can be only partially justified for its contributions through training of students.
That's tough stuff, if you ask me. Incidentally, I've considered many times recently whether I myself would go to college if I was just graduating high school today. It would not be a straightforward choice.
Then Allen turned to the qualities of the things that engineers actually read. On page 70:
A MORE DETAILED EXAMINATION OF WRITTEN MEDIA
Looking first at the identity of the publications that were read, there are two major categories of publications that engineers use. The first of these might be called formal literature. It comprises books, professional journals, trade publications, and other media that are normally available to the public and have few, if any, restrictions on their distribution. Informal publications, on the other hand, are published by organizations usually for their own internal use; they often contain proprietary material and for that reason are given a very limited distribution. On the average, engineers divide their attention between the two media on about an equal basis, only slightly favoring the informal publications (table 4.3). Because engineering reports are usually much longer than journal articles and because books are used only very briefly for quite specific purposes, each instance of report reading takes twice as long as an instance of journal or book reading. The net result is a threefold greater expenditure of time on informal reports. We can conclude from this brief overview that the unpublished engineering report occupies a position that is at least as important as that of the book or journal in the average engineer's reading portfolio.
Here I should note that I read this through the lens of someone whose public blog is essentially an ongoing and highly detailed series of informal reports. I'm certainly no scientist, and in general my writing isn't particularly academic. I'm doing decidedly applied work, and I document it (including what most companies would call proprietary information about my products and the results of my research) for anyone to read and repurpose as they please.
Allen continues, explaining why engineering journals aren't really used by practicing engineers. On page 73:
The publications of the professional engineering societies in all of these diverse fields are little used by their intended audience.
Why should this be so? The answer is not difficult to find. Most professional engineering journals are utterly incomprehensible to the average engineer. They often rely heavily upon mathematical presentations, which can be understood by only a limited audience. The average engineer has been away from the university for a number of years and has usually allowed his mathematical skills to degenerate. Even if he understood the mathematics at one time, it is unlikely that he can now. The articles, even in engineering society journals, are written for a very limited audience, usually those few at the very forefront of a technology. Just as in science, the goal of the author is not to communicate to the outsider but to gain for himself the recognition of his peers.
It's funny: the purpose of this blog is to communicate with outsiders AND gain the recognition of my peers. I'd like to think, in fact, that it fits the description of the ideal engineering literature that Allen puts forth on page 75:
The professional societies could publish a literature form whose technical content is high, but which is understandable by the audience to whom it is directed...The task is not an impossible one. Engineers will read journals when these journals are written in a form and style that they can comprehend. Furthermore, technological information can be provided in this form. Why then do the professional societies continue to publish material that only a small minority of their membership can use? If this information can be provided in a form that the average engineer can understand, why haven't the professional societies done so?
The obvious answer to these questions is that the societies have only recently become aware of the problem. In the past, they were almost totally ignorant of even the composition of their membership, and they still know littler of their information needs. Thus, they have never had the necessary information to formulate realistic goals or policy. Perhaps the most unfortunate circumstance that ever befell the engineering profession in the United States is that at the time when it first developed a self-awareness and began to form professional societies, it looked to the scientific societies, which had then existed for over 200 years, to determine their form and function.
Interestingly, though, I do not fit the description of the engineer that Allen gives on page 99:
THE IMPORTANCE OF COMMUNICATION WITHIN THE LABORATORY
Most engineers are employed by bureaucratic organizations. Academic scientists are not. The engineer sees the organization as controller of the only reward system of any real importance to him and patterns his behavior accordingly. While the academic scientist finds his principal reference group and feels a high proportion of his influence from outside the organization, for the engineer, the exogenous forces simply do not exist. The organization in which he is employed controls his pay, his promotions, and, to a very great extent, his prestige in the community.
To be clear, I get a ton out of working closely with people. I worked alone building bikes for a full three years, and was solo and very isolated during much of the two year construction project I completed after college; the lack of camaraderie in those situations was hard on me. I learned through that process that working with people - and having a mutual feeling of respect and enthusiasm - was incredibly important. I've gotten a ton out of all of the colleagues I've had since then - including many who I initially clashed with.
But exogenous forces in my life absolutely exist, and are important too. I benefit greatly from keeping contact with people elsewhere in my industry - and people outside of it - and I'm confident that the companies I've worked for have benefited from my network too.
My belief is that there's more room for these things to coexist than most companies realize. As evidence, I would present that when I began working on metal 3D printing, I knew nothing about it - and didn't work at a company that had any particular interest about it in the first place. I believe that it is only through my openness that I've gotten where I am today, and through that openness I've also vastly improved my access to experienced engineers across the industry. I've gotten cold emails from people working at some of the biggest and most advanced R&D organizations in the world, something I don't think would ever have happened had I not shared the way I did. And I'm confident that the my relationships with these people are mutually beneficial - both to us as people and to the companies who employ us.
I'm about a third through Managing the Flow of Technology now; I'll probably finish it in the next month. I recommend it.