The Art of Doing Science and Engineering: Learning to Learn

Teachers should prepare the student for the student’s future, not for the teacher’s past.

Apparently an “art”— which almost by definition cannot be put into words—is probably best communicated by approaching it from many sides and doing so repeatedly.

I am, as it were, only a coach. I cannot run the mile for you; at best I can discuss styles and criticize yours.

Either you will be a leader, or a follower, and my goal is for you to be a leader.

I have frequently observed great scientists and engineers do this much more often than “the run of the mill” people, hence it requires illustration.

I found it very valuable at the physics table I used to eat with; I sometimes cleared up misconceptions at the time they were being formed, thus advancing matters significantly.

If you were at times awed by the mass of knowledge you faced when you went to college, or even now, think of your children’s troubles when they are there!

In science if you know what you are doing you should not be doing it. In engineering if you do not know what you are doing you should not be doing it.

In a lifetime of many, many independent choices, small and large, a career with a vision will get you a distance proportional to n, while no vision will get you only the distance square root of n.

No vision, not much of a future.

It has often been observed the true gain is in the struggle and not in the achievement—a life without a struggle on your part to make yourself excellent is hardly a life worth living.

Noise introduced at one spot, if not too much to make the pulse detection wrong at the next repeater, is automatically removed.

It has rarely proved practical to produce exactly the same product by machines as we produced by hand.

We are now looking more and more in books and less and less at Nature! There is clearly a risk we will go too far occasionally—and I expect this will happen frequently in the future.

The people at the bottom do not have the larger, global view, but at the top they do not have the local view of all the details, many of which can often be very important, so either extreme gets poor results.

I believe computers will be almost everywhere since I once saw a sign which read, “The battle field is no place for the human being”.

Often a new innovation will set the growth of a field onto a new “S” curve which takes off from around the saturation level of the old one.

You will hardly get a system installed and working before there are significant improvements which you can adapt by mere program changes.

Slide rules in the 1930s and 1940s were standard equipment of the engineer, usually carried in a leather case fastened to the belt as a badge of one’s group on the campus.

Babbage insisted the printing be done by the machine to prevent any human errors creeping in.

I well recall a group of us, after a session on the IBM 701 at a meeting where they talked about the proposed 18 machines, all believed this would saturate the market for many years!

Thus in 3 seconds a machine doing 109 floating point operations per second (flops) will do more operations than there are seconds in your whole lifetime, and almost certainly get them all correct!

The purpose of computing is insight, not numbers.

We see the machine does not know where it has been, nor where it is going to go; it has at best only a myopic view of simply repeating the same cycle endlessly.

History of Computers—Software

Paper tapes are a curse when doing one-shot problems —they are messy, and gluing them to make corrections, as well as loops, is troublesome (because, among other things, the glue tends to get into the reading fingers of the machine!).

As a result the control path of the program through storage soon took on the appearance of a can of spaghetti.

Third, even if it did work, no respectable programmer would use it—it was only for sissies!

It has been said in physics no creator of any significant thing ever understood what he had done.

Please remember, the inventor often has a very limited view of what he invented, and some others (you?) can see much more.

It goes on top of the machine’s language, making the machine into any other machine you want.

Almost no one can write dialog so that it sounds right, and when it sounds right it is still not the spoken language.

What I wanted to know was how the job of communication can be efficiently accomplished when we have the power to design the language, and when only one end of the language is humans, with all their faults, and the other is a machine with high reliability to do what it is told to do, but nothing else.

But you do not expect novelists to “engineer the production of novels”. The question arises, “Is programming closer to novel writing than it is to classical engineering?” I suggest yes!

In practice you may actually be better off to pay the worst to stay home and not get in the way of the more capable (and I am serious)!

On thinking this over very seriously, I came to the conclusion I could not afford to be crippled that way and still become a great scientist.

Your excitement tends to be communicated to the audience, and if you seem to be perfectly relaxed then the audience also relaxes and may fall asleep!

Yes, we did some of the hardest problems on the most primitive equipment—it was necessary to do this in order to prove machines could do things which could not be done otherwise.

They enable us to deal with variety without excessive standardization, and hence we can evolve more rapidly to a desired future!

I believed then, as I do now, that cheap, small SDS 910 machine at least doubled the effective productivity of the huge, expensive cyclotron!

You simply cannot use a constantly changing data base for an optimization study.

One of the main reasons is there is a great ego satisfaction in having your own special chip and not one of the common herd.

I have no objections to 10% improvements of established things, but from you I also look for the great new things which make so much difference to your organization that history remembers them for at least a few years.

Among other troubles with this idea is in many fields, especially in medicine, the world famous experts are in fact not much better than the beginners!

dubious title of Artificial Intelligence (AI), which does not have a single meaning.

Thus you cannot afford to either believe or disbelieve—you must come to your own terms with the vexing problem, 'To what extent can machines think?'

As to the soul, in the Late Middle Ages some people, wanting to know when the soul departed from the dead body, put a dying man on a scale and watched for the sudden change in weight—but all they saw was a slow loss as the body decayed.

Such people are forced, like the above mentioned Jesuit trained engineer, to make the definition of thinking to be what machines cannot do.

Thus when to go on the attack is a touchy matter; too soon and you lose the initiative, too late and the opponent starts and wins.

If you deny the machine learns from experience because you claim the program was told (by the human programmer) how to do improve its performance, then is not the situation much the same with you, except you are born with a somewhat larger initial program compared to the machine when it leaves the manufacturer’s hands?

Before the checker playing program which learned was exposed in simple detail, you probably thought machines could not learn from experience—now you may feel what was done was not learning but clever cheating.

Perhaps it is not a separate thing, it is just an artifact of largeness.

Thus we have the paradox; the existence of the program automatically turns you against believing it is other than a rote process.

It is now clearly a matter of what sounds are worth producing, not what can be done.

However, I have long publicly doubted you could take many coal miners and make them into useful programmers.

But with a machine error whom do you sue? The machine? The programmer? The experts who were used to get the rules?

No human mind could go reliably through the layout of more than a million transistors on a chip; it would be a hopeless task.

That, at least is what they think they think—what the human mind actually does when playing chess is another matter!

The logic of the situation is inescapable— the reality is hardly believable!

We simply do not know what we are talking about; the very words are not defined, nor do they seem definable in the near future.

It is the combination of man-machine which is important, and not the supposed conflict which arises from their all too human egos.

I have some times wondered whether it might be better if I asked people to apply computers to other areas of application than their own narrow speciality; perhaps they would be less inhibited there!

You think you live in three dimensions, but in many respects you live in a two dimensional space.

Note as the numbers get larger and larger the ratio approaches 1 but the differences get greater and greater!

As we say, the volume is almost all on the surface.

Yes, the sphere is convex, yes it touches each of the 1024 packed spheres on the inside, yet it reaches outside the cube!

In this space a circle in two dimensions looks like a square standing on a point, Figure 9.V.

It is the abstraction from details that gives the breadth of application.

Recall, again, sending through space is the same as sending through time, namely storage.

You have to wait until you get to the end of the message before you can start the decoding process!

When examined closely this inequality says there cannot be too many short symbols or else the sum will be too large.

This inability of the receiver to “hear what is said” by a person in a higher management position but to hear only what they expect to hear, is, of course, a serious problem in every large organization.

The average length of the two codes is the same, but the codes, and the decoding trees are different; the first is “long” and the second is “bushy”, and the second will have less variability than the first one.

Indeed, you can write a program which will sample the data to be stored and find estimates of the probabilities, find the Huffman code, do the encoding, and send first the decoding algorithm (tree) and then the encoded data, all without human interference or thought!

A sensible criterion is to minimize the variance of the code so that messages of the same length in the original symbols will have pretty much the same lengths in the encoded message.

Any error was caught by the machine almost in the act of its being committed, and hence pointed the maintenance people correctly rather than having them fool around with this and that part, misadjusting the good parts in their effort to find the failing part.

The dashes are merely for decorative effect and are not used in the code at all.

I was angry to say the least, and said, 'If the machine can locate there is an error, why can it not locate where it is, and then fix it by simply changing the bit to the opposite state?'

My smugness vanished immediately! Did I have the best code this time?

If it seems magical, then think of the all 0 message, which will have all 0 checks, and then think of a single digit changing and you will see as the position of the error is moved around then the syndrome binary number will change correspondingly.

It is obvious if the centers of these spheres are code points, and only these points, then at the receiving end any single error in a message will result in a non-code point and you can recognize where the error came from.

When, during initial installation, any unit is set up and running properly, and you then turned your back on it to get the next part going, if the one you were neglecting developed a flaw, it told you so!

Of course as you go through life you do not know what you are preparing yourself for—only you want to do significant things and not spend the whole of your life being a “janitor of science” or whatever your profession is.

Shannon identified information with surprise. He chose the negative of the log of the probability of an event as the amount of information you get when the event of probability p happens.

The same mathematical form does not imply the same interpretation of the symbols!

If n is large enough then there is an arbitrarily small probability of there occurring a received message point bj which falls outside this sphere.

Not knowing how to encode, error correcting codes not having been invented as yet, Shannon chose a random encoding.

How large is this n? Very, very large indeed if you want to be both close to channel capacity and reasonably sure you are right!

Their conclusion arose from the tool used and not from reality.

He looked me square in the eye and said, Yes Hamming, you should.” and walked off!

Doing what needed to be done, though I did not want to do it, paid off handsomely in the long run.

Lo, and behold, the famous transfer function is exactly the eigenvalues of the corresponding eigenfunctions!

I have found it convenient to think once the samples have been taken then all the frequencies are in the Nyquist band, and hence we do not need to draw periodic extensions of anything since the other frequencies no longer exist in the signal.

Debugging a large program across the continent based on the sampling theorem!

Do not let that worry you as we were speaking of a window in a metaphorical way and hence negative transmission is possible.

Those who claimed there was no essential difference never made any significant contributions to the development of computers.

In words, the output of the filter is the sum of three consecutive inputs divided by 2, and the output is opposite the middle input value.

When Michelson did this he observed an overshoot and asked the local mathematicians why it happened. They all said it was his equipment—and yet he was well known as a very careful experimenter.

Thus the rate of convergence is directly observable from the function along the real line—which is not true for the Taylor series whose convergence is controlled by singularities which may lie in the complex plane.

Multiplication on one side is convolution on the other side of the equation.

The Hamming window was devised to make the maximum side lobe a minimum.

The Hamming window has a mysterious, hence popular, aura about it with its peculiar coefficients, but it was designed to do a particular job and is not a universal solution to all problems.

It is a “trial and error” design method.

All I remembered was it was one of Tukey’s few bad ideas; I completely forgot why it was bad— namely because of the equipment I had at time.

Moral: when you know something cannot be done, also remember the essential reason why, so later,

We force all nonharmonic frequencies into harmonic ones—we force a continuous spectrum to be a line spectrum!

A little knowledge is a dangerous thing—especially if you lack the fundamentals!

In picture processing, a recursive digital filter which used only data from one side of the point being processed would be foolish since it would not to use some of the available, relevant information.

I found myself, in spite of many experiences, in the same classic hunting situation of instability.

If you will only ask yourself, “Is what I am being told really true?” it is amazing how much you can find is, or borders on, being false, even in a well developed field!

The curse of the expert with their limited view of what they can do.

When this was pointed out to them, their reply seemed to be they did not know what else to do, so they persisted in doing the wrong thing!

It is as if you put on blue tinted glasses; everywhere you look you must see things with a bluish tint!

Intellectual shelf life is often more insidious than is physical shelf life.

In many situations, the power of the machine itself so far exceeds our powers to program it is wise to look early and constantly for the repetitive parts of a proposed simulation.

Indeed, it is claimed whether or not a butterfly flaps its wings in Japan can determine whether or not a storm will hit this country and how severe it will be.

Volume output seems to me to be a poor substitute for acquiring an intimate feeling for the situation being simulated.

Jargon is both a necessity and a curse.

It is as good an example as I know of to illustrate the fact both of us understood exactly what the mathematical symbols meant—we both had no doubts—but there was no agreement in our interpretations of them!

It turned out the equations they chose were designed to show a catastrophy no matter how you started or chose most of the coefficients!

His refusal to reply, under repeated requests, was a clear admission my point went home, he himself knew the Director did not understand this difference but thought the report was the reliability of the actual shot.

I noticed, after a while, not one adult ever got the idea of what was going on enough to play successfully, and almost every child did!

You are used to the idea combining data can obscure things, but it can also create effects is less well known.

In the stock market, if there were any widely known strategy for making lots of money, the very knowledge of it would ruin the strategy!

It is not something you can take for granted just because a big machine gives out nicely printed sheets, or displays nice, colorful pictures.

Analog machines are generally ignored these days, so I feel I need to remind you they have a place in the arsenal of tools in the kit of the scientist and engineer.

The four circle figure in two dimensions, leading to the n-dimensional paradox by ten dimensions, shows how tricky such imagining may become.

In the frequency approach we will concentrate on getting the frequencies right and let the actual positions be what happen.

I had earlier realized the nature of the field trials being simulated was such that small deviations from the proposed trajectory would be corrected automatically by the guidance system!

Hence garbage in, but accurate results out never-the-less!

In the past I had used the obvious trick when facing a divergent direction field of simply integrating in the opposite direction and you get an accurate solution. But in the above problem you are, as it were, walking the crest of a sand dune, and once both feet are one side of the crest you are bound to slip down.

You are lovingly taught how one theory was displaced by another, but you are seldom taught to replace a nice theory with nothing but randomness!

Simulation is essential to answer the 'What if...?', but it is full of dangers, and is not to be trusted just because a large machine and much time has been used to get the nicely printed pages, or colorful pictures on the oscilloscope.

During the early part of the talk the speaker remarked, “God loved sand, He made so much of it”.

They said if the ocean waters were as clear as were some of the glasses then you could see to the bottom of the Pacific Ocean!

One of the virtues of solitons is they can be amplified without changing their shape (which does not degrade as it goes

The use of satellites means broadcasting the signal—cables give a degree of privacy and the ability to make the user pay rather than get a free ride.

Light beams can pass through one another without interference (provided the intensity is not too high) which is more than you can do with wires.

Just because it can be done economically does not mean it should be done.

There is a story from ancient Greek times of a Mathematician telling a ruler there were royal roads for him to walk on, and royal messengers to carry his mail, but there was no royal road to geometry.

Hawthorne effect vitiates most educational experimentation.

The Hawthorne effect strongly suggests the proper teaching method will always to be in a state of experimental change, and it hardly matters just what is done, all that matters is both the professor and the students believe in the change.

Another terrible fact is carefully watching the students to see what happens in practice has shown a good student often picks what they know is the wrong answer simply out of either boredom or amusement to see what the book will say.

What you learn from others you can use to follow; What you learn for yourself you can use to lead.

The fact is, what they knew in one class at on e hour with one professor did not transfer to the another hour in a room across the hall with another professor.

At their age then it was practically impossible to make them so overlearn the alphabet they could use such information sources easily.

Just because something can be done, especially using computers, does not mean it should be done.

You have only to look at the legal system and the income tax people, and their use of the natural language to express what they mean, to see how inadequate the English language is for clear thinking.

For them all of Mathematics is a mechanical game where no interpretation of the meaning of the symbols is permitted lest you make an all too human error.

Euclid did not lay down postulates and make deductions as it is commonly taught; he felt his way back from “known” results to the postulates he needed!

I will tell you to go back and get new assumptions—I know Cauchy’s theorem is “true”.

The professors are too busy doing the details of Mathematics to ever discuss what they are actually doing—a typical technician’s behavior!

We have passed from absolute certain truth in Mathematics to the state where we see there is no meaning at all in the symbols—but we still use them!

Indeed, a tone of voice, a lift of an eyebrow, the wink of an eye, or even a smile, can change the meaning of what is being said.

The problems will not go away—hence you will be expected to cope with them—and I am suggesting at times you may have to invent new Mathematics to handle them.

Fortunately for Planck, the formula fitted only so long as he avoided the limit, and no matter how he took the limit the formula disappeared.

It was quickly shown by Schrödinger, Eckart, and others the two theories , though looking very much different we re, in many senses, equivalent to each other.

There are smells you can not smell, wave lengths of light you cannot see, sounds you cannot hear, all based on the limits of your sense organs, so why do you object to the observation given the wiring of the brain you have then there can be thoughts you cannot think?

Heisenberg derived the uncertainty principle that conjugate variables, mean ing Fourier transforms,

Man is not a rational animal, he is a rationalizing animal.

QM is stranger than we ever believed, and seems to get stranger the longer we study it.

We all know what we mean by “understand” until we try to say explicitly just what it means—and then it sort of fades away!

This is also true in many large organizations today; the elders are sure they know how the future should be handled and the younger members of the tribe when they do things differently are not appreciated.

By a kind of inverted logic it does allow many people to believe because they are unappreciated therefore they must be a great artist!

Creativity seems, among other things, to be “usefully” putting together things which were not perceived to be related before, and it may be the initial psychological distance between the things which counts most.

I often suspect creativity is like sex; a young lad can read all the books you have on the topic, but without direct experience he will have little chance of understanding what sex is—but even with experience he may still not understand what is going on!

My method, and it is implied above, is to saturate the subconscious with the problem, try to not think seriously about anything else for hours, days, or even weeks.

In the past I have deliberately managed myself in this matter by promising a result by a given date, and then, like a cornered rat, having at the last minute to find something!

If you cannot drop a wrong problem then the first time you meet one you will be stuck with it for the rest of your career.

An expert is one who knows everything about nothing; A generalist knows nothing about everything.

The record of the experts saying some thing is impossible just before it is done is amazing.

They simply kept the polynomial approach, though under questioning they could give no real reason for doing so-simply that was the way things had been done, hence was the right way to do things.

Outs ide the field there are a large number of genuine crackpots with their crazy ideas, but among them may also be the crackpot with the new, innovative idea which is going to triumph.

In some respects the expert is the curse of our society with their assurance they know everything, and without the decent humility to consider they might be wrong.

What you did to become successful is likely to be counterproductive when applied at a later date.

If you are passed over for an important (to you) promotion in an organization, then it will tend to affect all the relevant memories of a great career and taint them darker.

As the saying goes, “There is never time to do the job right, but there is always time to fix it later.”

You cannot even trust a machine to gather data about itself correctly!

Now you are ready to gather data, but first you fine tune the equipment. How? By adjusting it so you get consistent runs!

Careful estimates are combined with wild guesses, and the reliability of the whole is taken to be the reliability of the engineering part.

What is now called “poverty” is in many respects better than what the Kings of England had not too long ago!

Small samples carefully taken are better than large samples poorly done.

Those under you will often do what they think you want, and often it is not at all what you want!

It is characteristic of most people they keep a myopic view of their work and seldom, if ever, connect it with the larger aims they will admit, when pressed hard, are the true goals of the system.

If you optimize the components you will probably ruin the system performance.

As I said, the improvement of a component in such a machine, even where each component is apparently self-standing, still ruined the system performance!

Systems engineering is a hard trade to follow; it is so easy to get lost in the details!

I have already observed I did not immediately grasp the systems approach as I was running the computers, but at least I gradually realized the computers were but a part of a research—development organization, vital to be sure, but it was their value to the system which mattered in the long run.

The closer you meet specifications the worse the performance will be when overloaded.

The optimal strategy for the individual was clearly opposed to the optimal strategy for the whole of the laboratories, and it is one of the functions of the systems engineer to block most of the local optimization of the individuals of the system and reach for the global optimization for the system.

In a sense systems engineering is trying to solve the right problem, perhaps a little wrongly, but with the realization the solution is only temporary.

There is a great need for real systems engineers, as well as perhaps a greater need to get rid of those who merely talk a good story but cannot play the game effectively.

Accuracy of measurement tends to get confused with relevance of measurement, much more than most people believe.

If I could make up an exam which was uniformly hard, then each student would tend either to get all the answers right or all wrong.

Hence, as at Bell Telephone Laboratories, usually the research people go out to do the hiring for the research area, and the personnel department shudders!

Never mind humans are at least as complex as vectors, and probably even more complex than matrices or tensors of numbers; the complex human, plus the effect of the environment they operate in, must somehow be reduced to a simple measure which makes an ordered array of choices.

Any change in the rating system you think will improve the system performance as a whole is apt to not work out well unless you have thought through the response of the individuals to the change—they will certainly change their behavior.

If the whole organization is working together to fool the top, there is little the top can do about it.

There is never time to do the job right, but there is always time to fix things later.

Our society frowns on those who say this too loudly, but I only ask you say it to yourself!

“If what you are working on is not important and not likely to lead to important things, then why are you working on it?”

While playing chess Shannon would often advance his queen boldly into the fray and say, “I ain’t scaird of nothing”.

What had seemed to be a defect now became an asset and pushed me in the right direction!

Great people can tolerate ambiguity, they can both believe and disbelieve at the same time.

As the old song says, “It ain’t what you do if s the way thatyou do it”.

Yes, it is nice to end up where you wanted to be, but the person you are when you get there is far more important.

A plan for the future, I believe, is essential for success, otherwise you will drift like the drunken sailor through life and accomplish much less than you could otherwise have done.

Socrates, 2, 12, 359INDEX 217

Teachers should prepare the student for the student’s future, not for the teacher’s past.

I am, as it were, only a coach. I cannot run the mile for you; at best I can discuss styles and criticize yours.

In science if you know what you are doing you should not be doing it. In engineering if you do not know what you are doing you should not be doing it.

In a lifetime of many, many independent choices, small and large, a career with a vision will get you a distance proportional to n, while no vision will get you only the distance square root of n.

We are now looking more and more in books and less and less at Nature! There is clearly a risk we will go too far occasionally—and I expect this will happen frequently in the future.

I believe computers will be almost everywhere since I once saw a sign which read, “The battle field is no place for the human being”.

Often a new innovation will set the growth of a field onto a new “S” curve which takes off from around the saturation level of the old one.

You will hardly get a system installed and working before there are significant improvements which you can adapt by mere program changes.

We see the machine does not know where it has been, nor where it is going to go; it has at best only a myopic view of simply repeating the same cycle endlessly.

It has been said in physics no creator of any significant thing ever understood what he had done.

What I wanted to know was how the job of communication can be efficiently accomplished when we have the power to design the language, and when only one end of the language is humans, with all their faults, and the other is a machine with high reliability to do what it is told to do, but nothing else.

But you do not expect novelists to “engineer the production of novels”. The question arises, “Is programming closer to novel writing than it is to classical engineering?” I suggest yes!

On thinking this over very seriously, I came to the conclusion I could not afford to be crippled that way and still become a great scientist.

Your excitement tends to be communicated to the audience, and if you seem to be perfectly relaxed then the audience also relaxes and may fall asleep!

I believed then, as I do now, that cheap, small SDS 910 machine at least doubled the effective productivity of the huge, expensive cyclotron!

You simply cannot use a constantly changing data base for an optimization study.

I was angry to say the least, and said, 'If the machine can locate there is an error, why can it not locate where it is, and then fix it by simply changing the bit to the opposite state?'

Of course as you go through life you do not know what you are preparing yourself for—only you want to do significant things and not spend the whole of your life being a “janitor of science” or whatever your profession is.

All I remembered was it was one of Tukey’s few bad ideas; I completely forgot why it was bad— namely because of the equipment I had at time.

His refusal to reply, under repeated requests, was a clear admission my point went home, he himself knew the Director did not understand this difference but thought the report was the reliability of the actual shot.

You are lovingly taught how one theory was displaced by another, but you are seldom taught to replace a nice theory with nothing but randomness!

I will tell you to go back and get new assumptions—I know Cauchy’s theorem is “true”.

We have passed from absolute certain truth in Mathematics to the state where we see there is no meaning at all in the symbols—but we still use them!

I often suspect creativity is like sex; a young lad can read all the books you have on the topic, but without direct experience he will have little chance of understanding what sex is—but even with experience he may still not understand what is going on!

My method, and it is implied above, is to saturate the subconscious with the problem, try to not think seriously about anything else for hours, days, or even weeks.

An expert is one who knows everything about nothing; A generalist knows nothing about everything.

The record of the experts saying some thing is impossible just before it is done is amazing.

It is characteristic of most people they keep a myopic view of their work and seldom, if ever, connect it with the larger aims they will admit, when pressed hard, are the true goals of the system.

If you optimize the components you will probably ruin the system performance.

Accuracy of measurement tends to get confused with relevance of measurement, much more than most people believe.

There is never time to do the job right, but there is always time to fix things later.

Great people can tolerate ambiguity, they can both believe and disbelieve at the same time.

A plan for the future, I believe, is essential for success, otherwise you will drift like the drunken sailor through life and accomplish much less than you could otherwise have done.