The Wysiwyg Universe
The Wysiwyg Universe
by
Simon Free
15,725 visitors have experienced this page.
Document Version 1.2,
April 2004
Copyright © 2002-2004 Simon Free
Permission is granted to copy, distribute and/or modify this
document under the terms of the Open Content License.
View Table of Contents
Introduction
WYS.I.WYG /'wi-zE-"wig, adjective: What YouSeeIsWhatYouGet
Much has been written in the fields of
philosophy, psychology and physics regarding the role of the observer
in understanding the nature of reality. Physicists are zeroing in
on what they hope to be a "Theory of Everything", a set of mathematical
equations that describe all that there is to know about the basic
building blocks of the Universe. At the same time, psychologists
and neurobiologists are closer than ever to a complete map of how the
brain allows us to see, think, feel, and even dream.
Yet despite these advances in our scientific understanding, there
remain some unsettling gaps between what we say about the world
and what we actually experience. For example, no
description of water, no matter how poetically vivid or mathematically
correct, will quench your thirst. Similarly, the abstract beauty
of Maxwell's electromagnetic field equations seems pale by comparison
to the myriad hues of a sunset. Even the most elegant theories of
the origins of the Universe seem somehow devoid of substance and leave
us uncomfortably unsatisfied. In the words of physicist Stephen
Hawking: "What is it that breathes fire into the equations and
makes a universe for them to describe?"
It will be argued here that these explanatory gaps arise from a
simple ontological error commonly committed when discussing what we
take to be reality. At the root of this confusion is the assumption of
a
world beyond what we
experience. This assumption generally begins as a model
of
a set of experiences which is formulated either verbally or in the
language of mathematics. If a model is successful, it allows us
to make predictions about observations within a certain domain of
experience. However, the more successful a model becomes, the
greater is the tendency to speak of the model as though it describes
the way the world really is. But this confuses the map
for the territory. Few would argue that a map of the Rocky
Mountains actually is the mountain range, no matter how
detailed
or elaborate the map. Similarly, no neurological description of
the taste buds in your tongue and their connections to your brain will
enable you to experience a flavor you've never tasted before.
It will be shown below that by adopting two simple postulates
regarding the relationship between experience and reality, we can
eliminate the confusion which tends to arise between our models of
reality and reality itself. But before we take the plunge, let us
examine two particular cases where the apparent gap between model and
experience seems just too great to cross.
The Quandary
about Qualia
There is a long standing debate in
philosophy and psychology on the nature of qualia which are
defined as those elementary forms of experience such as color, taste,
sound etc., with which each of us is intimately familiar. Despite
great strides in the understanding of brain function over the past
century, little progress has been made toward the elucidation of how
the firing of neurons in the brain can give rise to these subjective
qualitative states.
For example, a pin prick to the finger causes nerve endings beneath
the skin to send impulses to pain centers in the spinal cord and
brain. These centers are connected to reflex motor actions which
cause the withdrawal of the finger away from the pin. However, we
not only react to the pin prick in this neuromechanical fashion; we
also feel pain. How this subjective feeling of pain
arises
from the firing of neurons in the brain is a mystery. To put it
another way, we have no idea why neurons firing in one part of the
brain give rise to pain, while neurons firing in a different part of
the brain give rise to sounds, and still others produce the experience
of colors.
It appears that qualia are made of a different "stuff" than the
cells in the brain or the molecules which make up the cells. In
fact, many have argued that there can never be a physical explanation
of how we come to have experience. Others deny that we really
have experiences at all! This apparent ontological "gap" between
the physical world and the world of experience is what philosophers
call the hard problem of consciousness or the mind-body
problem.
Quantum
Quirkiness
The second domain in which experience
does not seem to follow directly from the physical world is the realm
of the very small, such as the level of atomic and subatomic
processes. At the more macroscopic levels of experience, the
process of measuring the property of an object, such as the position of
a rock, generally yields a single value for any given state of the
object. However, in the world of electrons and other subatomic
entities, measuring the property of an object can yield more than one
result, even though the initial state of the object is the same from
one measurement to the next. For example, an electron has a
property called "spin" which can take on only one of two values, "up"
or "down". (Think of spin as a little arrow attached to the
electron, where the arrow can point either up or down but not in
between.) If a suitable measurement of spin is made on a large
number of identically prepared electrons, half the time we get the
value
"up" and the other half the time we get the value "down". For any
given electron, there seems to be no way to predict which value will be
measured.
The branch of physics called quantum mechanics deals with
this uncertainty by describing objects such as electrons using a
formula called the wave function. The wave function of a
quantum mechanical system prescribes both the set of possible values
one can obtain during a measurement of a given property, as well as the
probabilities of obtaining each result. For example, when the
wave function is computed for an electron passing through a magnetic
field, the formula admits just two possibilities: spin up and spin
down. It also prescribes probability values of 0.5 for obtaining
the value "up" and 0.5 for obtaining the value "down". So when
measuring the spin of an electron, quantum mechanics can only tell us
that the probability of measuring spin "up" is 0.5 and the probability
of measuring spin "down" is 0.5. We will definitely obtain one or
the other value, but which value we obtain is apparently
determined by the toss of a coin.
Needless to say, many physicists are not happy with this apparent
irreducible randomness in the subatomic realm. Consequently, a
number of theories and conjectures have arisen over the years in an
attempt to understand what actually determines the value of a
quantum measurement. In the case of electron spin for example,
some physicists speculate that, prior to a measurement, the wave
function of the electron actually describes a state of being both
spin up and and spin down--a 50-50 superposition of opposite
spin
states. Since we know that we only ever observe either
spin up or spin down, this superpositional state is not a
physical one. To turn this non-physical state of superposition
into a real value of the spin, the process of measuring the spin is
said
to "collapse" the wave function into one of its particular
values.
This seems to imply that the process of observation actually helps to
create the reality which we ultimately observe.
Imagine how strange this would seem at a macroscopic level.
Suppose that a rose can appear either red or yellow. On half of
the occasions we look at the rose it appears red, and the other half of
the time it appears yellow. As much as we study the other
properties of the rose, there seems to be no way of telling before hand
which color we will see--only that there is a 50-50 chance it will be
red or yellow. So what color does the rose have when we are not
looking at it? Is it both red and yellow? Is it
orange? Does it even have a color at all? Or is the
question even meaningful? And what actually happens when we do
look at the rose? We only ever see one color at a time so how
does the unobserved and ambiguous color state of the rose (whatever
this means) actually become the color we observe? Does the very
process of our looking at the rose actually cause one color to
be selected over the other?
The Wysiwyg
Hypothesis
To clarify the respective roles of
experience (qualia), models (science) and reality, we will begin with
two simple postulates.
First Postulate: Experience is the only reality for an
observer.
Second Postulate: There is no reality other than
experience.
In general, postulates are assumed to be irreducible assumptions of
a theory. However, one normally chooses these assumptions because
they seem reasonable or necessary, at least on the surface. In
this case, the two postulates may or may not seem self-evident.
We therefore begin with arguments for their reasonableness before
proceeding to the issues of qualia, quantum mechanics and a few other
puzzling issues in science and mathematics.
Arguments for the First Postulate
Since the issue is experience, and you are the only one who can
examine your own experience, ask yourself the question: "What do
I know of reality besides what I experience?" Suppose you
answer: "I know that I have a brain in my head which allows me to
have experience." Of course, you don't directly experience your
brain--you can't even see it. So the "knowledge" that you have a
brain in your head is an assumption based on what you have learned from
the cultural database of knowledge that our society has collected to
this point in time. Do you think this knowledge in any way
affects the way you experience, say, color? Most non-scientists
are probably completely unaware of the detailed neurobiology behind
color vision, yet they see colors just fine. So the model we have
of how brain activity enables us to perceive color is not the same as
the experience of color itself. Hopefully this much is clear.
Next you might argue: "While our model of brain function does
not determine how I see colors, surely the function of the physical
brain does." This is where the "map for the territory" error
creeps in. To demonstrate how this comes about, imagine that you
could peer inside your own head and actually see your brain in
action. (In fact, we can do something just like this using the
technology of functional MRI.) By viewing the workings of your
own brain, what additional knowledge do you now have that relates to
your experience of color? Let us focus on the visual subsystems
which neuroscientists know to be involved in color vision. At
first, you may not identify the various neural components in front of
you. They appear to you as various shapes and sizes--in other
words, as visual experiences. If you have read about the
brain mechanisms behind color vision or you are a researcher in the
field, you will have associations between these visual experiences and
the names and functions of each component. In the end you could
put together a verbal explanation (a model) of how you see color:
"Light passes through the lens of my eye stimulating the rods and cones
in the retina. The rods and cones activate my optic nerve which
sends impulses to my visual cortex. Here I can see specialized
neurons that have evolved to detect the ratios of wavelengths in the
original light. For example, whenever I experience the color red,
I see that cortical cells in region A are active. Whenever I
experience the color blue, I see that cells in region B are active."
In the end you have correlated one experience with another:
your subjective experience of color on the one hand, and your
subjective experience of your brain's activity (as observed in our
imaginary scenario) on the other hand. This is actually very
useful knowledge. For example, it allows you to predict that if
you were to remove or otherwise damage the cortical cells in region A,
you would probably have difficulty experiencing the color red.
However, what you have not done is observe how a physical
process (brain activity) gives rise to your experience.
What we call the "physical brain" is actually a collection of
experiences just like the experiences we are trying to explain.
It makes no difference how detailed a description or model we make of
the brain--right down to the quantum mechanical fluctuations in the
spacetime continuum if you like: each component of the
description will arise from its own collection of experiences and, as
with the map of the mountain range, even the most articulated model of
the brain is not the brain itself. The distinction between the
physical world and the experiential world is therefore seen to be
artificial; hence our first postulate: Experience is the only
reality for an observer.
Arguments for the Second Postulate
It is one thing to say that experience is the only reality for a given
observer (yourself for example). The second postulate takes this
a
step further: There is no reality other than experience.
For thousands of years, thinkers in all cultures have wondered what is
the "true nature" of reality behind our experience. Tribal
societies imagine animated spirits behind objects and events.
Religious leaders believe in the existence of an overarching
theological plan. Philosophers have argued for a world of ideal
platonic forms behind the objects we perceive with our senses.
And scientists postulate a physical or mathematical foundation to
reality. But for thousands of years, no one has come up with an
answer that satisfies everyone. Perhaps there is a very simple
reason for this: Experience is all there is.
It should be clear from the arguments for the first postulate that
ideas about a "reality behind experience" are really just models in
disguise. Such ideas rely on constructions using verbal language
or mathematics which postulate unobservable entities that give rise to
our experience. But this is just what a model is, and the
hypothesized unobservable entities are just place holders for
correlations among experiences. Furthermore, models are changing
all the time as observers discover new phenomena to be explained.
For example, the experience of gravity has been conceptualized and
modelled very differently from the time of Aristotle's physics, through
Newton's dynamics, and on to Einstein's theory of General
Relativity. But rocks fall to the ground all the same. In
any case, a model cannot be identified with something outside
our experience. There is no way out of the experiential
loop. Hence our second postulate: There is no reality other
than experience.
Applications of the Wysiwyg Hypothesis
The arguments given above are meant to
show that the postulates are not inherently unreasonable. Their
usefulness will become apparent when they are employed to solve a
number of long standing puzzles in philosophy and science. We now
proceed to the application of the postulates to the problem of qualia,
the collapse of the wave function in quantum mechanics, and to a number
of other troublesome issues, including the uncanny fit between
mathematics and experience.
Qualia, Qualia, All the Way Down
Let us return to the example given above wherein you were able to
view the activity of your own brain. Let E be an experience, such
as the experience of the color "red". Suppose that whenever you
experience E, you observe activity in a collection of cells, C, in your
brain. Note that C is itself an experience: the experience you
have when observing your brain cells and their activity. We
denote by the notation, E ~ C, the correlation you have established
between E and C. By the first postulate, E ~ C is the only
reality you know: any additional facts you might acquire about
the collection of cells, C, would simply add to the collection of
experiences which we are calling C. By the second postulate,
there is no other reality besides E ~ C. In other words, there is
no hidden "physical reality" behind C nor is there some "ethereal
reality" behind E. Since E ~ C is simply a correlation between
like entities (experiences) there is no ontological gap which needs to
be bridged between the "physical" world of neurons and molecules and
the "experiential" world of mind or consciousness. In the end,
there is only experience, all the way down.
Exorcising Quantum Ghosts
Recall from the introduction that
measurements made on identically prepared quantum mechanical systems
can lead to different results. Which result arises during the
measurement appears to be random, but each result can be assigned a
probability of occurrence by computing the wave function of the system
before the time of the measurement. The philosophical issue which
concerns physicists is that it seems to be impossible to assign a
definite value of the measured property to the object under study
before someone actually observes it.
Let us apply our postulates to the specific case of electron
spin. What do we experience during the experiment? We
experience a series of spin values, one for each electron which passes
through our apparatus. If we count up the number of spin up
electrons versus the number of spin down electrons, we find that they
appear in roughly equal numbers. That's it. That is all we
experience. According to the first postulate, reality for the
experimenter is simply this: when you measure the spin of a batch
of identically prepared electrons, about half of them have spin up and
the other half have spin down. At the present time, there does
not seem to be a way to determine which value (up or down) one will
obtain for a given electron at the time of measurement.
So let E be the experience of measuring the electron spins.
And let C be the collection of recordings of those spins--for example,
writing a tally of "ups" versus "downs" on a piece of paper. Note
that C is also an experience--the experience of recording the
spins. After all, if we did not record our results (even in
memory), there would be no way to do the analysis. We can then
establish a correlation, E ~ C between the two experiences which says
that the measurement of electron spins tends to generate a record of
equal numbers of spin ups and spin downs.
Physicists can take this correlation one step further. Quantum
mechanics prescribes a precise method for actually computing the wave
function of a system from its energy. The energy of the system is
in turn computable from other well known properties of the
system. We can therefore compute the wave function which
represents the spin experiment ahead of time and thereby predict
what the results of the measurements will be. The wave function
therefore provides a model for how the system will
behave.
Since the wave function predicts the probability of obtaining spin up
or spin down in the experiment, we can measure the spin of a large
number of electrons and see if the relative numbers of spin ups and
spin downs reflect the predicted probabilities (they do). In
essence, the wave function takes the place of the correlation symbol,
~, between our experience of measuring spins (E) and our experience
of recording the results (C). The only difference is that
we
now have a formula which enables us to predict the results ahead of
time.
What happens next is where all the confusion arises. Rather
than recognizing the wave function as a succinct representation of the
correlation between measurements and results, some physicists would
like to endow the wave function with a reality beyond its role as a
mathematical formula. The argument goes something like
this: since the wave function appears to represent the state of
the system before a measurement is made, then the system must actually be
in that state before the measurement is made. Besides confusing
the map for the territory, this is an especially puzzling view to take
of the wave function, since the very role of the wave function is to
represent the system before it has been observed. The
ontological status of the unobservable wave function is therefore, in
principle, untestable.
If we apply our second postulate, there is no reality other than
experience, to the problem of quantum mechanical measurement, we
run into no trouble at all. In the case of the electron spin
experiment, our experience includes the measurement of the spin, the
recording of the results, and the wave function (formula) which
correlates one with the other. We recognize the wave function for
what it is, a recipe for generating probability predictions.
There is no ghostly wave function in the experiment itself and
therefore no need to collapse it when making a measurement. The
observer therefore plays the same role as observers always do--they
have experiences.
Mathematics and Experience
(A more rigorous treatment of this section for the mathematically
minded will appear in a forthcoming Appendix. The version
presented here is intended for a more general audience.)
One of the more interesting puzzles in the history of physics and
mathematics is why mathematics applies to the world at all.
Einstein once said, "The most incomprehensible thing about the world is
that it is comprehensible." Other physicists and mathematicians
have asked: Why should a symbolic system as abstract as
mathematics be so good at describing physical reality?
Our two postulates about experience and reality allow us to solve
this mystery. First of all, it helps to know that all
branches of mathematics are actually variations on a single
theme. This common starting point is the definition of a set
of objects, together with a structure defined among the
elements
of the set. A structure is defined simply as a collection of
pairings between elements of the set. In fact, a structure on a
set is another set, a set where the elements are certain pairings of
elements from the original set.
For example, consider the set whose elements are the words "apple",
"banana", "red", "yellow". We write this set using the notation
{apple, banana, red, yellow}. An example of a structure on this
set would be the pairings {{apple,red},{banana,yellow}}. Notice
how the pairings of elements from the original set become new elements
in the structure set. You can immediately begin to see how such
structures can define a model. For example, the structure given
above defines pairings between "apple" and "red", and "banana" and
"yellow". If we were interested in the colors of various fruits,
this model of fruit colors would predict the right colors for apples
and bananas. A different model, say {{apple,yellow},{banana,red}}
would make the wrong predictions.
If this sounds familiar, it should. As we have seen, a model
of a domain of experience is a correlation, E ~ C, between the set of
experiences to be explained (E) and the set of recordings (C) made
during these experiences. By our first postulate, recordings of
results are just other experiences (like writing down numbers on a
piece of paper). We can therefore combine the two sets of
experiences, E and C, into a single set of experiences which we could
call U. The combined set U would look something like this: {e1,
e2, e3 ..., c1, c2, c3
...} where e1, e2, etc., are experiences in the
set E, and c1, c2, ... etc., are recordings (also
experiences) in the set C. By our second postulate, these
experiences are all we have. So the best we can do to model our
experiences is to define a collection of associations between elements
of this set. Such a model would look something like this: {{e1,c1},
{e2,c2}, {e3,c3}, ...}
which implies that experience e1 predicts or correlates with
recording c1 (and vice versa), e2 predicts or
correlates with c2, and so on. But such a collection
of parings is none other than a mathematical structure as defined
above. We have therefore shown that any model of
experience is by definition a mathematical structure. Conversely,
any mathematical structure can be viewed as a model, although the vast
majority of such structures will be very poor models when it
comes to predicting results.
Physics 101
It might seem at first glance that this definition of a model bears
little resemblance to the formulas commonly found in physics,
such
as Newton's law of motion, F=ma, or the wave function in quantum
mechanics. However, we can easily show that these formulaic
models
are just special cases of the more general mathematical structures
defined above.
Take the example of objects falling to the earth. The great
Italian physicist Galileo Galilei recorded the time it took a ball to
roll down a long inclined plane by using his pulse as watch (and later,
a water clock). We see immediately that Galileo's
experiment involved correlating one experience with another; namely,
the experience of viewing the rolling ball at different points along
the inclined plane, and the experience of feeling his pulse. Let
E be the set of experienced pulse beats and let C be the set of
positional recordings of the rolling ball. The set E can be
written as {p1, p2, p3, ..., pN}
where pk corresponds to kth pulse beat since the
release of the ball at the top of the inclined plane. The set C
can be written as {d1, d2, d3, ..., dN}
where dk is the distance the ball has rolled down the plane
by the kth pulse beat. The simplest model of the
results would be the structure {{p1,d1}, {p2,d2},
..., {pN,dN}} which says that the object will be
found at distance d1 at pulse p1, at d2
at pulse p2, etc.
Of course, this model would only work for one angle of the inclined
plane. Fortunately, both pulse beats and distances can be given numerical
representations. If we assume that the pulse beats are equally
spaced apart, then we can represent the pulse beats with integers and
write this new set as T = {1, 2, 3, ..., N} and call it "time".
Next, regardless of the units used to measure distance, we can write
their values relative to the distance traveled during the first pulse
beat. What Galileo discovered was that, independent of the angle
of the plane, the distance measurements took on the relative values D =
{1, 4, 9, ..., N2}. Our new
model therefore relates time and distance expressed numerically and
looks like this: {{1,1}, {2,4}, {3,9}, ... {N,N2}}.
Even with Galileo's crude timing measurements, it was clear that these
numbers indicated that the distance traveled by the ball (the second
number in each pair) was related to the square of the time
since
the ball was released (the first number in each pair).
What's more, this squaring relationship did not depend on the angle of
the inclined plane or the mass of the ball used to do the
experiment. These results implied two important properties of
falling objects: that objects do not fall to the ground with a
constant speed but are accelerating; and that this acceleration
does not depend on the mass of the object. This latter result
actually invalidated the model of falling objects proposed by Aristotle
who thought that heavier objects fall faster than lighter ones.
Only by slowing down the rate of motion by using the inclined plane was
Galileo able to collect the experiences required to disprove
Aristotle's claim.
Given the numerical form of Galileo's model, we can write the
results in more succinct form as D = kT2
where k is a constant of proportionality related to the angle of the
inclined plane. It is this form of the model which we generally
recognize as a mathematical or physical "law". But note that
Galileo himself actually started with a formulation of his results
exactly like the set notation used above. It was the numerical
representation of the experiment that made the results amenable to a
simple formulaic analysis. And this numerical representation was
possible only because the experiences under study were of such a nature
that numbers make a good representation of the results. How else
would you choose to represent distance? Or what could be more
natural than counting your pulse to measure time? The fact that
certain aspects of our experience such as distance, time, mass, force,
temperature, brightness, etc., can be easily represented numerically is
the reason we can turn our observations of correlations among such
properties into numerical formulas.
Genetic Gymnastics
Observations in other domains of experience, such as how the genes
in biological cells direct the construction of proteins, do not lend
themselves so naturally to numerical representation. Nonetheless,
we can still generate a model of protein synthesis in terms of the
genetic code which allows us to fully understand how genetic variation
can produce changes in the structure and function of biological
organisms. Furthermore, this model qualifies as an example of a
mathematical structure using our more general definition.
Biologists have known since the 1950's that all proteins are made up
of combinations of 20 different amino acids. They also know
that genes are composed of combinations of just 4 nucleotides.
Somehow genes direct the construction of proteins but how? The
set of all possible amino acid sequences and the set of all possible
nucleotide sequences are both potentially very large. For
example, even smaller proteins consisting of only 100 amino acids could
come in potentially 20100
variations. This number is so huge (larger than the total number
atoms in the universe) that it defies imagination. However, it
can
still be treated as a set. Similarly, there are approximately
100,000 genes in human cells which code for proteins and each of these
genes contains anywhere from 1,000-2,000 nucleotides. So a gene
can come in roughly 41000 variations
which is again an unthinkably large number. The challenge before
the biologists was to figure out which sequences of nucleotides in
genes coded for which sequences of amino acids in proteins. And
you thought Galileo had it tough!
The story of how this coding problem was finally solved is one of
the most intriguing pieces of detective work in the history of
science. Theorists from physics, information processing,
mathematics and biology all had a crack at it. The key was the
realization that a genetic "alphabet" of four letters (nucleotides)
could easily code for 20 amino acids if these letters formed at least
three letter words, one word for each amino acid. For example,
with four letters to choose from, one letter words give us only 4
possibilities. Two letter words yield 4x4 = 16
possibilities--still not enough to code for 20 amino acids. Three
letter words come in 4x4x4 = 64 variations which would be more than
enough to account for the 20 amino acids. Much theoretical
speculation was aimed at understanding how these 64 genetic codes could
be reduced to just 20 unique amino acid codes.
In the end, the solution had to be determined empirically, that is,
by actually looking inside the cells to see how segments of a gene were
transcribed into pieces of a protein--just as Galileo had to record the
positions of the rolling ball at each beat of his pulse. The
result was a mapping between three letter genetic sequences and amino
acids which bore no resemblance to the theoretical models. As we
might expect using 20-20 hindsight, Nature just threw things together
one code at a time over the course of evolution as necessity demanded
invention. The net result is a built-in redundancy whereby
multiple genetic triplets code for the same amino acid. In this
way, genes have a tolerance for genetic errors which the theorists
hadn't considered.
The empirical data laid out a mapping between three letter genetic
codes and amino acids. Our mathematical problem can therefore be
recast in terms of the set of amino acids, A, and the set of
three letter genetic codes, C. If A = {a1, a2,
a3, ..., a20} and C = {c1, c2,
c3, ..., c64} where there are 20 elements in the
set A and 64 elements in the set C, then the model of the empirical
results takes the form: {{a1,c1}, {a1,
c27}, {a2, c9}, {a2, c58},
..., {a20, c16}, {a20, c32}}
where we have indicated that a given amino acid may be coded by more
than one genetic triplet.
This is our mathematical model of the genetic code. We have
defined a structure of pairings between genetic triplets and amino
acids. One might wonder if we could take this a step further, as
Galileo did for time and distance, and come up with a formula that
would allow us to compute the value of an amino acid from the
genetic code(s) that stand for it. This would require that we had
some meaningful way of assigning numerical values to both the set of
amino acids and the set of genetic codes. Of course, we could
arbitrarily label the amino acids {1, 2, 3, ..., 20} and the codes {1,
2, 3, ..., 64}, but these values do not bear any relationship to the
properties of amino acids or nucleotides. In other words, what
would stop us from labeling the same amino acids in reverse order;
i.e., {20, 19, 18, ..., 1}?
So while it does not make sense to look for a formulaic version of
our genetic model, it does make sense to figure out how genetic
triplets are actually translated into amino acids using the processes
available to a cell. The series of steps involved in this process
is now well understood and is called protein synthesis.
Segments of genes containing many coding triplets are lined up with
another type of genetic molecule called messenger RNA. Messenger
RNA then takes its mirror copy of the genetic triplets into the
cytoplasm of the cell where some very mechanical processes in
structures called ribosomes line up free floating amino acids with the
appropriate genetic triplet along the length of the RNA molecule. When
the assembly process is complete, the string of amino acids peels away
as a freshly made protein molecule.
Summary
Hopefully it is now clearer why the world appears to obey
mathematical laws. For when we say that the world "obeys"
mathematical laws, we simply mean that there are correlations
(pairings) among elements of our experience which enable us to make
predictions from one experience to another. For some domains such
as physics, the correlations between experiences can actually be given
functional form, like the wave function in quantum mechanics or F=ma in
Newtonian dynamics. These formulaic models are possible because
the observations of interest can be given a numerical
representation. Since formulas and equations are what most of us
think of as mathematics, we marvel at how such abstract looking
constructs seem to fit so nicely with our experiences.
But from what we have shown above, even the most complex functional
equations can be defined as examples of sets with structure.
Conversely, models found in other sciences, such as the genetic code in
biology, are no less mathematical in their essence than those found in
physics. In these cases, the relevant objects under study do not
lend themselves to numerical representation, but the structural mapping
of one set of observations to another still forms a mathematical model.
Unfortunately, the undeniable success of numerically precise models
in physics has generated an implication that these models are somehow
more "fundamental" with regard to an ultimate understanding of reality
than the models used in, say, biology or psychology. However, it
is important to realize that even the most successful grand unified
theory in physics will be essentially mute when it comes to many of the
more macroscopic aspects of our experience. For example, the
structure of the Empire State building is entirely compatible with the
laws of physics, but those same laws cannot predict the existence of
the Empire State building. In the same way, the model of the
genetic code described above cannot be reduced to the laws of
physics: the genetic code is a result of billions of years of
haphazard biological evolution, and while these evolutionary processes
are consistent with the laws of physics, they are not necessitated by
them.
The Wysiwyg hypothesis allows for "fundamental laws" at all levels
of experience. There is no a priori reason to expect that
one domain will be in some sense more fundamental than all the
rest--after all, all such domains are ontologically equivalent insofar
as they are all elements of our experience. For example,
attempts to force an explanation of qualia in terms of Planck-scale
quantum fluctuations in the spacetime continuum seem somewhat
misguided. From our experience of brain function, the
relevant level of description is that of neurons, their
interconnections and their chemical messengers. And it is at
this level that we are most likely to find the relevant correlations
between qualia and brain mechanisms.
Experimental Support for Wysiwyg
The Wysiwyg postulates seem to clarify
many otherwise confusing issues in philosophy and science.
However, is there some empirical set of observations which give the
postulates a firmer ground on which to stand? Surprisingly, the
answer is a qualified "yes".
The Wysiwyg postulates claim that there is no "objective" reality
other than experience--nothing behind the scenes pulling the strings or
holding everything together. A similar implication can be drawn
from the fallout of a paradox first described in 1935 by Einstein and
two co-authors, Boris Podolsky and Nathan Rosen. It is therefore
referred to as the Einstein-Podolsky-Rosen (EPR) paradox.
The EPR paradox was meant to show that the quantum mechanical
description of the physical world was not complete. The
apparently irreducible randomness at the quantum level of matter
greatly annoyed Einstein who felt that "God would not play dice with
the Universe." He therefore speculated that there must be "hidden
variables" lurking behind quantum measurements which, although they had
not yet been observed, would allow us to explain the random behavior of
the results once the variables had been identified.
The basic setup of the EPR paradox is fairly easy to describe in
general terms. Suppose we have two electrons A and B which are
initially prepared such that if one of them has spin "up" then the
other must have spin "down" and vice versa. Such spins
are
said to be "correlated" and can actually be produced in the lab.
Now imagine that, before we actually measure the spin of either
electron, we allow them to fly apart so that they are now separated by
a very large distance. If we then measure the spin of electron A
and find that it is "up", quantum mechanics must predict that a
measurement of the spin of electron B will yield a value of
"down". The question becomes: how does electron B "know"
that our measurement on electron A resulted in a value of spin
up? We can move the two electrons as far apart as we like so that
even a light signal could not communicate the result of the measurement
on A fast enough for B to be informed of the result. Einstein
called this hypothetical result "spooky action at a distance" and
concluded that the only way quantum mechanics could handle such a
situation was to admit that additional hidden variables were at work
which determined the spins of both electrons before they were
separated. In other words, what you see is only part of what you
get.
Due to the technical sophistication required to actually carry out
the correlated spin experiment, a resolution of the EPR paradox had to
wait another 30 years. In the meantime, an Irish mathematical
physicist named John S. Bell took a keen interest in the EPR
argument. Bell wondered, what if we accept the EPR hypothesis and
assume that there are hidden variables behind the spin
experiment? Bell was then able to prove a theorem which showed
that for any such hidden variable theory, he could derive an
equation (actually, an inequality) which put severe constraints on the
types of experimental outcomes such a theory could predict. The
inequality derived from Bell's Theorem for the case of
correlated electron spins clearly showed that the results predicted by
quantum mechanics could not be accounted for by any hidden
variable theory. What's more, Bell's work led to the design of
actual experiments which could test Bell's inequality in the lab and
hence determine whether or not quantum mechanics was correct, despite
the EPR paradox.
A series of experiments followed, the most famous being that of Alan
Aspect and co-workers. These experiments are very similar in
principle to the original EPR thought experiment. The results are
fairly unequivocal: quantum mechanics is right and Einstein was
wrong.
Bell's Theorem and the subsequent experimental vindication of
quantum mechanics by Aspect and others has been widely regarded as "the
most profound discovery in science." To appreciate the full
impact of Bell's work, the EPR paradox can be recast so that it pits
quantum mechanics against three fundamental assumptions about reality:
1. Deductive logic is valid.
2. No signal can travel faster than light.
3. There is an objective world independent of our experience.
Since the quantum mechanical predictions for the results of the
Aspect experiment were confirmed, one or more of these assumptions must
be wrong. There is little reason to date to reject either
deductive logic or the wealth of experimental evidence behind the
maximum speed limit imposed by light. This leaves assumption 3
which, if rejected, becomes the equivalent of our second Wysiwyg
postulate. In this way, one can see that the Wysiwyg hypothesis
is not so far fetched as one might seem--in fact, it appears to find
direct experimental confirmation in the physicists' lab.
Wysiwyg
versus Mysticism
The Wysiwyg hypothesis puts all the
emphasis on experience. Another pattern of thinking which tends
to emphasize experience is "mysticism". No doubt there will be a
temptation to equate the Wysiwyg view with the picture of the world
described by various mystical schools. However, this would
be a mistake.
First of all, unlike science, there is no one set of procedures or
generally accepted truths about the world which we can lump into a
single category called "mysticism". Scientists, by and large,
speak the same language the world over. The results found in one
lab need to be reproducible in anyone else's lab before the results
enter the general database of scientific knowledge. New theories
in science must be testable, at least in principle, and any data
collected which contradicts the predictions of a theory necessarily
invalidates it. It is very difficult to make a living in this day
and age pedaling false science--someone will call your bluff in very
short order.
Mysticism generally begins with a simple process of reflection
called meditation. The basic idea is to allow experience
simply to "happen", without preconceptions and without intellectual
analysis. In of itself, this may be the only way to truly
appreciate the essence of experience as a foundation for all that is
real. But from this simple beginning, whole schools of mystical
thought have sprung forth over the millennia. Countless volumes
are written on the insights gained through meditation regarding the
true nature of reality and our place in the cosmic order of
things. Alas, as testable models of the world, most of these
descriptions fall far short of even basic commonsense checks which
anyone can perform in their daily lives. Needless to say, without
testability, many such ideas are ripe for suggestibility, preconception
and autosuggestion. Furthermore, many such descriptions date back
thousands of years and seem starkly out of place in today's
world. It would be as if we continued to follow Aristotle's model
of physics simply because he was considered to be an authority two
thousand years ago!
As we have seen in previous sections, the Wysiwyg Universe is a
Scientific Universe. There seems to be no end to the correlations
among the rich and varied experiences each and every one of us enjoys
on a daily basis. And these correlations tend to be agreed upon
by anyone who cares to look. Should we happen to find new
experiences during the course of meditation, these will simply add to
our repertoire of experiences. If there are correlations between
these experiences and others, time will reveal their structure and they
will become generally known amongst a greater number of people.
Perhaps the closest parallel one can find between the Wysiwyg
hypothesis and mysticism can be summed up by the famous anonymous Zen
saying:
Before satori (enlightenment), the mountain is a mountain
During satori, the mountain is no longer a mountain
After satori, the mountain is a mountain again
Objects and
Space
(This topic is treated in greater mathematical detail in Appendix A.)
Take a garden variety rock--no
really--pick
one up, heft it, turn it around in your hand. Feel the coolness
of
the stone, the roughness of the surface, its weight against your
palm.
Perhaps it is colored in some interesting way or just a dull
grey.
If it has some crystalline structure, it might sparkle in the
sunlight.
Rotate the rock in your hand and notice the changing visual image. Toss
the rock a few feet away and watch it fall through the air to the
ground
with a thump. What is this thing, this rock?
By and large, our experiences seem to come packaged together as objects
which are arranged in space. Traditionally, we say that
these
objects have certain properities such as shape, hardness, color, etc.,
which we come to know by way of our senses. And the arrangment of
these objects in space is defined by their placement and orientation
relative
to some frame of reference (usually one's body). It is difficult
not to think of objects
as something "out there" that somehow give rise to our
sensations. But from the Wysiwyg point of view, what we call
objects
are just those sets of experiences which are correlated in a certain
way.
Of course, it's easy to be swayed by the third person point of view,
where objects really are outside the brain. But from this same
perspective,
brains
are (ontologically) no different than the objects they are trying to
comprehend. For example, if we are trying to build a computer
vision system that can recognize different objects, the computer is
obviously just another object. So objects are certainly outside
the brain, but they are not outside experience.
We can clarify this point by taking the first person point of
view.
Here the recognition of objects has the flavor of discovery.
Imagine for a moment that you are having your first visual
experience.
All you see is a mosaic of color, shading, maybe some lines or
edges.
Now suppose you reach toward a smudge in your visual field and grasp
something
that the rest of us call a "rock". If you now rotate your hand,
the
smudge that is the rock forms a succession of visual experiences that
are
related by the structure of rotations in three dimensional space.
It is the rotational cohesion of these visual perspectives that defines
the rock as bounded and rigid and therefore allows it to "pop out" from
the visual background as an object. Note also that it is your
action that generates these transformations. If,
instead
of rotating the rock, you were to move it left and right, you would
generate
a series of visual experiences that trace out a translation in space
rather
than a rotation.
We are tempted to say that the rotated or translated rock forms a
cohesive
succession of visual images because it is a rigid object.
But from the Wysiwyg perspective, it is precisely the correlated nature
of these experiences which enable us to call the rock an object in the
first place. If we replace the rock with a walnut, the visual
experiences
will be different, but the correlations between successive experiences
during the rotation or translation will be exactly the same as
with
the rock. (Mathematically, we say that the succession of visual
experiences
forms a group; in this case the group of rotations and
translations
in three dimensional space.) What defines a visual experience as
a rigid object is therefore not some property of the object "behind the
experience" but the correlations among our changing experience.
If
you were to grasp a different smudge in your visual field that the rest
of us call "water" and you then rotated your hand, the resulting visual
experiences would not be of a rigid rotation but rather a "flowing"
from
your hand to the ground. In this case, you would be unlikely to
consider
the experience an "object" at all.
So powerful is this rotational cohesion that we can see objects in a
field of random dots, as long as those dots move in a way consistent
with a rotation. Click on the icon below for a demonstration:
 Click to see an
AVI movie (5Mb filesize). Notice how the square disappears as soon as
the rotation stops.
Physicists and mathematicians have a special word for this kind of
transformational
structure: it is called covariance. In fact,
all
meaningful "objects" in physics are defined by their covariant
properties
with respect to some set of transformations (such as rotations and
translations).
In other words, the features we observe during our interaction with
some
phenomenon must co-vary with our actions in a lawful manner before we
will
consider it an object. Have you ever noticed a spot on your
windshield
but weren't sure if it was really a spot or some distant object in the
sky? And did you notice that the way you could tell them apart
was
to move your head a little? A spot on the windshield will appear
to move one way with a tilt of your head, and a distant object will
move
another way. The resulting experience immediately disambiguates
the
two possibilities.
But there is more. Not only do covariant transformations
define
objects, they also give away the structure of the space in
which
those objects interact. Consider again the rotation or
translation
of the rock in your hand. The correlated visual experiences
define the rock as a rigid and bounded object; but these same
transformations
define
space itself as three dimensional and (locally) Euclidean.
Mathematically we say that the succession of visual experiences defines
a representation of the rotation and translation group in three
dimensions, symbolized as SO(3). In other words, objects and
space
are two sides of the same correlational relation. Einstein once
said
that physical objects are not
in space; rather, objects are spatially
extended. By this he meant that empty space has no meaning.
We can now see that the notion of an object as a "thing in itself" is
also
meaningless: both objects and space are simply special
correlations
among our experiences.
A key point about the Wysiwyg view of objects and space is that
these
correlational patterns generally arise from our own actions. For
example, most objects do not oblige us by spinning on an axis against a
fixed background. Instead, we must literally reach out and
initiate
a change in our experience before the necessary transformations can
take
place. No doubt most of us take care of these important
exploratorations during infancy. It also suggests that what
constitutes an
object may be different for other species. For example, birds do
a lot of flying and not much reaching and grasping. The kinds of
transformations induced by flight (optic flow patterns) characterize
objects
as flat or round, broad or narrow, moving or stationary, coming toward
or moving away. A balcony railing to a bird is a perch, not
something
to keep it from falling off the balcony. The psychologist J.J.
Gibson
was the first to develop this idea which he called perceptual affordances.
[Gibson, J.J. (1966). The Senses Considered as Perceptual Systems.
Boston:
Houghton Mifflin.] In the field of modern physics, John A.
Wheeler
has proposed the notion of participators instead of "observers"
to account for the fact that the properties of the world seem to depend
on the kind of measurements we make; i.e. the way we choose to interact
with the system under study.
Objects are defined by additional properties which also tend to form
correlated bundles. Take again the garden variety rock and this
time
close your eyes. When held in your hand, the rock has a certain
felt
shape. If you now rotate the rock within your palm using your
fingers,
the result would be a collection of tactile experiences, one for each
orientation
of the rock. These tactile experiences would be correlated in
exactly
the same manner as those of your earlier visual experiences; both sets
of experiences result from a set of motor actions which generate the
group
of rotations. In this way, the rock is both a visual object and a
tactile object with the same correlational structure.
This common covariant structure is what "binds" together the
separate
visual and tactile properties of the rock. For imagine now that
you
open your eyes while you manipulate the rock in your palm with your
fingers.
Each orientation of the rock generates both a visual and tactile
experience.
Individually, these pairs of experiences need not bear any relation to
one another; afterall, how can one compare a static visual impression
with
a single pattern of tactile sensations? However, the two sets of correlated experiences both
form a representation of the three dimensional rotation group.
What's more, these representations are isomorphic; i.e. given any pair of
visual experiences, we can always find a pair of tactile experiences
and a rotation such that the same rotation will map one visual
experience into the other at the same time that it will map the
corresponding tactile experiences into each other. In this way,
the two collections of experiences are considered the same "object".
But wait, you might object, why are these two groups of separate
experiences isomorphic in the first place? Is it not because
there is "thing unto itself", a ding
an sich, behind the experiences? According to the Wysiwyg
view, this would put the cart before the horse. Any notion of
platonic-like objects behind our experience is simply a model of the
correlations among our experiences. We will delve into this issue
in greater detail in a forthcoming section on Mind
and Matter.
The Case of Inverted Retinal Images
Ever since Johannes Kepler first discovered it in 1604, people have
puzzled
over a
curious fact of our visual system. As it turns out, when the lens
of the eye focuses an image of an object onto the retina at the back of
the eye, that image is inverted in relation to the orientation of the
object and left-right reversed. Why then don't we perceive the
world upsidedown and left-right backward?
 A rose is a rose in any
orientation.
This puzzle stems from a fundamental misunderstanding about
perception that continues to cause confusion to this day. A
widely held view about object perception is that brains "represent"
the world around them. Taken literally, the word "represent"
implies a re-presentation, rather like projecting a slide show onto a
screen. Needless to say, such an idea suggests some form of
agent or "homunculus" inside the head which then views the show.
Many researchers realize that any theory that leads to this kind of
"head within the head" only pushes the problem back one step and
therefore results in an infinite regress without explaining
anything.
The Wysiwyg viewpoint allows us to avoid this trap by understanding
the experience of objects as bundles of covariantly transforming
sensations. From the first person point of view, there is no such
thing as a retinal image to begin with. No one has ever seen
their own retinal image. (Go ahead, try it!) It is
therefore irrelevant what its orientation might be relative to the
object that produces it. In fact, the retinal image is just the
first step in a long series of neural activations that run through the
optic nerve, the thalamus and the visual cortex. At the same
time, motor inputs from the muscles that control the eyeball, head and
body are constantly producing activity in other parts of the brain such
as the motor and parietal cortex. During an encounter with an
object, all of these systems sweep through a series of activity
patterns depending on the relative motion between the object and the
observer. As detailed earlier, the result is a covariantly
changing series of experiences that define what we mean by an
object. It is the entire bundle of such sensations, together with
the transformations among them, that corresponds to an object--not some
intermediate blip of neural activity at some point along the optic
pathway. It would be rather like looking inside the CPU of a
computer, viewing the current set of 1's and 0's being processed, and
trying to understand the computer program that was running at that time.
In short, objects are not "pictures in the head". Instead,
they are correlated bundles of experiences. In Appendix A we
provide a simple mathematical example of how such covariant bundles can
exist in the brain's visual system.
Questions
and Answers
Wysiwyg and the Brain
When describing the relationship
between experience and the brain, we seem to confront a classic
chicken-and-egg problem: What comes first, experience or the
brain?
When studying brain activity and its
relation to experience, there is a tendency to draw a rather arbitrary
balloon around the head and say: everything inside the skull we
will call "internal" and everything outside the skull we will call
"external". Most of us go along with this since, after all,
brains are very much hidden away inside their owner's heads, unlike,
for example, hands or faces. Like most internal organs, we
can't see our brain, or hear it, touch it, smell it or--ahem--taste
it. So when we are confronted with the problem of, for example,
how a brain hears the singing of a bird, we are tempt
L = a × L1 + b × L2 + c × L3
where L1, L2 and L3 are
three basis generators. Furthermore, it can be shown that to
produce a finite rotation from a generator, we apply the exponential
operator to the generator. In symbols, if R(ß)
is a rotation about some
axis through angle ß,
and L is the generator of that
rotation, then:
R(ß) = exp(ß
× L)
We immediately see the similarity between rotational generators and
the time course of activity in our neural network. This enables
us to conclude that our synaptic connections need only represent three
different rotational generators since any other rotation can be
generated from a combination of these three.
Let us summarize: the visual experience of a rotating object
corresponds to a continuously changing pattern of activity in a network
of neurons sensitive to visual input. If the neural network is to
recognize this changing visual pattern as a rotation of a single object
rather than a random series of images, it must somehow "abstract away"
the rotation. As we have seen, this is much easier than one might
think. For the entire rotation group in three dimensional space
can be generated by just three infinitesimal rotations.
Furthermore, these three generators can be easily encoded in the
synaptic connection strengths between neurons. Finally, the
operation
that maps small rotations to finite rotations about an arbitrary axis
in space is the same operation that governs the time course of neural
activity from one moment to the next. For a more in depth
treatment of the analysis presented here, see s said to construct the experience of sound from
the
raw sense data of vibrating air. Of course, the psychologist
doesn't actually have access to your experience, so the best he can do
is say that your brain constructs a neural representation of a bird
singing. This is the third-person perspective--the
neurospsychologist is the third individual viewing the interaction
between two other individuals, you, or rather your brain, and the bird.
The first-person perspective is simply what you experience.
From this perspective, the experience of the bird song really is over
there coming from the bird sitting on the branch--not somewhere in
your brain. In fact, you have no sensation of your brain activity
at all. Indeed, if you had been raised by wolves rather than in a
human society, you probably wouldn't even know that you had a brain
inside your head. But you would presumably still have the
experience of the bird singing over there on the branch. No
matter how much physics or neurobiology someone throws at you, from
your point of view, the bird is experienced as over there on the
branch, not inside your head.
Let's now bring these two perspectives together using a simple
thought experiment. There you are, sitting in the garden,
experiencing a bird singing on a branch. To better understand the
relation between the sound of the bird and the activity of your brain,
you attach a functional MRI device to your head. The device has a
large color display which shows in fine detail the activity of the
various parts of your brain. In this way you have taken your
brain outside of your head without having to cut open your skull.
As a result, your brain can now be seen for what it is: another
object of your experience just like the bird.
As the bird continues to sing, the MRI display reveals that various
parts of your auditory cortex light up, along with regions of your
association and visual cortex, and parts of the language areas in your
left hemisphere (if you are right handed). There is even
significant activity in your medial forebrain bundle which reflects the
fact that you are enjoying the sound!
You now have both first-person and third-person perspectives on your
situation. You have the experience of the bird singing, and you
have all the data you need to understand how the brain constructs the
experience from the pattern of vibrating air striking your ear.
Or do you? The MRI display appears to you as a visual experience
which is very nicely correlated with the auditory experience of the
singing bird. In fact, the correlation is of the very same nature
as you might notice if you viewed a closeup of the motion of the bird's
beak while it made the sound. In both cases you see a correlation
between the sound experience and a visual experience. And that's
it. Just correlations among experiences--mathematical structures
which allow us to predict one experience from another. No
ontological distinction between "internal" and "external" worlds.
No need to wonder how the "physical" brain and environment give rise to
our "mental" experience.
In the Wysiwyg view, there is only experience. As an object in
the world, the brain is a collection of correlated experiences--the
experiences we have when studying the organ inside peoples' head.
This structured set of experiences which we call the brain can in turn
be correlated with other experiences we have such as the singing of
birds, the color of the sky, sensations of hot and cold, hunger and
thirst, etc. In this way we can see that experience is not a
construction of the brain; rather, the brain is a structure in
our experience.
Wysiwyg and Occam's Razor
Doesn't the Wysiwyg hypothesis violate
Occam's Razor?
When first presented with the Wysiwyg
hypothesis, one might object to the second postulate as follows:
"I can accept that experience is the only reality for an
observer. But how do we account for all the myriad correlations
among our experiences without reference to an 'external world?'
In other words, how can experience be all that there is?"
First of all, we must be clear about what is meant by the phrase
"external world". As we saw in the previous question on Wysiwyg
and the Brain, we sometimes speak as if "external world" simply
means everything "outside the brain". But the brain is clearly in
the "external world" as much as any other object and certainly cannot
be given any special ontological status a priori. So by
"external world" we must mean "outside experience". Not just
outside current experience, but outside all experience, even in
principle. (Don't forget that observations we make using
telescopes or microscopes or particle accelerators, etc., are all still
part of our experience.) The Wysiwyg hypothesis postulates that
there is no reality other than experience. The objection raised
above can now be rephrased as: "How do we account for all the
myriad correlations among our experiences without reference to anything
outside experience?"
When evaluating competing models of some domain of our experience,
scientists often use a principle called Occam's Razor.
William of Occam was a medieval philosopher who first put forth the
idea that, given a choice between two explanations of some phenomenon,
we should choose the simplest--the explanation which requires the
fewest assumptions. For example, if you assume that the Sun and
the rest of the solar system revolve about the Earth, as did the
ancient Greek astronomer Ptolemy, then you need to also assume that the
other planets follow very peculiar orbits in order to account for the
observed motions through the night sky. On the other hand, if you
assume that the Earth and the other planets revolve about the Sun, as
Copernicus argued, then no further assumptions are required to account
for the observed motions. Both the Ptolemic model and the
Copernican model can explain the observed motions of the planets;
however, the Copernican model makes far fewer assumptions and is
therefore deemed preferrable.
Readers familiar with Occam's Razor might argue that the second
Wysiwyg postulate requires an infinite number of assumptions; namely,
one assumption for every observed correlation among experiences.
Wouldn't it be simpler to postulate the existence of variables or
entities beyond experience to account for these correlations? If
the number of these variables or entities is fewer than the number of
correlated experiences, then this non-Wysiwyg view might be considered
simpler and therefore preferrable to the assumption that there is no
reality other than experience.
There are a number of problems with this argument, some of which
have already been spelled out in the earlier parts of this
document. First of all, Wysiwyg makes no assumptions regarding
correlations among experiences; they are simply observed.
If we wish to model these correlations for a particular
domain of experience, then we have to choose among the large number of
all possible models (mathematical structures) which include the
observed correlations as a subset. And it is at this stage that
Occam's Razor can play a role in choosing among different models.
Occam's Razor was not intended to be applied to the set of all
experiences--or to any experience for that matter--only to models
of a subset of our experiences. And even if we could apply the
principle to the Wysiwyg postulates, which sounds simpler: that
there is no reality other than experience; or, that in addition to
everything we experience (including what we observe through microscopes
and telescopes and infrared glasses etc.), there is a whole "world
beyond experience" which we cannot observe but which somehow holds it
all together? In this respect, Occam's Razor seems to actually
favor Wysiwyg.
But there is an even more compelling reason to let go of the idea of
a "world outside experience": it is a mistake to assume
that
by simply referring to a world outside experience, we have then
accounted for the observed correlations among experiences. For
one could just as well ask: how then do we account for the
structure in this other world beyond experience? We can best
illustrate this point with a specific example.
If we regularly experience the sight of lightning followed by the
sound of thunder, we could sum up our experience with the phrase
(simple verbal model) "thunder follows lightning". Suppose we
then ask the question: "But why does thunder always
follow
lightning?" If we are simply making things up as we go along we
might answer: "Thunder is the sound of Thor grumbling whenever he
is blinded by a flash of lightning." Of course, then we have to
explain who Thor is, why he doesn't simply cover his eyes during a
thunder storm, and why he never seems to appear in public. But if
we are doing science, the first thing we do is broaden our set of
experiences by looking for intermediate phenomena (experiences) between
the sight of lightning and the sound of thunder. So we set up
some
lightning rods and some recording devices and wait for the next storm.
When the data are collected, we notice the following chain of
experiences: electrical surge, bright light, high temperatures, low
pressure near the electrical surge, high pressure radiating out from
the region of low pressure, loud sound. We put these together
verbally and say, "An electrical surge through the air leads to a flash
of light and a hot core of air around the surge. This hot air
expands (low pressure), compressing the surrounding air (high
pressure). When the high pressure air hits our ears we hear a
loud sound." Notice that at no point in this explanation have we
appealed to entities or processes beyond our experience. But by
expanding our collection of observations, we are able to produce a more
detailed model of why thunder follows lightning.
Of course, one can play the "why" game to any level of detail you
like: Why is there an electrical surge? Why does hot air
expand? Why is there an atmosphere at all? Why does the
Earth exist? And so on. In the end, you will always end up
with a set of correlated experiences. What's more, the resulting
explanation in terms of these correlations is just what we call a
scientific model (physics in this example). If you are studying a
reductive phenomenon like lightning and thunder, then you could
conceivably collect a (nearly) "complete set" of predictive pairings
which accounts for (almost) every "why" question in the explanatory
chain. Some of the final questions might go something like:
Why are there electrical charges? Why are there electrons?
Why is there matter at all? Why was there a Big Bang? Why
did Nothing suddenly become Everything? And then you are stumped,
because the correlation {Nothing, Everything} is a pairing that makes
no sense from a scientific point of view and cannot be further reduced.
This is all just a long winded way of saying that the set of
experiences fall into predictive pairings because that is the way the
world is. If this were not the case, we wouldn't be having this
discussion. As far as we can tell, science never answers the
ultimate "why" questions. Why is the experience of "red" red
rather than blue? Why does space have three dimensions instead of
twenty? Why did the Universe spring from Nothing? Why does
anything exist at all? The best science can do is to model the
predictive pairings found among our experiences so that we can in fact
use one to predict another. If you want to know the nature of
your experiences "directly", then you should study your experiences
themselves rather than a model of their correlations. The process
of studying your experiences directly is called meditation.
Wysiwyg and Idealism
Isn't the Wysiwyg hypothesis the same
as Idealism?
The philosophical school called Idealism dates back to
Bishop
George Berkeley, an Irish philosopher and Anglican clergyman living in
the 1700's. While the Wysiwyg hypothesis may seem similar
to
Idealism on the surface, the two approaches are actually quite
distinct. In particular, Berkeley's Idealism is really a kind of theological
epistemology, as many thinkers at the time were trying to reconcile
the new empiricism of John Locke with their own religious convictions.
First of all, the Wysiwyg hypothesis is contained entirely within
the two postulates which are repeated here as a reminder:
Experience is the only reality for an observer.
There is no reality other than experience.
Berkeley used the word "idea" where we use "experience" or
"sensation"; hence, the term "idealism". The modern
connotations of the word "idea" suggests something beyond basic
experience, such as "concept" or "thought." However, at the time
of Berkeley's writing, "idea" denoted something much simpler and we can
safely substitute "experience" in its place.
Berkeley assumed that each of us has a "mind" and argued that
"matter" cannot be conceived to exist independent of the mind. In
essence, Berkeley sets himself up for a kind of implicit dualism: mind
versus matter. However, according to Berkeley, matter can only
exist if it is perceived by the mind. Furthermore, to account for
the apparent ability of things to exist even when one is not observing
them (which apparently contradicts his original premise), Berkeley held
that all things must be constantly perceived by God. (Remember,
Berkeley was a member of the clergy as well as philosopher.)
By contrast, Wysiwyg simply says that each of us has experiences and
that there is no reality other than experience. "Matter" and
"mind" are labels we have for certain aspects of our experience.
The notion of objects outside our experience is part of our model
of the world. It permits us to summarize our experience and make
predictions about other experiences. However, there is no reality
outside experience.
To the non-philosopher (which includes the author), Idealism and
Wysiwyg will probably seem like birds of a different feather. But
just in case the distinction is still unclear, we can summarize the
differences as follows:
Idealism
Wysiwyg
Reality consists of mind and matter but...
Reality is experience. "Matter" and "mind" are labels
for certain aspects of our experience.
...Matter does not exist outside of the mind.
There is no reality other than experience. "Inside" and
"outside" are also labels for certain aspects of our experience.
Objects exists even in the absence of observers because they
are continually perceived by God.
The notion of objects outside our experience is part of our
model of the world. Models of reality should not be mistaken for
reality itself.
Ever since Berkeley put forth is ideas, philosophers have been
somewhat annoyed to find that Idealism is very hard if not impossible
to refute. What bothers them most is that an "external world"
clearly seems to exist "outside of the mind". Dr. Samuel Johnson
was a prominent physician and contemporary of Berkeley. In James
Boswell's biography of Johnson, Boswell recounts:
After we came out of the church, we stood talking for some
time together of Bishop Berkeley's ingenious sophistry to prove the
nonexistence of matter, and that every thing in the universe is merely
ideal. I observed, that though we are satisfied his doctrine is not
true, it is impossible to refute it. I never shall forget the alacrity
with which Johnson answered, striking his foot with mighty force
against a large stone, till he rebounded from it -- "I refute it thus."
A refutation of Idealism, maybe. But what could be a more
poignant confirmation of Wysiwyg than a stubbed toe!
Avoiding Solipsism
The Wysiwyg postulates seem to imply
that the Universe is only what I experience. Doesn't this lead to
solipsism?
Philosophers are generally skeptical of anything that smacks of
solipsism, which is the notion that the only thing that is real is
oneself and one's own experience. It is easy to caricature this
position in such a way that the solipsist seems a least mildly
disconnected from reality. Believing that the universe is
the creation
of one's own thoughts is such an example and suggests a rather
delusional disposition.
On the surface, the Wysiwyg hypothesis sounds like solipsism.
But
then solipsism, when stripped of its misplaced psychological
connotations, is nearly a truism which few would dispute. For
example,
from a third person point of view, it is clear that everything a person
knows derives from the activity in their brain. In a strict
sense, no
one ever has direct contact with the world outside their brain.
All we
know is the product of our own neurons. Nonetheless, no one would
deny
that the ultimate cause of this neural activity includes events outside
the brain. So while it is true that a person's reality is
circumscribed by the function of their brain, it is not logical to
conclude that brain function creates the world it perceives.
Similarly, the Wysiwyg hypothesis posits that there is no reality
other than experience. But this does not imply that you or I create the universe of
experience. At any given moment, our experience is infinitesimal
compared to the vastness of all possible experience. Furthermore,
we
have very little control over what experience comes next. While
we are
immersed in the world of
experience, we do not simply make it up. It would be as if the
tail on
an elephant, being able to see only the elephant to which it is
attached, were to conclude that it created and controlled the beast
lumbering about before it.
Finally, one must carefully distinguish between experience and
knowledge. While each of us seems privy only to our own
experience,
knowledge can be shared with others. What
we acquire in this manner is a shared model of the world.
And it is this shared model of the world which can be used to dispel
any
solipsistic
delusions in the psychological sense. For example, an individual
who
has never experienced snow
might claim: "Snow does not exist." To convince them otherwise,
we
might point out all the references to snow in books and on the
Internet. Then we might enlist some of our friends to share their
own
personal experience with snow. If this is still not enough, we
might
take our friend to Canada in February and pelt him with
snowballs. The knowledge that "snow
is cold" is not the same as experiencing snow itself. And as far
as we can tell, there is no way that one individual can have someone
else's experience. So in the end, if you really want to know
something about the world for yourself, you need to experience it
yourself. All the knowledge in the world about chocolate cake
won't taste as good as the real thing. But it will come in very
handy if you want to bake a cake for yourself.
He Said, She Said
How do we explain the ability of two or
more individuals to agree on the identification of an object without
reference to a world beyond experience?
Suppose you and I meet for the first time. We might not even
speak the same language. Assume that I have an experience, B,
to which I assign the verbal label, Blue. At the same
time, suppose you are having the experience, b, and that my
verbal utterance generates in you the experience, blah.
Then for me, B ~ Blue, and for you b ~ blah.
If at some later time I utter Blue, you will experience blah.
If at the time you are also experiencing b, then by your
correlation, b ~ blah, you will agree with my
utterance. If you are not experiencing b, then
you
will disagree.
So by using a verbal label when I have the experience B, I
can induce in you a corresponding label for your experience b.
Note that as far as we know, there is no way either of us can directly
compare our experiences, B and b. It is only
through the intermediate experience of language that we can agree on
the correlations of our other experiences. Of course "language"
here simply means any method by which we can signal each other.
One need only try to understand two people talking in an unfamiliar
foreign language to appreciate the arbitrariness of assigning labels to
experience. Yet once the labels are habitually assigned, two
individuals can (generally) convey the sense of their separate
experiences to each other.
Isn't Wysiwyg just another model?
Actually, Wysiwyg is just WYSIWYG! :-) Just
remember that the finger pointing at the Moon is not the Moon.
Appendix A
In the section on mathematics and experience we discussed how all
branches
of mathematics begin with a set of objects and a structure defined on
that
set. We then defined a structure on a set as a collection of
pairings
of elements from that set. This definition allowed us to see how
mathematics provides a natural way to model the correlations among
experiences. At a very basic level, experiences seem to come
packaged together as objects
in the world. We now show how this comes about.
Rigid Objects as Group Representations
Take the example of a rock. Suppose you have never experienced
a rock before and don't even have a word for it in your
vocabulary.
We'll assume all you know right now of the rock is that there is a
smudge
of something or other in your visual field. Let us call
this
smudge X. Now suppose you reach out in the direction of X, and
having
picked up something that might be X, you rotate it with your
hand.
This interaction with X generates two sets of experiences: a set
of visual experiences, V = {v1, v2, v3,
...}, and a set of motor experiences (moving your hand), M = {m1,
m2, m3, ...} . Since your motor actions are
in step with your visual experiences, your actions generate the
structure
S = {{m1,v1}, {m2,v2}, {m3,v3},...}.
This structure is not particularly "object like" since all you have
done
is paired particular positions of your hand during the rotation with
the
particular visual experience occurring at each orientation. If
you
had begun with your hand or X in a different orientation, the
structure
S would be completely different.
Fortunately, the collection of rotations itself forms a structure of
a very precise nature called a group. The elements of a
group
satisfy the following pairing relationships. If {a,b} and {b,c}
are
pairings in the group, then so is the pairing {a,c}. This
property
is called "composition". For example, if we rotate X around a
fixed
axis first through 5 degrees ({a,b}) and then through 10 degrees
({b,c}),
then the resulting visual experience is the same as if we had rotated X
through 15 degrees from the beginning; i.e. {a,c}. A second
property
of a group stipulates that there is one element which acts as an identity.
If we symbolize the identity element by i, then any pairing {a,i} is
the
equivalent of just {a} alone. For rotations, the identity element
is a rotation through zero degrees. From a visual point of
view, the identity operation is the equivalent to no rotation at
all. The final property of a group
guarantees that for every element a, there is an inverse, a-1,
such that the pairing {a,a-1} is the equivalent of the
identity
element. If we rotate X through ß
degrees
in one direction and then
reverse the rotation through -ß
degrees,
then we end up with the same visual
experience that we started with.
The fact that rotations form a group allows us to define our rock as
a collection of visual experiences which are related by the group of
rotations
in three dimensional space. (We could include translations too
for
completeness but we'll restrict ourselves to rotations for
simplicity.) The generator of this group is the action of our
hand
during the manipulation of the rock. The set of visual
experiences
resulting from our manipulation of the rock are therefore not
arbitrarily
paired with each proprioceptive motor experience but are in fact bound
together by the structure of the rotation group. Given any visual
experience of the rock, additional visual experiences can be predicted
by applying small rotations to the original experience. The group
structure of the set of rotations is therefore the "glue" which binds
together
the various visual experiences and allows the rock to "pop out" from
the
rest of the visual field.
It is important to note that all we have done while exploring the
rock
is to move our hand. The result was a set of pairings between
motor
experiences and visual experiences. These pairings naturally
satisfy
the properties of a group--we did not impose this structure
arbitrarily.
We can therefore define a visual object as a collection of visual
experiences
which are related by the group of rotations (and translations).
In
mathematical symbols, our rock is defined by the set of visual
experiences,
V = {v1, v2, v3, ...} and the
structure
R = {{v1, v2}, {v3, v4}, {v5,
v6}, ...} where, for any two visual experiences, vj
and vk, there is an element, r, of the group of rotations
(and
translations) such that vj = r(vk). Note
how
we can now drop the motor experiences from our definition of the
rock:
the group structure is embedded in the relationship amongst visual
experiences
themselves.
Recognizing the group structure behind the objects of our
experiences
enables us to understand how experiences in one modality (e.g. vision)
can correspond to experiences in another modality (e.g. touch).
For
imagine that while manipulating our rock, we kept our eyes closed and,
instead
of rotating our whole hand, we simply turned the rock about in our palm
using our fingers. The result would be a collection of tactile
experiences
for each orientation of the rock. These tactile experiences would
have exactly the same group structure (rotations and translations) as
those
of our earlier visual experiences, since both resulted from a set of
motor
actions which themselves generate this group. In this way, the
rock
is both a visual object and a tactile object with the same
structure.
In fact, the two group representations afforded by the visual rock and
the tactile rock are isomorphic.
Two representations of a group are isomorphic if (a) there is a
one-to-one mapping (correspondence) between the elements of each group
and (b) the same mapping preserves group transformations. For
example, if we label two visual perspectives of an object as vj
and vk, while the tactile sensations for these same two
orientations are tj and tk, then if vj
= r(vk) for some rotation r, then tj = r(tk)
for the same r. Similarly, given any one pair of corresponding
visual and tactile experiences, {v1, t1},
we can (for small rotations or translations), transform each member of
the pair by the relevant group operation and predict the new pairing {v2,
t2}
even though we may never have explicitly experienced this combination
in
the past.
Objects and Brain Activity
Neurophysiologists know that visual experiences
correlate with the activity of neurons in the brain. In the
previous section, we saw that the basic objects of our world are sets
of experiences bound
together
by the group of rigid transformations (rotations and translations) in
space.
What aspects of brain structure or activity might correspond to such
objects?
Consider a collection of neurons in the brain whose activities are
correlated with some aspect of visual experience such as the pattern of
lightness and darkness in the visual field. We can represent a
particular
pattern of activity across the set of these neurons with an array of
values v = [v1 v2
v3 ... vN] where the value vj
is
the activity of the jth neuron in the collection. Each
pattern of lightness and
darkness
in the visual field is correlated with one of these activity
arrays. Note that the dimensionality of the
activity space simply reflects the number of neurons under
consideration.
It has nothing to do with the 3-dimensional space in which the
objects are viewed. For example, the N visual neurons might be
sensitive to the first N
components of a Fourier
decomposition of the original visual stimulus. A rotation
of an object in the visual field will generate a
succession of
activity patterns over these N neurons. In symbols, if vi is the initial pattern
of activity and vr
is the pattern of activity after a rotation R, then vr
= R(vi).
In the meantime, the neurons in our network are interconnected
amongst themselves via
synapses that allow activity in one neuron to stimulate or inhibit
activity in other neurons connected to it. Given any initial
pattern
of activity, vt, at
time t, the
activity will evolve in the next instant to a pattern, vt+1,
that depends on the
strengths of the interconnections. If we have N neurons in our
collection, then there are NxN possible connections between them.
The values of these connection strengths therefore form an NxN
matrix. It is easy to see that the transformation of the activity
pattern in the N neurons from one moment to the next is given by
multiplying the current array of activity values by this connection
matrix. For example, if neuron v1
is connected
strongly to neurons v2
and v3, then if v1
is quiet initially but v2
and v3 are
moderately
active, then in the next instant, v1
will become active due
to the activity in v2
and v3 multiplied
by the
synaptic strengths between them and v1.
If we let S stand for this
matrix of synaptic connection strengths, then vt+1
= S × vt. It can be shown
that this kind of matrix difference equation has a solution of
the form:
v(t) = exp(k
× t × S) × v(0)
where exp is the matrix
exponential operator, v(0) is
the pattern of activity at some arbitrary starting time t = 0, and k is a constant of
proportionality. This equation has a simple interpretation if S
is a matrix representing a rotation about at fixed axis in space.
In this case, the activity in the visual cells will trace out a series
of values that mimics a rotation of the original object about the
fixed axis coded by S.
The speed of rotation is given by the
proportionality constant, k.
Such a result is consistent with experiments done with human subjects
who are asked to compare visual objects that differ only in
orientation. In these
experiments, subjects seem to mentally rotate objects at a fixed speed
so that greater angular rotations take a longer period of time to
complete. (See for example Shepard
and Metzler, 1971). If the
neural assembly is to process rotations of rigid objects,
then the connection strengths must be modified to mirror the actions of
the rotations, R.
Mathematicians would say that the synaptic matrices must form
a representation of the rotation group
relative to the visual activation space.
The careful reader might be wondering at this point how the NxN
matrix of interconnections among our visual neurons could possibly
encode representations of all
possible rotations. After all, there are an infinite number of
fixed axes in space about which objects can rotate. The answer
lies in a subtle but powerful property of the rotation group in three
dimensions. First of all, a finite rotation through some number
of degrees about a fixed axis in space can be realized by
applying an infinitesimally small rotation about the same
axis over and over again. We call such an infinitesimal action a generator of the rotation.
When we narrow our attention down to this microscopic level, it turns
out that one only needs three
such generators to produce a finite rotation about any axis.
These generators are known as a basis for the Lie algebra associated with the
three dimension rotation group SO(3). As an algebra, we can add
and subtract these generators to produce new generators. In
symbols, if L is the generator
for any given rotation about a fixed
axis, then we can always find three coefficients, a, b an c such that:
L = a × L1 + b × L2 + c × L3
where L1, L2 and L3 are
three basis generators. Furthermore, it can be shown that to
produce a finite rotation from a generator, we apply the exponential
operator to the generator. In symbols, if R(ß)
is a rotation about some
axis through angle ß,
and L is the generator of that
rotation, then:
R(ß) = exp(ß
× L)
We immediately see the similarity between rotational generators and
the time course of activity in our neural network. This enables
us to conclude that our synaptic connections need only represent three
different rotational generators since any other rotation can be
generated from a combination of these three.
Let us summarize: the visual experience of a rotating object
corresponds to a continuously changing pattern of activity in a network
of neurons sensitive to visual input. If the neural network is to
recognize this changing visual pattern as a rotation of a single object
rather than a random series of images, it must somehow "abstract away"
the rotation. As we have seen, this is much easier than one might
think. For the entire rotation group in three dimensional space
can be generated by just three infinitesimal rotations.
Furthermore, these three generators can be easily encoded in the
synaptic connection strengths between neurons. Finally, the
operation
that maps small rotations to finite rotations about an arbitrary axis
in space is the same operation that governs the time course of neural
activity from one moment to the next. For a more in depth
treatment of the analysis presented here, see Goebel (1990), The
Mathematics of Mental Rotations, Journal of Mathematical Psychology,
34:435-444.
A Concrete Example in Flatland
Let us use a simplified two-dimensional model where our visual
objects are restricted to a single plane. In such a world, rotations
can only take place around a single axis (the axis perpendicular to the
plane), so our rotation group is only one-dimensional. (We are
ignoring translations for the sake of simplicity although they could be
handled in the same way.)
Suppose we have just
two neurons
in our neural network which code the x-y coordinates of an arbitrary
fixed point on an object. For any given orientation of the
object, the
activity of our two
neurons will code the position of this point. These values
form a two-valued array v
= [vx vy].
If we
rotate the object ever so slightly about the origin through an angle dß,
the neurons will take on new activity values v' = [v'x v'y]. Since v' will differ only a little from v, we can write the transformation
as follows:
v' = (1 + dß × L) × v
where 1 is the identity
matrix and L is the generator
of the rotation. In words we say that v' is basically the same as v (hence the identity matrix) plus a
small "twitch" of magnitude dß
caused by the generator matrix L.
Our task is to find the form of the generator matrix L.
We can pin it down by studying its behavior at two special points: [1
0] along the x-axis and [1 0] along the y-axis. The point [1 0]
will
rotate to a new location given approximately by [1 dß]
since the rotation will leave the x coordinate essentially unchanged
and will move the y coordinate about the same distance as the arc
length of the rotation, which is dß. Similarly, the point [0 1]
will be moved to approximately [-dß 1] using the same logic. Using
these two results, we can solve for L
in the equation above and we find that:
L
= | 0 -1 |
| 1 0 |
This simple matrix generates the two-dimensional rotation
group. If the 2x2 matrix of synaptic connection strengths between
our
two neurons take on these values, then any initial pattern of activity
in the two cells will transform over time in a way that mirrors a
rotation of the original pattern. Indeed, one can show that
exponentiation of this matrix leads to a finite rotation; i.e.,
exp (ß × | 0 -1
|) = |
cos(ß)
-sin(ß)
| = R(ß)
| 1 0
|
| sin(ß) cos(ß) |
which is the matrix for finite
rotations through any angle ß.
Learning Lie Generators in Neural Nets
It has been shown by Rao
and Ruderman (1999) that the Lie generators for more complicated
rotations and in larger neural networks
can be readily found by a simple
learning algorithm (gradient descent). The result
is that the synapses among neurons come to encode the generators for
rotations. If an object is then presented to the network in a new
orientation, the pattern of
activity that unfolds naturally over time as a result of the network's
interconnections is equivalent to "mentally rotating" the object.
It
is important to point out that the encoding of the rotation generators
does not depend on the particular
object(s) used during training. Indeed, the generators encoded in
the
synapses do not correspond to any property intrinsic to the objects
themselves, but to the abstract group structure of the way these
objects transform when we manipulate them. It is the
covariant
behavior of changing patterns
of visual experience that signifies the presence of fixed objects.
Acknowledgments
Many thanks to Jeffrey A. Gray whose penetrating discussions on the
nature of consciousness got the whole ball rolling for this document.
References
Harrison, David M. (1999), Bell's
Theorem
Hayes, Brian (1998), The
Invention of the Genetic Code
Whitaker, Andrew (1998), John Bell and the
Most Profound Discovery of Science
Document Version 1.2, April 2004
Copyright © 2002-2004 Simon Free
Permission is granted to copy, distribute and/or modify this
document under the terms of the Open Content License.
|
|
