new arrangements and the flash of genius

Came across this nugget from Wittgenstein's Philosophical Investigations:

The problems are solved, not by giving new information, but by arranging what we have long since known.

Another way of stating this in terms of information theory, I think, is to say that new information consists now of applying past knowledge into new arrangements.  Consider this interesting exchange from the movie Flash of Genius:

You have a Ph.D in electronic engineering, is that correct?
Uh, that's correct. I've taught for the past several years...
No, that's fine, sir. Your credentials are already part of the record. Now, when you said earlier that Mr. Kearns didn't create anything new, could you explain what you meant by that?
 Yes. As you can see, Dr. Kearns's basic unit consists of a capacitor a variable resistor and a transistor. Now, these are basic building blocks in electronics. You can find them in any catalog. All Mr. Kearns did was to arrange them in a new pattern, you might say. And that, that's not the same thing as inventing something new, however.
 Did Mr. Kearns invent the transistor?
 No, sir, he did not.
 Did Mr. Kearns invent the capacitor?
 Again, no, he did not.
 Did Mr. Kearns invent the variable resistor?
 No, he did not.
Thank you, Professor.

Or more amusingly, in an episode of the series "3rd Rock From the Sun," High Commander Dick Solomon tries to discredit a novelist by pointing out that he has used information that is accessible to anyone:

[to Jeff] You think you're pretty clever, don't you? I happen to know that every word in your book was published years ago! [to everyone in the room] Perhaps you've read...The Dictionary!

Jeff wrote his novel by merely recombining meaningful words, and Robert Kearns merely rearranged the previously discovered transistors, capacitors, and variable resistors into new parts.  In the first instance, plagiarism would consist of stealing Jeff's particular improbable combinations of words, and in the second instance, intellectual theft consisted of stealing Kearn's ingenious arrangement of standard components.

Consider now that Charles R. Marshall and other biologists have argued that the Cambrian Explosion might have resulted from "simply rewiring the existing genetic networks." To say that this solves the informational problem is like saying that all I need to do to create useful computer programs is to "tweak" how the procedures invoke one another and how data flows between them.

Pierre-Paul Grassé wrote, "If to determine the origin of information in a computer is not a false problem, why should the search for the information contained in cellular nuclei be one?"  Indeed.

islands of functionality and the flash of genius

Hazen et al illustrate some alternative ideas of functional protein accessibility in protein space, where the plane represents the dimensionality of protein space (2 is much, much fewer than what would be needed) and the E axis represents the catalytic usefulness of the various points in protein space (in essence, a fitness landscape):

D has more of a needle-in-a-haystack problem than A, B, or C, due to its relatively small hypervolume (area in the figure) of protein space.  But it's not the only aspect that makes in inaccessible.  The relative distance from other islands of functionality.  Using the above figure a little out of its intended representation, we see that B and C are relatively close together, so that not as vast a distance of neutral mutation would have to be crossed to get from B to C as from A to D.

But vast oceans of neutral mutation to be crossed are not the only impediment to finding the points of especially high functionality ... There is also the fact that less optimal peaks might serve as attractors that divert computational resources away from the brass ring.  In this case, the good is the enemy of the best.

D of Hazen et al's figure above corresponds to this diagram from one of Douglas Axe's papers, where sequence/protein space is represented by only one dimension:

Here the white noise of suboptimal adaptations might be considered negligible, all of the being more or less neutral in that they don't change the survivability rate of the organisms enough to inhibit the traversal of sequence space.  It is possible that the neutral is filled with many little hills and valleys, a so-called rugged landscape.  In the Picasso-esque landscape below, it may be that the difficulty in finding the high peak of innovation is compunded, both by the volume of sequence/protein space to search but also by the attractive "force" of suboptimal solutions.

The size of the relevant space to be searched along with the distractive force of more accessible (more "obvious") solutions might contribute to the Non-Obviousness of the more optimal solution.

It would seem that both of these have relevance to Bennett's concept of "logical depth", as they both may drive up the necessary computational resources (or, the amount of brute force "tinkering") to realize the non-obvious solution -- where a flash of genius might render all that brute force tinkering unnecessary.  

In Shadows of the Mind, in which Roger Penrose argues for mathematical insight requiring something beyond computation, Penrose has a section on "Things that computers do well -- or badly":

Conscious understanding is a comparatively slow process, but it can cut down considerably the number of alternatives that need to be seriously consideredand thereby greatly increase the effective depth calculation.  

In other words, a flash of insight can cross large distances of "logical depth" a la Charles H. Bennett.  Insight is like a wormhole, a directed wormhole, through solution space.