The Secular Creationist: thermodynamics

Showing posts with label thermodynamics. Show all posts

Saturday, July 5, 2014

Thermodynamic Context for Fisher Information Matrices

The following is from Wikipedia on the reparametrization of a Fisher Information matrix:

The Fisher information depends on the parametrization of the problem. If θ and η are two scalar parametrizations of an estimation problem, and θ is a continuously differentiable function of η, then

${\mathcal I}_\eta(\eta) = {\mathcal I}_\theta(\theta(\eta)) \left( \frac{{\mathrm d} \theta}{{\mathrm d} \eta} \right)^2$

where ${\mathcal I}_\eta$ and ${\mathcal I}_\theta$ are the Fisher information measures of η and θ, respectively.^[12]In the vector case, suppose ${\boldsymbol \theta}$ and ${\boldsymbol \eta}$ are k-vectors which parametrize an estimation problem, and suppose that ${\boldsymbol \theta}$ is a continuously differentiable function of ${\boldsymbol \eta}$ , then,^[13] ${\mathcal I}_{\boldsymbol \eta}({\boldsymbol \eta}) = {\boldsymbol J}^{\mathrm T} {\mathcal I}_{\boldsymbol \theta} ({\boldsymbol \theta}({\boldsymbol \eta})) {\boldsymbol J}$

where the (i, j)th element of the k × k Jacobian matrix $\boldsymbol J$ is defined by

$J_{ij} = \frac{\partial \theta_i}{\partial \eta_j}\,,$

and where ${\boldsymbol J}^{\mathrm T}$ is the matrix transpose of ${\boldsymbol J}$ .
In information geometry, this is seen as a change of coordinates on a Riemannian manifold, and the intrinsic properties of curvature are unchanged under different parametrization. In general, the Fisher information matrix provides a Riemannian metric (more precisely, the Fisher-Rao metric) for the manifold of thermodynamic states, and can be used as an information-geometric complexity measure for a classification of phase transitions, e.g., the scalar curvature of the thermodynamic metric tensor diverges at (and only at) a phase transition point.^[14]In the thermodynamic context, the Fisher information matrix is directly related to the rate of change in the corresponding order parameters [emphasis mine].^[15] In particular, such relations identify second-order phase transitions via divergences of individual elements of the Fisher information matrix.

I wonder how this could be extended to the dispersion of mutations through configuration space.

Some possible sources for further inquiry:

Information Geometry by Arwini and Dodson
Information, Physics and Computation by Mezard and Montanari

Also Information Geometry: From Black Holes to Condensed Matter Systems, an editorial by Tapobrata Sarkar, Hernando Quevedo, and Rong-Gen Cai.

And for black hole thermodynamics look at Information geometries for black hole physics by Narit Pidokrajt, as well as Information Geometric Approach to Black Hole Thermodynamics by the same author.

The figure below is from Frederic Barbaresco's "Eidetic Reduction of Information Geometry" in Geometric Theory of Information by Frank Nielsen.

Monday, September 16, 2013

2^219 states

2²¹⁹ states

But that's just one star, and a measly one at that. A typical supernova releases something like 10⁵¹ ergs. (About a hundred times as much energy would be released in the form of neutrinos, but let them go for now.) If all of this energy could be channeled into a single orgy of computation, a 219-bit counter could be cycled through all of its states.
https://www.schneier.com/blog/archives/2009/09/the_doghouse_cr.html

Wednesday, September 11, 2013

The Grammar of DNA

Similar to David Abel's(?) distinction of change contingent and choice contingent variation is Lila Gatlin's D1 and D2 distinctions. D1 is "context-free" variation in the statistic Shannon sense and D2 is "context sensitive" in a grammatical sense. Gatlin is/was a biochemist/biophycisist and is not using these terms in their specific computer science meanings, where a grammar-generated language is specifically referred to as context-free.

Gatlin referred to a Law of Conservation of Information, not too coincidently similar to Dembski's interpretation of Wolpert's. Since at that time, the modern ID Movement had not taken shape, and the guardians of the light of science were only concerned with creation science based on religious axioms, Gatlin wasn't derided, marginalized, and silenced the way that ID proponents now are. She was published in Nature and the Journal of Molecular Biology (as has Douglas Axe in the latter, if I recall). Jeremy Campbell leaned quite a bit on her ideas for his Grammatical Man, which garnered praise from mathematician and Skeptic (in the anti-supernatural and anti-creationist sense) Martin Gardner. Surely, Gardner would be expected to have a negative reaction to a speculative book that depended on pseudoscience. Way back in the 70s, you must realize, these ideas weren't pseudoscience.

This gets back to my Conjecture: The opposition leveled at an idea in biology will be in proportion to the perceived utility of the idea to supporting theistic belief. Professional skeptics would not close ranks against this idea until it was associated with the idea of Intelligent Design. Note here that all 2nd Law of θ∆ics arguments are summarily dismissed with a "guilt by association" argument (you know, one of them there fallacies) of the Barbara Forrest kind.

Anyway, Gatlin recognizes that the statistical measure of information of the Shannon sort is insufficient to reason about the interrelatedness of grammatically organized information that has semantic value. Where here analysis possibly falls short, is that "grammatically correct" strings, or strings that follow grammatical rules, do not necessarily convey meaning, and in fact, are usually nonsense. But it is certainly a step in the right direction. There is a limited context-sensitivity in grammar that is relevant to protein structure. Subject must have its predicate and vice versa. In English, the choice of a plural noun will affect the form of the predicate verb. In protein space, particular "choices" in one part of the protein will have consequences for residue choices in remote parts of the same protein (or possibly, in a part of a co-enzyme). The context-freedom in grammar is that rules only get you so far:

Greenness is a square light bulb, that grazes in the death of fragrance. A bus, on the other shoe, does not swim with the lug nuts, but flies in a river of boots.

Grammatically correct, more or less, but not likely to inform the hearer about much of anything. And for every meaningful sentences, there are many combinatorially more that would be unlikely to be useful sentences in any context. The birth of tables preceded the art of melting dishes by a trillion dollars. My sofa isn't feeling well.

See far ogle.

Thursday, September 5, 2013

macroscopically describable

If we repeat an experiment 2^k times, and define an event to be “simply describable” (macroscopically describable) if it can be described in m or fewer bits (so that there are 2^m or fewer such events), and “extremely improbable” when it has probability 1/2^n or less, then the probability that any extremely improbable, simply describable event will ever occur is less than (2^(k+m))/(2^n). Thus we just have to make sure to choose n to be much larger than k + m. If we flip a billion fair coins, any outcome we get can be said to be extremely improbable, but we only have cause for astonishment if something extremely improbable and simply describable happens, such as “all heads,” or “every third coin is tails,” or “only every third coin is tails.” Since there are 10^23 molecules in a mole of anything, for practical purposes anything that can be described without resorting to an atom-by-atom accounting (or coin-by-coin accounting, if there are enough coins) can be considered “macroscopically” describable.
Granville Sewell, "Entropy, Evolution and Open Systems", note 5