Shannon Entropy (ΔS)

Target Optimization

“What mattered was not how much visual information an image contained. It was how unevenly that information was distributed.”

The Correlation of the Gradient of Shannon Entropy and Anomalous Cognition: Toward an AC Sensory System, Journal of Scientific Exploration 14(1), 53–72.

In 2000, May, Spottiswoode and Faith published a finding that ought to have reshaped how every remote-viewing target pool is built. Working from the hypothesis that anomalous cognition behaves like a sensory system — responsive to change in its input rather than to the level of the input, the way vision responds to changes in brightness across a field rather than to absolute illumination — they tested whether AC performance correlates with the spatial structure of the target image itself. It did. Quality of correspondence, measured by fuzzy-set figure of merit against 75 trials from five experienced receivers, correlated with the gradient of Shannon entropy across the target (rs = .212, p = .034), and pointedly did not correlate with the total entropy of the target (rs = .042).

What mattered was not how much visual information an image contained. It was how unevenly that information was distributed.

Their computation was straightforward. Divide the photograph into square patches, histogram the raw 8-bit intensity values in each patch separately for the three colour planes, take the Shannon entropy S = −Σ p log₂ p of each histogram, sum the planes, and report ΔS as the average magnitude of the two-dimensional gradient of that patch-entropy field. They tested patch sizes of 4, 8, 20, 40 and 100 pixels against 800×600 photographs; the correlation held for the small sizes and died for the large ones. Every target pool we generate is screened against a descendant of exactly this quantity, and we implemented their method faithfully as the baseline. We then found it wanting in ways that matter — some obvious from a modern vantage point, some not — and the algorithm we run today shares little with the original beyond its purpose.

The first fault is structural. A histogram of intensity values is order-blind: shuffle every pixel in a patch into random positions and the entropy is bit-for-bit identical. The measure cannot tell a zigzag boundary from a smooth one, a checkerboard from two solid halves, arrangement from palette. For full-frame photographs this blindness is diluted by the between-patch gradient; for our imagery it is fatal. Our replacement scores adjacency instead of membership. Every 8-connected pair of neighbouring pixels — enumerated once, four forward directions per pixel — contributes its signed per-plane difference to a histogram keyed jointly by direction and difference, 4 × 511 bins per colour plane, and the patch score is the Shannon entropy of that joint distribution. The consequences are the ones a measurement of visual structure should have: a shuffled patch no longer scores like an intact one, a zigzag edge spreads its crossings across direction bins and outscores a straight edge, while a horizontal grating and a vertical grating — equally complex, merely rotated — score identically. Related constructions appear in the texture literature as local binary patterns, co-occurrence statistics, and gradient-field entropy; ours is tuned to the specific question the 2000 paper posed.

The second fault is subtler and cost us more. Their targets were full-frame photographs; ours are isolated subjects on transparent backgrounds, and transparency breaks the published method in both directions at once. Treat transparent pixels as data and the algorithm measures the silhouette instead of the subject — thin and diagonal objects ride manufactured entropy cliffs to the top of the scale. Our first faithful implementation ranked its entire catalogue essentially by outline sparseness; the same object photographed diagonal versus upright moved from the 98th percentile to the 13th. Gate pixels by an alpha threshold instead and you tell two lies simultaneously: a pixel at 50 percent opacity is counted as fully opaque, and a pixel at 49 percent — plainly visible to any viewer — is counted as nonexistent.

The resolution is to stop treating transparency as either data or a gate and treat it as what it is: light. Every pixel is composited over the actual display background our viewers see before any difference is taken, so a half-transparent pixel contributes exactly its visible light, an antialiased edge is measured as the soft ramp it actually is, and nothing is thresholded, rounded up, or discarded. Which pixels participate at all is decided by geometry rather than opacity: the visible mass is shrink-wrapped by morphological closing — a rolling ball whose radius scales with the subject — with fully enclosed voids filled in. Negative space inside a subject is real visual structure: the gaps between a wheel's spokes alternate against its rim in exactly the way an observer perceives, and they are measured. The open bay of a concave silhouette is background, and it is not. No pixel is invented anywhere in the pipeline; the region either exists to the eye or does not participate.

The third fault is statistical. A patch entropy computed from a handful of pixels is not a small measurement, it is a fabricated one — one stray pixel yields exactly zero, two of opposite colour yield the maximum — and the published formulation happily scores such patches. We put the whole computation on a single currency: pairs. A patch is scored only when its participating pairs clear a floor set as a fraction of that patch's exact geometric pair capacity; the same pair fraction is the patch's weight in the gradient field; and each local gradient is scaled by the weakest support among the neighbours it was estimated from. Thinly evidenced regions cannot fabricate structure, cannot vote beyond their evidence, and cannot masquerade as measured flatness.

The net effect is a measure with the same intent as the 2000 result and none of its failure modes, and the practical consequences were not subtle. Recalibrating the full catalogue under the corrected algorithm rejected several thousand images — on the order of a fifth of the pool — that the published method had scored as acceptable, most of them silhouette artifacts: wireframes, sparse strands, and diagonals whose apparent complexity was an accident of their outline. What remains, and what every session now draws from, is a pool ranked by the thing the original researchers were actually pointing at: how strongly the structure of an image varies from place to place, as a viewer would see it.