Information Engines Pay the Piper

Physicists sometimes get a bad rap. Theoretical physicists even more so. Consider Sheldon Cooper in the TV sit-com The Big Bang Theory:

Sheldon: I’m a physicist. I have a working knowledge of the entire universe and everything it contains.
Penny: Who’s Radiohead?
Sheldon: (after several seconds of twitching) I have a working knowledge of the important things in the universe.

Mikhael Semaan

But a working knowledge of anything is always informed and arguably improved — even transformed — by robust and analytical “thought experiments.” In fact, theoretical physics is key to advancing our understanding of the universe, from the cosmological to the particle scale, through mathematical models.

That is why Mikhael Semaan, Ph.D. and others like him spend their time in the abstract, standing on the figurative shoulders of past giants and figuring out what could happen . . . theoretically. That Semaan is also one of the celebrated postdoctoral researchers/mentors in the Science Research Initiative (SRI), is a coup for undergraduates at the University of Utah who “learn by doing” in a variety of labs and field sites.

“The SRI is awesome,” Semaan says. It’s “a dream job where I can continue advancing my own research while ‘bridging the gap’ in early undergraduate research experiences, giving them access to participation in the cutting edge alongside personalized mentoring.”

Want to learn how to bake something? Hire a baker. Better still, watch the baker bake (and maybe even lick the bowl when allowed). And now that Semaan’s second first-author paper — done with senior investigator Jim Crutchfield of UC Davis, his former PhD advisor — has just “dropped,” students get to witness in real time how things get done, incrementally adding to the trove of scientific knowledge that from past experience, we know, can change the world.

Theory’s abstraction lets us examine certain essential features of the subjects and models we study, which in Semaan and Crutchfield’s case concern the first and second laws of thermodynamics. Is it possible to run a car from the hard drive of a computer? In the parlance of this brand of physics, the short answer is, “Yes, theoretically.”

Thermodynamics of Information Processing

From that question as a jumping off point, Semaan explains further. “The primary impact of our contribution is, for now, mostly to other theorists working out the thermodynamics of information processing. … [W]e suggest a change in viewpoint that simplifies and unifies various preceding lines of inquiry, by combining familiar tools to uncover new results.”

The physicist and writer C.P. Snow said that the first three laws of thermodynamics can be pithily summarized with, “You can’t win. You can’t even break even. You can’t stay out of the game.” Semaan elaborates on the second law, “the universe must increase its entropy — its degree of ‘disorder’ — on average…[b]esides offering an excuse for a messy room, this statement has far-reaching implications and places strict limits on the efficiency of converting one form of energy to another … .”

These limits are obeyed by everything from the molecular motors in our bodies to the increasingly sophisticated computers in our pockets to the impacts of global industry on the Earth’s climate and beyond. Yet in the second law’s case, there’s a catch: it turns out that information in the abstract is itself a form of entropy. This insight is key to the much-celebrated “Landauer bound:” stated simply, learning about a system — going from uncertainty to certainty — fundamentally costs energy.

But what about the converse situation? If it costs energy to “reduce” uncertainty, can we extract energy by “gaining” it — for example, by scrambling a hard drive? If so, how much?

Ratchet Information

To answer this question, previous researchers, including Crutchfield, imagine a “ratchet” which moves in one direction along an “information tape,” interacting with one “bit” at a time. As it does so, the ratchet modifies the tape’s statistical properties. That “tape” could be the hard drive in your computer or could be a sequence of base pairs in a strand of DNA.

“In this situation, by scrambling an initially ordered tape, yes: we can actually extract heat from the environment, but only by increasing randomness on the tape.” While the second law still holds, it is modified. “The randomness of the information in the tape is itself a form of entropy,” explains Semaan further, “and we can reduce the entropy in our thermal environment as long as we sufficiently increase it in the tape.”

In the literature, the laws bounding this behavior are termed “information processing second laws,” in reference to their explicit accounting for information processing (via modifying the tape) in the second law of thermodynamics. In this new paper, Semaan and Crutchfield uncover an “information processing first law,” a similar modification to the first law of thermodynamics, which unifies and strengthens various second laws in the literature. It appears to do more, too: it also offers a way to tighten those second laws — to place stricter limits on the allowed behavior — for systems which have “nonequilibrium steady states.”

Non-equilibrium steady state systems — our bodies, the global climate, and our computers are all examples — need to constantly absorb and dissipate energy, and so stay out of equilibrium, even in “steady” conditions (contrast a cup of coffee left out: its “steady” state is complete equilibrium with the room).

“It turns out,” says Semaan, “that in this case we must ‘pay the piper’:  we can still scramble the tape to extract heat, but only if we do so fast enough to keep up with the non-equilibrium steady states.” To demonstrate their new bound, the authors cooked up a simple, tunable model to visualize how much tighter the new results are with concrete, if idealized, examples. “This sort of idealization is a powerful tool,” says Semaan, “because with it we can ‘zoom in’ on only those features we want to highlight and understand, in this case what having nonequilibrium steady states changes about previous results.”

This uni-directional “ratcheting” mechanism may, in fact, someday lead to engineering a device that harnesses energy from scrambling a hard drive. But first, beyond engineering difficulties, there is much left to understand about the mathematical, idealized limits of this behavior. In other words, we still have a ways to go, even “in theory.” There are plenty of remaining questions to address, the fodder for any theoretical physicist worth their salt.

Complex Systems

However, far from being “only” a theoretical exercise, says Semaan, “these continued extensions, reformulations, and corrections are necessary for us to be able to understand how real-world, highly interconnected, complex systems,” like the human body, forest ecosystems, the planetary climate, etc., “exploit (or don’t) the dynamical interplay between energy and information to function. Since so many of the intricate systems we see in nature (including ourselves) exhibit non-equilibrium steady states,” he continues, “this is a [required] step to understanding how they [do this].”

Information ratchet system: At each time step, the ratchet moves one step to the right along the tape, and interacts with one symbol at a time. As it does so, it exchanges energy in various forms with its environment — signified by the T, aux, and λ bubbles in the picture. After running for a long time, the “output tape” generated by the interactions with the ratchet has different statistical properties compared to the “input tape” it receives. The information processing first and second laws are statements about the fundamental relationship between the energy exchanged with the environment and the information processing in the tape. Credit: Semaan and Crutchfield.

This is heady stuff, and the Southern California native is positively thrilled to be sharing it with young, eager undergraduates at the U through the SRI. Semaan is keenly aware of how critical the undergraduate experience in research needs to be to turn out future physicists. A son of Lebanese immigrants who both attended college in the U.S., neither were research scientists and no one he knew had studied physics. At California State University, Long Beach, where Semaan first declared electrical engineering as his major, he was “seduced into physics” through a series of exceptional and inspirational mentors. In the SRI, he hopes to carry this experience forward, and open new doors for undergraduate students.

It was the Complexity Sciences Center at UC Davis, when he applied to graduate school, that caught his attention because of its interdisciplinary nature and concern with systems in which “the whole appears to be greater than the sum of its parts.” The study of emerging systemic behaviors, helmed by Crutchfield, the Center’s Director, ultimately inspired both his PhD and his decision to join the SRI, working with students across the entire College of Science.

Following the third law of thermodynamics, Mikhael Semaan clearly “can’t stay out of the game” (nor would he want to), but one could argue he’s more than breaking even at it.

The release of this paper, titled “First and second laws of information processing by nonequilibrium dynamical states” in the journal Physical Review E is proof of that.

by David Pace