In the early 1870's, Eadweard Muybridge produced the first stop-action images of trotting horses, proving that at one point in the gait of a fast trotting horse all four feet were off the ground.
Now, scientists who study biomechanics are using high speed digital video to track more fleeting movements like the stutter-step flight of butterflies and the frenetic skitter of cockroaches.
In one of the most recent experiments, Dr. Michael Dickinson at the California Institute of Technology in Pasadena used high-speed digital videotape, computer analysis and a crow-size robo-fly immersed in mineral oil to show what happens when fruit flies make right-angle turns in midflight.
Muybridge, whose work gave rise to motion pictures, worked as the machine was coming into its reign, writes Rebecca Solnit in the recently published " River of Shadows: Eadweard Muybridge and the Technological Wild West." His photographs allowed viewers to see the previously smooth, or blurred, organic movement of the horse "in a mechanical way." As Ms. Solnit points out, the horse was, then, an important biological machine "state-of-the-art power for war, agriculture and transportation."
Dr. Dickinson and his colleagues work in the information age, with digital tools. Their images are captured in gigabytes, and for many of them, mechanics is only a point of entry into an animal's neurobiology < how it processes information.
"We're really interested in how flies' brains work," he said. In fact, he started out as a neurobiologist and was drawn to flight behavior because of the complexity of the action and the relatively small number of neurons involved.
"Brains evolved integrally related to bodies, and bodies evolved in the physical world," he said.
Part of the goal of the research is to build better robots, and insects are fascinating neurological machines. The insects themselves are not important to war or transportation. Not yet anyway. But as robotic models, they may have great significance.
In the April 18 issue of Science, Dr. Dickinson reported on the turns of fruit flies called saccades. It had been thought that the main force the flies had to cope with in changing direction so quickly was friction, the resistance of the air itself.
To capture the flies' maneuvers, Dr. Dickinson and Steven N. Fry and Rosalyn Sayaman, both in Dr. Dickinson's lab, shot digital video of the flies' turning in a space size of a die in a board game. They used three cameras to capture the three-dimensional movement, shooting 5,000 frames a second in infrared light.
The resolution was poor, about one-hundredth of a megapixel, to use a measurement familiar to anyone who has shopped for digital cameras, a field in which point-and-shoot models for amateurs often have a resolution of three megapixels or more.
The images lacked sharpness because Dr. Dickinson was pushing the speed of his system. The researchers used points on the head, abdomen and wings to mark changes in the fly's position, and they synchronized the information from the three cameras for six flight sequences to come up with 30,000 measurements that together provided a clear description of how the turn was made.
The information was "played through" a robot, not a tiny insect robot but one with wings about a meter long immersed in mineral oil to mimic the viscosity of air for something the size of a fruit fly. The translation of the data from one size and medium to another size and medium is called dynamic scaling, and it allowed the researchers not only to see how the flies moved, but also to analyze the forces at work.
They found that the fly was executing maneuvers to compensate for inertia, not friction, a far more sophisticated calculation for its neurological apparatus to handle. The finding is a challenge to roboticists, because the fly conducts its complicated movements with relatively few neurons.
What makes it possible to carry out such experiments is not the shutter speed of digital photography. Muybridge took pictures with an exposure of a thousandth of a second. Making high speed movies has been possible for years. What the scientists value about the new technology is the ability to see immediately what they have captured with the camera, to know whether they need to adjust the equipment or rerun the experiment and to have the images quickly available on a hard drive or DVD to study, manipulate and analyze.
The value of the technology is great enough that Dr. Roy E. Ritzmann at Case Western Reserve University in Cleveland compared it to the electron microscope in a news article last summer in Nature on the importance to motion research of high speed video.
Dr. George Lauder, who runs the Lauder Laboratory at Harvard, is a pioneer in high-speed video. "When I began we were using high-speed movies," he said. "And the resolution is very good. But the inability to actually see what you get as you're doing it is a huge handicap."
With the first video systems, he said, "the enormous benefit was that you could see what you had done within a second or so."
Over the last decade, the resolution has steadily improved, to the point where a $30,000 system can deliver 1.3 megapixels at 1,000 frames a second. As speed increases, resolution decreases. One reason Dr. Dickinson's images are not visually clear is that he was shooting at 5,000 frames a second.
The new tools have often been focused on insects. But Dr. Lauder and his colleagues have studied the movement of fish and have come up with some surprises, like the ways that "fish manipulate the flow of water" to swim faster or with less energy.
Trout, he said, "tune" their bodies to the turbulence in a stream by changing the timing of the body-flexing that propels them. Dr. Lauder said that others were trying to use the information for industrial purposes, to make morphing wings for aircraft, for instance, that can change shape to accommodate different speeds and wind conditions.
Much of the research is intended to help build better robots. Dr. Ritzmann and his colleagues synchronize recordings of neural activity with video of movement to connect what is occurring in the brain with what is occurring with an insect's legs. Presumably, that information will one day allow a crude mimicry of what the insect is doing.
But the insects are not simple, said Dr. Ritzmann, who has studied the way cockroaches move. He said researchers began studying insect movement in the 1800's.
"For over 100 years," he said, "we've been studying insects and we still don't know how they walk. Either insects are not simple or the people that study them are stupid."
He said he much preferred the first premise.
Dr. Dickinson said his research had led him to think of fruit flies as extraordinarily sophisticated.
"A fly has, on average 350,000 neurons," he said. "Most of them dedicated to processing sensory information."
Two-thirds to three-quarters of the brain are devoted to the eyes alone. It flies with great dexterity and precision, using only some of the remaining neurons. And yet, packed into this relatively small bunch of brain cells are all the instructions for flight maneuvers. Unlike mammals or birds, the fly doesn't have to learn anything.
"When a fly breaks out of its pupa, it can fly as well as it ever will," he said. Human beings seem to think that being constructed so that people learn complex behaviors is a superior life plan, Dr. Dickinson said. But it was an equally challenging evolutionary problem to make a brain about the size of a poppy seed, in the case of the fruit fly, "that can do everything in the behavioral repertory of a fly."
In one sense, the digital video is simply an extension and refinement of what Muybridge did. The real increase in speed has been in the time from taking the picture to seeing it. That technique may not influence the public's view of motion or speed, but it may contribute to the general acceleration of the pace of life. Stop-action was called instantaneous photography, but that was because it captured an instant, not because it could be seen instantly.
Video cameras and digital still cameras that take quick movies are now part of the standard family photographic apparatus, no doubt breeding greater and greater impatience.
On the other hand, when technology shows aspects of the world that are not apparent with human senses, it can be enormously enriching, as well. Dr. Dickinson finds himself entranced with his subjects and the window he has opened on their movement.
"I don't know if I can convey the feelings I get watching animal behavior at that level of temporal resolution," he said. "I think the world really opens up in a way. We think about, say, physics, as being stuff that takes place in black holes, and then at the atomic level. But physics is taking place all around us. People see flies flying across the room, making zigzaggy turns. How often do you think about it? When people say that flies are simple, I cringe."
Where the new technologies will lead is, of course, unknown. In fact, when asked for her thoughts on the potential unintended consequences of the technology that Dr. Dickinson and others use, Ms. Solnit relied on the now famous formula of Defense Secretary Donald H. Rumsfeld. In a news conference, Mr. Rumsfeld categorized threats yet to be discovered in the world as "known unknowns" and "unknown unknowns." The latter, in Rumsfeldian epistemology, are things "we don't know we don't know."
"The wonderful thing about a new technology like this," Ms. Solnit said, "is that it brings us into the unknown unknowns."
Copyright 2003 The New York Times Company
File Date: 06.12.03