Iron Man Month 2010

Marvel Science: Iron Man's User Interface (UI)

Find out some real world explanations for how Tony controls his suit!

Share:

Comments:

 

We're celebrating the May 7 (April 28 internationally) release of "Iron Man 2" with Iron Man Month 2010! Get ready for a monthlong mega mix of all things Shellhead right here on Marvel.com!

 

By Ryan Haupt

Marvel has a proud history of science-heroes, with many Marvel heroes emerging as accidents of science or the product of their own scientific ingenuity. Tony Stark is one such hero and in order to analyze his plausibility we brought in an honest to goodness scientist to figure it out for us

Ryan Haupt holds two Bachelor's of Science in Environmental Geology and Ecology & Evolutionary Biology from the University of California, Santa Cruz and is going back to school in the fall to get a Masters in Paleontology from Vanderbilt University. Currently, he helps research a variety of topics ranging from stable isotope geochemistry, mammalian paleoecology and oceanographic paleoclimatology. He hosts the podcast "Science... sort of" (http://www.sciencesortof.com/) with two grad student friends where they hang out while talking about science and geek culture. He occasionally fights rabid and rogue elephant seals, but only for science.

 

THE UI (USER INTERFACE)

Before we begin there’s an experiment you should consider doing. It’s pretty simple. Unplug your keyboard, now unplug your mouse. Done? Good. Now close this browser window. Kinda tricky, right? What’s that? You were on your phone and it’s a touch screen? Oh, well now we just look foolish, but hopefully we’ve also proved a point. All of our incredible technology is great, but it’s only as great as your ability to control and interact with it. Without these input links between us and our stuff it’s all nothing more than power-sucking paperweights. Well the Iron Man armor is a bit more complex than your average laptop, so Tony needs more than a keyboard and a mouse to get stuff done. This is exactly the issue we’re tackling today as we explore, IRON MAN: The UI (user interface).

This article is a bit different from the previous two in that we’ve established that for our purposes the Iron Man suit exists and functions as it should. So within those rules the way it’s operated actually falls to some pretty realistic and available technology. Some more available than others, so we’ve decided to organize this from available to not-so-much. It’ll build suspense and you might just learn something!

 
Subtle Movements

If you had to scroll down to read this all it took was a flick of your index finger. Tony doesn’t have a mouse or keyboard but he still has his fingers and they’re still inside a suit capable of receiving commands. More than just his fingers his entire body is capable of inputting commands depending on how the suit has been programmed. As mentioned last week, there are likely times, such as during high powered flight, where the suit itself takes over certain portions of Tony’s body to make sure they’re in the correct position. Outside of those constricted circumstances, the freedom of movement allowed by the suit is basically the same as any other human would have. It’s one of the main benefits the suit has over a plane in the first place.

Well if you spend enough time at the keyboard you may have the thing memorized and don’t even really need the keys in front of you to know where to put your fingers to type what you want. If you had access to one of these (http://www.virtual-laser-keyboard.com/demo.asp) that skill could come in very handy. Jarvis could easily project a virtual keyboard in Tony’s helmet that he could then use his fingers to manipulate just as if he was using a real keyboard. This is a bit inelegant for something so complex, it barely works for video games, but we know Tony uses something like Bluetooth to make phone calls from within the suit so a keyboard may have some utility. Anyone with an e-mail enabled smart phone will attest to the convenience of e-mail on the fly, and Tony does have a corporation to run. Also, as writer Matt Fraction pointed out in an early issue of INVINCIBLE IRON MAN, there is no manual for the armor and no tech support. Tony programmed it initially and he may need to make adjustments to the code literally on the fly. A keyboard would allow just that.

A similar system could even fill the function of a mouse. Move the hand, which is in the suit so its motion is detected, and the mouse moves. Click the index finger, the mouse left clicks. You get the idea. The keyboard and mouse combination is pretty useful, but it is far from sexy in terms of futuristic technology. So let’s see if we can’t up the ante a bit with…

Voice Control

It’s clear when watching "Iron Man" that Tony’s voice is modified when speaking through the suit. It makes sense because he did have a secret identity to that point, plus the suit is thick and makes noises of its own so I’m not sure if Tony could even shout loud enough to be heard through his helmet if he wanted to. But that’s actually a nice thing; it allows Tony to use his voice without anyone knowing it. This would be good for something more routine than what would be needed with the keyboard and mouse. Like testing the control surfaces, flight paths of other flying objects, and calling a friend on the phone. These things probably happen regularly enough and don’t lend themselves to any specific motion to activate that it’s probably as simplest to just say it out loud to Jarvis.
 
While not quite a ubiquitous as the keyboard and mouse, voice recognition technology is readily available and becoming even more advanced with each cycle of Moore’s law (http://en.wikipedia.org/wiki/Moore's_law). It used to be that a person had to really train the software to recognize their specific voice and speech patterns. Nowadays a phone can recognize even complex names like Wisecup, Olsson and Beil without any prior training with one’s voice or how those names should be pronounced. It's not perfect, but the progress it’s made in such a short amount of time is nothing short of astonishing (Marvel really does have the best adjectives).

Well Tony has a huge advantage over you and your phone; and that advantage is Jarvis (the super-computer, not the human butler, though a butler would be quite an advantage too).  Obviously we don’t know much about Jarvis’ inner-workings, but we do see what he’s capable of doing and it’s a lot. It’s safe to say Jarvis (no doubt an acronym, but let's call him Just Another Really Very Incredible Supercomputer) is a from scratch AI (artificial intelligent) made for Tony by Tony. Jarvis runs Tony’s house, it presumably is the force controlling the holographic display in Tony’s shop that he manipulates with his hands, and it can upload itself without difficulty into a recently built, completely experimental suit of armor. Think of Jarvis like the R2-D2 in the X-Wing, not running the show, but providing much needed back up. Additionally, Tony has those two other more corporeal bots in his lab as well, and they seem to at least rudimentarily understand his commands. So Jarvis isn’t even a prototype, making it that much more effective.

Eyeballs

Humans are extremely visual animals. Much of how we interact with the world is based on what we see, and while operating the Iron Man suit Tony is under the same biological restrictions. So why not turn that limitation into a strength? As soon as Tony puts on his helmet the audience experiences just how that is done. Information is flooding Tony’s field of view within the suit, or the heads up display (HUD). Every object he looks at immediately has information brought up and displayed, so clearly Jarvis is tracking Tony’s eye movement and doing what it can to provide environmental data. It’s unlikely this is much more than a convenience and less of a control mechanism, but is an integral part of how Tony interacts with the suit and thereby the world around him.

Grand Gestures

Tony in the suit is far from subtle and quiet. He’s a man of action and even though he’s highly controlled by the suit itself while flying, fighting is an arena where he needs to be the one calling the shots, fast and with great accuracy. While Tony loves his projectiles, he’s not without his brawler side while in the suit. This kind of control is the easiest to explain. Since the suit is in sync with his actual body all it needs is to sense his movements and provide a 1:1 response and augmentation. The computer in Tony’s suit must be fast enough to sense motions as he’s doing them (if not, there’s a really cool way around that, but you’ll have to keep reading) and since the suit is much much much stronger than any human it can give Tony a boost so that he can punch through walls, toss guys around like something that’s lighter than a guy you’d like to toss around, or hold onto a plane while it’s flying.

As for actions that have no human analog, like firing wrist mounted missiles, there needs to be a preprogrammed “grand gesture” that the suit understands and responds to. For example: Tony always spreads his fingers apart whenever he fires the repulsor from his palm. This is a good idea to avoid losing any much needed fingers, but we suspect it’s also the initiation of the firing itself. Think of it the same way as Spider-Man’s webs. The webs only fire with that one specific hand motion, so when Peter makes a fist he’s not spraying gunk all over the place. Well when Tony locks his elbow and spreads his fingers, the suit knows he wants to fire the repulsors, probably uses his eyes and arm to figure out where he wants to aim, and makes it happen. This would all have to happen extremely quickly, but that’s why computers are so wonderful. The example here is repulsors but this is probably how a majority of the offensive capabilities are controlled. It may seem a bit unnatural and cumbersome but there are many martial arts that seem unnatural and cumbersome until your body learns the movements and can incorporate it to make everything a fluid form. This may be another reason Tony, who could easily build more suits, keeps them among friends. It’s not as simple as just putting it on and going for a stroll, there’s a rhythm to working it that someone not familiar with could really get hurt trying to do.

The Futurey Stuff

As is tradition at this point, this article couldn’t possibly be complete without showing you something that has hugely cool potential future applications but barely works in the modern world. Just enough to whet the appetite of someone interested in future tech, and leave the door wide open for rampant speculation. Well one of the things scientists are working hard on that Tony could really use is a direct mind-machine interface. The two top contenders for the most functional of these are rat neurons and Italians.

We’ll just say this and give you a moment to process it before moving on: rat neurons can fly planes! (http://www.sciencedaily.com/releases/2004/10/041022104658.htm) Back? Good. Here’s the skinny. These researchers took rat neurons and put them on top of a bunch of electrodes. The electrodes were connected to an F-22 flight simulator. The electrodes fed the neurons information from the simulator and the neurons connected to each other in such a way that it created a neural network that can control the aircraft. So it’s not even taught how, it just learns from the raw data input. This may not be very applicable to Tony and his suit but it does demonstrate the relative ease with which nerve cells will accept novel input and adapt to function with it.

The second and more Italian prospect is Pierpaolo Petruzziello, who had to have his left arm amputated after a car crash. Fortunately, science has come a long way since the days of pirates and now he has a prosthetic arm he can control with his mind even when it isn’t attached to him! (http://sify.com/news/experts-man-controlled-robotic-hand-with-thoughts-news-health-jmdaEzgijdh.html) What’s even more amazing is that he can feel feedback from the hand too. The experiment only lasted a month, electrodes had to be attached directly to the brain and that leads to other complications. Like the rat neurons, it takes significant training to get good as using this type of interface, but the younger generation all learned to type from an early age, in the future mind-machine interface may be equally commonplace. Tony installing something similar, presumably “wireless” to avoid the holes in his skull, would allow him an unprecedented level of control over the suit in every circumstance.

Well that we’ve figured out how he tells the suit to fire the weapons, we should figure out just what the heck some of those weapons are. Get ready for a fight next week…

Acknowledgements:

This article would not have been possible without the help of some spectacular science-types who have specialized in fields I ran screaming from my freshman year of college. I'd like to thank Ben Tippett (University of New Brunswick), Daniel Oliphant (University of Pittsburgh) and Jacob Stump (Embry-Riddle Aeronautical University). I couldn't have done this without their knowledge and willingness to converse endlessly about the awesomeness that is the Marvel Universe. Thank you very much. To anyone I've forgotten to mention, thank you too.

**Get discounted seasons of your favorite Marvel shows on iTunes.
For a limited time at iTunes.com/MarvelComics.**

 

Visit the official "Iron Man 2" movie site now and the Marvel.com "Iron Man 2" movie hub! Also, be sure to check out StarkExpo2010.com, the official site of Stark Expo!

Find out when the movie opens in YOUR country with the official "Iron Man 2" international release schedule!

The official Marvel Shop features all things Iron Man!

Read Iron Man Digital Comics right now! Not a subscriber to Marvel Digital Comics Unlimited yet? Join now!

Check out the official Marvel Shop for your favorite Marvel Heroes!

Required Reading: check out key Iron Man collections available now in bookstores everywhere!

Related Characters
Tags

MORE IN Iron Man Month 2010 See All

MORE IN Marvel Science See All

MORE IN Comics See All

Comments

6 comments
sanchezas
sanchezas

this article freaks me the **** out

cephalon
cephalon

I sincerely hope that this type of article, which blends the futuristic ideas of the comicbook world together with actual trends and developments in the scientific world, will be continued in future Marvel News articles. This article was incredible and the suggested ancillary articles from various scientific magazines and journals were equally unbelievable yet truly complimentary to the basic explanation of the technology built into Ironman's helmet controls. Thank you for taking the time to provide an enlightening, enjoyable article that merged science fiction with reality. Please keep it up.

Elijah1066
Elijah1066

I think the eye control has been overlooked, and I think it would be possible to control certain elements of the suit with the eyes. For example, in the fight scene in the first film, he uses his shoulder cannons to take out the terrorists holding the hostages, and he targets them simply by looking at them. He could have brought up this subroutine by looking at a section of the interface not within his normal range of vision. This kind of interface is used by people who are almost completely immobilized to communicate, such as Stephen Hawking. A camera tracks the movement of the eye and calculates which part of the interface the eye is looking at.

johnnysword
johnnysword

[i] "It’s not as simple as just putting it on and going for a stroll, there’s a rhythm to working it that someone not familiar with could really get hurt trying to do."[/i] Except for Pepper Potts becoming Rescue in about 5 seconds.