Sensor fusion algorithms can give a more precise 3D orientation (and possibly postion?) of a device by combining readings from an accelerometer, gyroscope, and magnetometer.
Can anyone explain, or provide links, explaining the details behind this? I want to understand the math and physics, such that if I have raw 9-DOF sensor data, I might be able to implement sensor fusion. Assume I have sufficient background of linear algebra, calculus, etc.
Cheers!
Sensors provide analog or digital insights into a reality that is difficult to grasp. Just like your senses do.
Sensors are designed to seize a special portion of a measurable phenomenon: electrical, chemical, physical... Just like your senses do. The usual five ones are sight, hearing, touch, smell and taste, but there are more to human skills.
Understanding a physical phenomenon via signal processing is like trying to tell the main ingredients from the recipe of the meal you ordered in a restaurant last evening. You do not know what happened in the kitchen, yet you have chosen what to order, and have seen, tasted, smelled, even touched the plate.
Each of your senses and understanding provided you with hints or knowledge, but you can never be sure, as some cooks master sense deception. What you see as an artifial green marmelade made of strawberries can taste like apple jelly.
Based on you cooking expertise, the combination of all your senses can bring you closer to the the actual recipe, provided that:
Sensor fusion is the art and science of combining sensory data, knowledge and models from disparate sources so that the resulting information has more validity or less uncertainty than individual sources.
Examples: The act of averaging $N$ sensor data for the same deterministic signal, with different stochastic realizations of a noise term, is the most basic sensor fusion operation. With the model of independent Gaussian noises of deviation $\sigma$, the average theoretically yields a $\frac{\sigma}{\sqrt{N}}$ deviation(less uncertain). The combination of three channels (Red, Blue, Green) offers a color image (more valid than a monochannel gray-scale).
The first example combines information from a single sensor model. The second one from sensors operating in the same domain (electromagnetic waves), yet in different portions of the spectrum. In general, fusion operates on different sensors with different rates, ranges, domains and mostly units.
The problem is highly dependent on the phenomenon you are looking at, the available sensors, and the information you are looking for.
The physics tell you the potential information you can get from your sensors. The mathematics can model how they are related or complementary, or what is irrelevant information (noise). The algorithms will combine the previous knowledge as optimally as possible, in terms of precision, accuracy or speed.
The topic is related to the realms of Sensor fusion, Data fusion or Information integration, with a short overview in Principles and Techniques for Sensor Data Fusion. Many more books are available, as in Best book for learning sensor fusion, specifically regarding IMU and GPS integration.
Regarding your actual problem, a first step would consist in understanding What are the differences between a gyroscope, accelerometer and magnetometer? which could help you push techniques a little further. And achieve the goal of fusion: using sensor differences, sum their data, in the most clever way.
Sensor fusion for 3D orientation is all about joining multiple sources of data (sensors) to extract more accurate information.
More specifically in the case of IMUs, you can join many measurements (technically DoM, and not DoF) to get orientation and position data (this is the technically DoF).
Suppose you have a system with accelerometer, gyrometer, magnetometer, barometer and GPS (usual high-end cellphone capabilities, and some hardware boards features). Here is what you actually have:
You have acceleration measurement, but if you are moving, you can't separate what is the gravity component of the acceleration and the actual movement acceleration (unless you are not really moving and only the gravity is left). You can assume your movement varies more rapidly than the gravity change and apply a filter to separate high-frequency components from the low-frequency ones, but you still won't get good results where the assumption doesn't apply (such as a long turn). You also can't simply subtract the gravity from where you think it is, simply because that is part of the orientation result, not an input. Anyway, even if you knew where gravity is pointing, you will only know where down is, but you won't know where you are around (your angle in the plane perpendicular to) the gravity vector.
You have a girometer that registers turn ratio. It is not a gyroscope. A gyroscope would actually result in an attitude information, whereas you have to integrate the gyrometer output to obtain that information. The problem with this kind of sensor is that, first, you don't have a starting orientation (unless you assume one, and assumptions have problems as above); and second and more important, you have bias and noise (from sensor, numerical, etc), which messes up any integrator.
You have a magnetometer. The magnetic field of the Earth is very weak and thus subject to a lot of ambient noise. Think any RF emitters such as WiFi, cell phone, FM radio, walkie-talkies and even unlikelier ones such as HAM radios, airplane radios, pirate radios and TVs/monitors. If you think your flat-panel energy-saving TV doesn't mess with a compass, think again. So, it is noisy as hell. Magnet-and-oil analog compasses fix this by immersing the needle in a viscous fluid which acts like a mechanical low-pass filter, and because of this it isn't fast enough and even so you still will get wrong readings if you're near a power line.
You have a barometer. You can estimate height from this information, but it is really just an estimate. As it is, beware of atmospheric effects (if there's wind, there's pressure difference somewhere) and you have to keep in mind that your data is relative to some reference, but not necessarily ground. Remember that usually the reference is ground at sea level and hey, sea levels change all the time with tides. So, it gives some insight into a single axis of position, and is usually only good for detecting variations.
You have GPS. However, GPS signal is obtained from satellites in space. Do you really expected to get millimeter position info from that? You can even get that if holding still for many minutes or even hours, by filtering, but one-meter accuracy is pretty standard for most civillian GPS receivers, and enough for most commercial applications. Anyway, even if you didn't have position noise, lack of signal in closed spaces or bad weather, you still would get only position information, nothing like attitude.
With all that in mind, sensor fusion itself is the combining of all the data you get to try to get more accuracy. Comparing what movement is common to different sensors with different characteristics is key. But as it is, there are many ways to do it and each implementation usually differs from the other in implementation. Some ideas:
Filter stuff. Low-pass and high-pass filter may improve the data (by canceling some noise) and may separate what is one information into two.
Kalman filtering (in various forms) is quite common to clear some noise and join multiple sources, because it is computationally fast and can also be used as predictor/corrector and to compensate data delay differences between sensors.
Use the knowledge of the problem to enhance comparison equations. Use current orientation plus future estimates to predict gravity; integrate the acceleration to obtain velocity; integrate again to get position, use GPS to correct it, filter and derivate back to get estimates; the GPS also has some velocity info from doppler effects, so use that; use GPS dilution of position to estimate how accurate is your GPS data; integrate the gyrometer to correct the next gravity vector; low-pass the magnetometer and use as a weak north reference, with magnetic declination corrected from GPS position; use barometer info from start to get approximate ground level, and derivate the pressure data to enhance Z position information. Create your own method.
Filter it all, but don't filter too much or you won't get useful information.
Even if "Sensor Fusion" is sold as a done method, it is almost an open ended problem. Even using an established methods will still require quite a lot of tuning to get the best results, to exploit different sensor characteristics to get better data.
Roger R Labbe Jr explains this in his fantastic book "Kalman and Bayesian Filters in Python" as
You can not throw out any information even if how noisy it is.
Two Gaussians are always better than one in Bayesian framework. If you multiply two Gaussians you will have smaller covariance matrix.
