Real Time Data Collection – What is it & Why Does it Matter?
In March, our inaugural blog identified concepts that our clients had highlighted as important features of motion capture systems. One of the most important was real time data collection. This blog expands on that feature and identifies what constitutes real time data collection, the benefits that arise from real time as well as questions that can help ensure your system will generate the benefits you are hoping for.
What is Real Time? Eddie Cramp put it well in a 1/10/2018 post on Biomch-L: “Some manufacturers will say the real-time means data within ten seconds, some have delays of less than one second, and others will provide the data with 0.01 of a second (10ms) – but they all say that their system is ’real-time’.” So when is “real time” truly real time or just fast post processing? Our view is that real time means processing data time-period by time-period, as data are being collected. In effect, the process is: read data, compute data, display data, read data, compute data, display data, etc. Of course the requirement is to do this fast enough that the display of data is virtually instantaneous and the underlying collection rate is fast enough to satisfy Nyquist requirements. More on this later.
Could I benefit from Real Time Collection? Obviously the answer depends on your situation. Here are some benefits we have heard mentioned by clients.
Wow Factor – This is probably the least important reason for considering real time. However, clients have commented on how real time lets them demonstrate their work to important constituencies including decision makers that control lab space and other resources. Interaction with subjects is also enhanced in many settings, especially clinical and athletic settings where the live skeletal animation can excite and motivate patients and athletes. And it is a great way to demonstrate the accuracy of collection systems, with photo-realistic objects and the application of basic biomechanical principles as in this demo playing darts.
Time & Money – Real Time data collection can be a source of significant savings in both time and money. These savings generally come from the elimination of rework and increases in processing speed. How many times have you collected data only to find out long after your subject is gone that dropped markers or some other anomaly made your data unusable? If you are compensating subjects it is not just the lost time, it is also the cost of recruiting additional subjects who satisfy the study criteria. With real time data collection, you know immediately if your data is acceptable. And if not, an immediate re-take is possible before instrumentation is removed from the subject or the subject has left the session.
Processing speed is also a big factor in the cost equation. By eliminating post processing, the time consumed ensuring the integrity of marker trajectories and untangling exchanges is eliminated. And this is time that can be spent analyzing data, writing conclusions or searching for funding…all of which have a higher return to you than data processing. There is a corollary to this called data quality which may be more important than the time savings. Eliminating the time spent fixing marker trajectories also eliminates the possibilities of mistakes. The benefit of better data can be immeasurable.
Of course there is no free lunch. There is an up-front, one-time cost associated with real time data collection. First there is the need for adequate camera coverage which sometimes requires more cameras than may be recommended by the vendors. Emphasis is often placed on camera resolution and measurement speed which favors fewer, higher cost cameras. However, the benefit derived from this approach is usually infinitesimal relative to other sources of error such as soft tissue movement which negates the benefit of higher resolution cameras. Real time collection places emphasis on more cameras to achieve better coverage. And in today’s low-cost camera environment, this approach can often be achieved at a lower cost than the alternative. The other cost is the setup of the collection protocol. Typical collection approaches involve the application of markers; recording of the activity; then marker identification and model definition. Real-time reverses the typical process by first defining the model; then ensuring markers or clusters can be tracked during the activity and then collection of the activity data.
The important fact is that the higher cost of real-time are one-time costs while the typical post processing approach involves an on-going cost that quickly exceed the costs of real time.
Creativity is another benefit that is sometimes mentioned. It is not unusual that data is reviewed or analyzed many months after collection. Anomalies in data that can only be guessed at months after collection, often lead to new insights and research ideas when viewed at the time of collection.
Advances in Neuro & Motor Control research have been facilitated with real time collection. Real time collection enables visual, audio and tactile feedback that is based on movement patterns such as joint angles, joint forces and moments, muscle recruitment, brain waves and point of eye focus. This capability is useful for diagnostics, rehabilitation or performance enhancement. This video is an example of measuring response times to visual stimuli that are presented in a randomized fashion.
With real time and virtual reality, it is also possible to control the environment in ways that cannot be done in the real world. For example, it is possible to introduce perturbations in perceived hand movement to observe how the subject responds. Dr. Jim Patton at the Rehabilitation Institute of Chicago has used these concepts and ErrorAugmentation as a rehabilitation aid.
The introduction of interactive control feedback loops where data on the subject’s movement affects the virtual environment while at the same time monitoring the subject’s response to the environment are all possible with real time data collection and can be used for both basic research and rehabilitation. Dr. Jim Thomas (Virginia Commonwealth University) has used these concepts to target the fear-avoidance model of low back pain using bi-directional feedback as shown in this video.
Measurement rates and software architecture are extremely important for the successful use of real time data collection. Earlier I mentioned that the display of visual feedback had to be sufficiently fast as to eliminate the appearance of latency and the underlying collection rate had to be fast enough to satisfy Nyquist. The ability of the subject to observe, process and react to feedback is one level of specification for display measurement rates. Processing speeds and monitor refresh rates can also limit the speed at which feedback can be presented. These measurement rates are “display rates”. To make real time data collection usable for research, the underlying collection rate must be much faster than the display rate. A good ground rule is to insist that the collection rate be the same as used in post processing data collections. For example, force plate data or EMG data, if normally collected at 1000hz or 2000 hz, should still be collected at 1000 or 2000hz in a realtime setting.
Another issue is the latency or the time between data actually happening and when it is available for processing and display. Each separation between the hardware device and the processing of visual data introduces latency. The only way that we have found to ensure minimal latency and to collect data at high measurement rates is to collect data directly from the underlying hardware.
Questions to ask when evaluating systems
So Eddie Cramp was right. There are lots of claims made for real time collection. To help ensure that your system will result in the benefits you are expecting there are a few questions you should ask and understand before committing to a system.
What is the system architecture? Does the software access data directly from the hardware? And at what measurement rate? Is the measurement rate consistent with the frequency content of the activity being collected?
Is the system real time or just really fast post processing? Is data collected, processed, and collected in a repeated sequence? Or just quickly processed at the end of the recording? Is the processed data available to provide real time feedback during the activity?
What is the system latency between occurrence of the data and display of feedback? Is the latency appropriate for the nature of the study? And more importantly, what is the process for determining the latency?
We’d love to hear about your applications & experiences with real-time! Feel free to use the comments section below or send us an email at [email protected].
-Mona