Enabling battlefield big data ‘on the move’
- By Greg Gardner
- Mar 13, 2014
From Defense Department commanders down to troops on the battlefield, the most challenging aspect of big data today is that it is “big.“ Data has swept into every military function and is now a critical component of most every mission, particularly in a restricted budget climate where the military must extract more valuable information with less manpower.
It is estimated that the U.S. military has at least 2 to 5 terabytes of stored data for every soldier, and the increased use of unmanned vehicles with data-collecting sensors has compounded the data challenge. Add in mobile devices that soldiers use to communicate from the battlefield, and a growing data surge has become an avalanche. This wasn’t the case as recently as a couple of years ago, when military devices capable of acquiring data outpaced military networks’ ability to store the data.
The challenge of data volume
There are a number of ways in which that data creates huge value: First, big data makes information transparent and much more rapidly usable. Second, big data enables more precisely tailored and targeted missions, activities, and services. Third, sophisticated analytics substantially improve decision-making. Finally, big data improves products and services. For instance, data obtained from sensors embedded in military aircraft enable innovative service capabilities such as proactive maintenance that can prevent battlefield equipment failures before they occur – often with catastrophic outcomes.
Unfortunately, there are major challenges processing and using data in tactical environments. Digesting and drawing conclusions from the massive amount of intelligence and information available to military commanders and their staffs is increasingly difficult. Since 9/11, the amount of data from drones and other surveillance technologies has increased a staggering 1,600 percent. Our armed services now have approximately 7 million computing devices – a number expected to double by 2020. That proliferation of sensors has helped fuel the big data glut. It’s all relevant data. The problem is simply that there’s too much of it.
The challenge of data analysis
Getting a handle on big data increasingly requires striking a balance between growing military requirements and shrinking budgets. Impending reductions in military personnel and program accounts include limitations on intelligence analysis and production at all operational levels. At issue are the variety, veracity, velocity and volume of big data. Military organizations simply don’t have enough analysts to take advantage of all the information that’s there. As Lt. Gen. Michael Oates, former head of the Joint Improvised Explosive Device Organization, summarized: “There is no shortage of data. There is a dearth of analysis.”
The challenge of data mobility
To date, soldiers operating ground and aerial military vehicles have been restricted by the ability of products and technology to enable data processing “on the move.” The nature of data storage devices requires that military vehicles come to a rest in order to send and receive critical data – a step that can delay the delivery of critical data and greatly hinder the success of military operations.
Data mobility also challenges data security. As data flows across the battlefield, through mobile devices, across land, sea and air domains and in the cyberspace of data centers and headquarters offices, security challenges emerge. Data must be secured not only across levels of security and access, but also at rest and in motion.
Battlefield data storage evolves
The challenges around battlefield big data are being address by various DOD agencies. One key approach is to effectively multiply the number of intelligence analysts by extending solid-state data storage and processing devices down to the tactical vehicle level. This initiative enables leaders at the company level and below — officers and NCOs who traditionally only consumed the results of the intelligence analyzed at higher headquarters — to ingest, process, share and act directly on information and intelligence while on the move and in contact. Harvesting, analyzing and rapidly converting large data sets into actionable intelligence is a critical capability for ground forces. This initiative significantly speeds information processing and on the battlefield in what is often a matter of life or death.
Moreover, the data processing capabilities of this initiative significantly enhance mission execution. Whether related to intelligence, operations or logistics, the data from vehicles enabled with solid-state storage drives (SSD) is automatically replicated to tactical operations centers for both additional analysis and continuity of operations. SSDs – utilizing flash memory drives rather than hard-disk drives – are optimally suited for mobile, tactical environments where reliability, scalability and non-disruptive operations are paramount, and are enabling the move from data processing at rest to battlefield data on the move.
Additionally, technology advances enable the packaging of robust data processing capabilities and up to 20 terabytes of solid-state storage into a device no larger than a shoebox. It is cost-effective as well. Advances in commercial data storage and processing technologies make this approach much less expensive than more cumbersome and less effective methods employed in Iraq and Afghanistan, and even in some recent Network Integration Evaluation events.
Whether data is structured or unstructured, generated by a UAV, or reported by a cavalry scout, it must be processed in some fashion before it can inform military decision-makers. Advances in storage and processing capabilities are enabling the Army and other branches of the military to handle the volume of big data being produced, and also shift from data at rest to data on the move – in turn creating big data processing capabilities at the tactical vehicle level that will significantly enhance operational capabilities and save lives.
Greg Gardner is chief architect, Defense and Intelligence Solutions, NetApp US Public Sector.