Neural nets find niche

Artificial neural networks are close to being deployed on Navy ships as part of a firedetection system, based on work that the Naval Research Laboratory oversees.

A warfighter has no problem distinguishing a tank against the foliage of a tree. The turret, the treads, the squat hull — they’re signs that trigger split-second recognition.

It’s a simple task for the human brain, with 100 billion neurons firing away. But it’s no easy feat for a computer, despite the Defense Department’s once heavily funded efforts to duplicate the processing power of human brains using artificial neural networks (ANNs). ANNs are meant to excel in recognizing the relationships among complex variables. The financial industry has embraced them wholeheartedly as fraud-detection systems, and they’re enjoying a renaissance in the medical field.

A few years ago, there were legions of DOD-funded ANN projects. “In the 1990s, neural networks was a hot word,” said Leonid Perlovsky, principal research physicist at the Air Force Research Laboratory. “You would say ‘neural networks,’ and you would get funding.”

The funded projects included an eight-year effort to put ANNs in M1A1 Abrams tanks as engine diagnostic tools. Officials also considered using them as automated target-recognition tools on board the canceled Comanche helicopter.

ANNs are close to being deployed on Navy ships as part of a firedetection system, based on work that the Naval Research Laboratory oversees. The British Royal Navy has deployed a similar system on its ships, said George Privalov, founder and chief technology officer of axonX. That company is working with the Naval Research Laboratory on the multisensor fire-recognition system, which uses neural networks embedded in video cameras.

Despite such promising applications, ANNs have fallen out of favor at defense agencies. Proponents such as Privalov and Dennis Braunreiter, chief scientist of sensor system operations at Science Applications International Corp., say recent developments show that it’s time for a new, albeit more critical, look at military uses for the technology.

Observers say the Defense Advanced Research Projects Agency in particular gave too much money to neural networks in the past decade and expected too many returns in a short period.

“I think the big-bang approach was not appropriate,” said Harold Szu, a program officer at the Office of Naval Research. A DARPA spokesperson said the agency could not provide anyone to talk about neural networks and doesn’t have ongoing ANN projects.

Although work on ANNs never disappeared, the military now seems to have an aversion to such systems. Perlovsky said he still conducts research on ANNs but is careful to avoid using the term during funding discussions because some people don’t want to hear about them.

Skeptics simply point to ANN failures to keep a lid on financial support.

Almost everyone in the neural-networking field has heard a story about military researchers attempting to use an ANN to detect tanks amid foliage. The story might be apocryphal, but it goes something like this: Scientists fed pictures into a neural network of trees with and without tanks parked beneath them. At first, they had stunning success — the machine had a 100 percent detection rate. But when they tried reproducing the results with new data, the ANN failed.

The computer hadn’t learned to detect tanks at all. Instead, it had focused on the color of the sky to determine whether tanks were present because the test photos had been taken on different days. In the pictures with the tanks, the sky was cloudy; in the pictures without tanks, the sky was bright blue. The network had learned to recognize the difference in the weather.

INSIDE THE BLACK BOX

Scientists argue about the effectiveness of ANNs, but their adoption — or lack thereof — could be attributed to a perceived flaw that might be more cultural than technological. It stems from the nature of ANNs.

Such networks mimic the human brain’s activities in solving complex problems by breaking them down into component pieces for parallel processing. The human brain operates when one neuron builds up enough energy to excite another neuron with electrical and chemical signals. So digital nodes — the ANN equivalent of neurons — must collect enough energy before the data inside is sent forward for more processing. Nodes mimic the buildup of neuron energy by assigning different numerical weights to incoming data.

“Each node is specializing, each node is learning a piece of the problem,” said Steven Templeton, a senior research scientist at Promia. The cybersecurity company developed an intrusion-detection tool used by the Space and Naval Warfare Systems Command that incorporates a neural network.

When scientists know the correct answer, they can train a network with continuous feedback until it adjusts to reach that conclusion. Such so-called back propagation networks are the most mature ANNs, but there are also unsupervised networks for which researchers don’t know the correct answer.

Nevertheless, ANNs have a reputation for being black boxes — mysterious things that transform data in unknown ways. “What you get is just numbers that are connecting the nodes in the neural networks, so there is not an easy explanation for why did you get this result or that result,” said Zvi Boger, an Israeli scientist who worked with the National Institute of Standards and Technology on a research project to use remote sensors to detect toxic airborne chemicals.

Although neural network engineers might disagree, concerns about ANNs’ lack of transparency are genuine.

MORE ART THAN SCIENCE

After Boger returned to Israel, the NIST project abandoned neural networks in favor of traditional statistical methods at the behest of Boger’s replacement, Barani Raman, a postdoctoral researcher whose work was funded through a National Research Council grant. His concern about neural networks stems from their tendency to produce unpredictable results: Researchers can’t guarantee a particular output with the same set of input data.

Raman said ANNs have a tendency to get stuck in local minima — network weights that might be valid locally but not globally as a problem solution. Neural networks attach to the first solution they find, but that set of internal weights might not be the best.

“Once it gets into this particular set of weights…it is very difficult to get it out,” Raman said. Even diehard fans say training an ANN can be tricky. It’s more art than science, Privalov said. “It’s not that a neural network by itself is a panacea that lets you just put data in, train [the network] and voilà, everything works,” he added.

“If you don’t know what you’re doing, you can do a lot of wrong things,” said Lars Kangas, a scientist at the Pacific Northwest National Laboratory who developed a neural network to detect fuel flow problems in Abrams tanks. He still touts the benefits of ANNs. His network was able to piece together data from 30 sensors to monitor engines’ health in real time.

Much of the sensors’ data had nonlinear relationships that are difficult to map, which is why Kangas turned to ANNs — not despite their black-box reputation but because of it. “The neural networks can learn that mapping for you,” he said.

He produced three generations of operational models, but the project was canceled in 2002. “There was a change in command and other things,” he said. “They just decided that the funding could be [used] somewhere else.”

THE FUTURE

Researchers say successfully training a neural network requires large amounts of data. A lack of data is one reason why ANNs failed to live up to expectations in the past, Braunreiter said. But a recent explosion of available data — made possible by the Internet — has accompanied an exponential leap in computing power that makes processing that data possible, he added. DOD stopped funding SAIC’s research into neural networks, so the company now finances its own projects.

Another criticism of ANNs is that they are not adaptable to changing situations. For example, if a neural network is meant to detect anomalies, it can struggle to adjust to a dynamic baseline, Raman said. “What is normal and what is abnormal has changed a little bit,” which means the ANN was trained on a set of data that’s no longer valid.

ANN proponents say they’re overcoming that limitation. Neural networks that can dynamically shift their internal weights are replacing networks that fix their weights once and stick to them forever, Braunreiter said. And rather than yield just one weighted output, nodes will create multiple weights.

“That gives you more dimensions to separate the objects at a finer level of detail because I’ve now added more weights,” he said. “That means I can create more dimensionality.”

“There are probably not a lot of ANNs in military systems today,” he added. “I think that’s going to change over the next five to 10 years.”

However, scientists want to avoid another round of 1990s-like excitement. “Things will never live up to the hype,” Perlovsky said.

NEXT STORY: Return to sender

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.