Russian invasion highlights growing importance of open source intelligence

Yuichiro Chino/Getty Images

The Russian invasion of Ukraine demonstrates the practice’s real-time value.

During a morning political opinion show on March 2, Marc Polymeropoulos, a retired CIA field agent and supervisor, said he is sure his former employer is scouring its intelligence sources for signs of cracks in the walls of oligarchs and military supporters of Russian President Vladimir Putin as the invasion of Ukraine grinds on.

One hopes that the analysts are checking Twitter, TikTok, Telegram, and the broader social media environment.

“Ukraine Forces Reportedly Kill Russia General Andrei Sukhovetsky in Blow to Invading Army,” Newsweek reported March 3, citing a message posted to VKontakte, a Russian social media and social networking service.

Open source intelligence, or OSINT, is defined by the Center for Strategic and International Studies as “intelligence collected from publicly available or available-for-purchase information, obtained for addressing a specific intelligence requirement and processed to derive new insights.”

In January—weeks before Putin launched the invasion—CSIS released a report on the potential of OSINT, when combined with artificial intelligence and machine learning, to provide “fast, attributable, relevant reporting from anywhere, including austere field environments and [analysts’] homes.”

“High-level [intelligence community] policymakers have publicly stated the necessity of embracing open-source intelligence as a core analytic discipline,” the report stated. “[T]he intelligence community should stop ‘recreating the internet in a classified environment’ … and instead accept a small amount of risk to run applications on an unclassified cloud, taking advantage of increasingly sophisticated and automated cloud security and obfuscation capabilities.”

Anyone paying attention to the Russian invasion of Ukraine is getting a crash course in the value of OSINT, said Kari Bingen, chief strategy officer of HawkEye 360 and former principal deputy under secretary of defense for intelligence, the second-highest ranking civilian intelligence job in the Pentagon.

“We are seeing the unprecedented release of intelligence, sharing of information and transparency that is shining a light on Russian disinformation efforts and creating no doubt about Russia's aggressive actions,” she said. “The urgency of the moment demands it. The question going forward is, how do we better institutionalize this greater sharing of information and greater use of open-source intelligence to benefit our national security and that of our allies and partners?”

But the IC has been slow to incorporate OSINT into its intelligence products, for a number of reasons, and is lagging behind other countries—particularly China—in tapping into the vast oceans of publicly available information, the report noted. “While China has an estimated 100,000 science and technology intelligence officials, who do most of their work on open sources, the United States has an estimated 100.”

The author of the report, Emily Harding, deputy director and senior fellow of CSIS’ International Security Program, said the IC’s risk equation is part of the obstacle to committing to using OSINT.

“The IC lives in risk all the time, so they minimize it wherever they can,” Harding said. “That turns into restrictions and regulations.” She said the belief within the IC has been that if you don’t need to ask questions in the open, don’t, because it lets hostile parties know what they are interested in.

"Part of [the challenge] is culture, risk taking, the process of acquisition, but a lot of it just comes down to trust—building trust with OSINT providers, understanding the heritage of the information and having confidence in the data and analysis from those providers. And that takes stronger user-provider relationships, working together to better understand the problems and iterate on potential solutions," said Bingen.

There is indeed a big cultural component to the IC’s resistance to embracing OSINT, a combination of the “not invented here” syndrome, an ingrained belief in secrecy because field agents are out risking their lives, and budget fears. “Often, adding funding [to OSINT projects] would require removing funding from existing, proven systems—anathema to a government bureaucracy,” the report observed. “For contract officers, in particular, flexibility and risk-taking are not part of the job description … Contracts that finish under budget and on time, with deliverables that match exactly the specifications in the initial contract, count as successes for contracting officers.”

Harding suggested that the IC should keep the constellation of OSINT capabilities “on the low side”—that is, outside of classified networks—and accept the small amount of risk, because the upside is gaining real-time intelligence and insights.

“What OSINT traditionally meant was translating foreign documents,” said Bingen. “Open source was [considered] secondary” to the information gathered by clandestine sources and methods. “When you look at what’s become open source [today], the different types, it’s a far cry from just translating foreign articles. It’s not just your news articles or social media, it’s financial, it’s satellite information, the ability to bring it all together [and] apply advanced compute, AI and ML.”

CSIS’ Harding said the IC risks losing credibility with elected officials if it doesn’t move quickly to embed OSINT into its intelligence products. “In a best-case scenario, the IC will lose policymaker attention and trust as they compete with private intelligence,” she wrote in the report. 

This is particularly true as news media make greater efforts to capitalize on open source information; elected officials expect their classified briefings to be at least as well informed as the latest articles out of Ukraine, for instance.

“So much of the intelligence world is being commercialized,” Harding said when asked to elaborate on that observation. “You could commercialize analysis … You could definitely see Stratfor or other firms out there [get into this.] So many of these companies [started by] former U.S. government officials or military are still committed to the mission.”

Nor is the generation of OSINT limited to commercial activity. For instance, Bellingcat is an “international collective of researchers, investigators and citizen journalists” that uses social media and open source information to investigate a wide range of activities, from climate change to the spread of COVID-19 to the Ukraine invasion, to name just a few. “Using publicly available information, they have discovered illegal shipping of chemical weapons precursors, identified a high-ranking Russian intelligence officer as a key suspect in the shooting down of Malaysian Airlines Flight 17 and identified Russian intelligence officers as suspects in the poisoning of Sergei and Yulia Skripal,” according to the CSIS report.

“I come from a world where you buy billion-dollar satellites that launch every five to 10 years,” Bingen said. “These innovators are now building satellites that are launching every three to six months. It’s just such a different paradigm when you’re constantly evolving capabilities.”

There is a convergence of three technology areas happening right now that impels the OSINT discussion, she said. “First is the data, the variety of data that's out there now, open and unclassified ... Second is the advanced analytic and machine learning tools that allow us to bring those datasets together and draw out insights on activities, patterns of behavior, etc. Third is the cloud, which allows you to bring all the data together, and apply the analytics, and deliver a product to users across the globe."

There are ethical concerns to be addressed. Harding said the IC should not “vacuum up” all the available open source information, which is China’s approach. Included in her report’s recommendations is the need for “a clear framework of ethics and governance principles to guide how technology is applied to U.S. intelligence.”

"Open source intelligence is valuable to the Intelligence Community; it's one of the IC's core collection disciplines,” an ODNI spokesperson said.

An NSA spokesperson referred Nextgov to the Office of the Director of National Intelligence and the Defense Intelligence Agency on questions of OSINT as an intelligence discipline.

“On open source more generally, NSA will use open source information to support authorized mission activities. For intelligence activities, in addition to other applicable law and policy, NSA follows the procedures contained in Department of Defense Manual 5240.01 which, among other things, governs the collection, use, retention, and dissemination of open source information for Defense intelligence activities. These procedures are designed to protect the civil liberties and privacy of U.S. persons,” the spokesperson said.

In the meantime, every person following the Russian invasion of Ukraine is seeing the value of OSINT, as well as the risk—Google Maps has turned off its live traffic data in Ukraine for the safety of the people there.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.