Skip to main content

Government Wants a Social Media Data Mining Toolset

The U.S. government already contracts hundreds of intelligence analysts to sift overseas Twitter and Facebook posts. Their mission? Not to stay hip by learning about the latest #trend, but to sniff out foreign uprisings, possible terrorist attacks and more. Yet, that's clearly not enough manpower, and the government now wants an actual digital toolset that can scan the entire social media universe in one fell swoop, processing more data than humans can even chip at.

An Associated Press report states that the FBI, the Department of Defense and the Office of the Director of National Intelligence have issued a formal "request for information" from potential contractors. They're asking the private sector to come up with better ways of identifying scenarios like upheavals and emerging threats that lurk within the billions of posts each day.

"Social media has emerged to be the first instance of communication about a crisis, trumping traditional first responders that included police, firefighters, EMT, and journalists," the FBI wrote in its request. "Social media is rivaling 911 services in crisis response and reporting."

The FBI's proposed tool, a "secure, lightweight web application portal using mashup technology," would add to the government's already dodgy practice of spying on people via Facebook, MySpace, LinkedIn and Twitter. Two years ago an internal Justice Department document obtained via the Freedom of Information Act clearly stated that the FBI and other law enforcement agencies had undercover agents using these social mediums to browse through private information including posts, videos and photographs. The IRS also uses social media to investigate taxpayers, and just last month, the U.S. Department of Homeland Security was found to be using social media monitoring practices similar to those currently used by the FBI.

However with the proposed universal digital social website scanner, the FBI promises that it won't be used to spy on specific individuals or groups. Instead, it will monitor publicly available information and zero in on words related to criminal activity.

"The application must have the ability to rapidly assemble critical open source information and intelligence that will allow [the FBI's Strategic Information and Operations Center] to quickly vet, identity and geo-locate [potential threats to the U.S.]," the FBI states.

The Office of the Director of National Intelligence wants a tool that pools everything together including web searches, Wikipedia edits and traffic webcams. The Defense Department wants a tool that tracks social media and weed out information that could affect soldiers in the field, and to counteract enemy campaigns on social networks.

Ginger McCall, director of Electronic Privacy Information Center (EPIC) Open Government Project, recently spoke out against Homeland Security's social media monitoring, claiming that some of its activities had little to do with public safety. Now McCall is asking for a full investigation of the FBI's plans with the proposed social snooping tool.

"You need to be sure that there is no overreaching and that the program operates [legally]," she said.

Full government disclosure? That's a funny thought, right Mulder?

Kevin started taking PCs apart in the 90s when Quake was on the way and his PC lacked the required components. Since then, he’s loved all things PC-related and cool gadgets ranging from the New Nintendo 3DS to Android tablets. He is currently a contributor at Digital Trends, writing about everything from computers to how-to content on Windows and Macs to reviews of the latest laptops from HP, Dell, Lenovo, and more. 

  • drwho1
    A should be careful of what I post on this thread....

    jokes aside this could be both good and bad.

    sadly the bad might outweigh the good.
  • sirmorluk
    "the FBI promises ", "the Government promises". .......

  • greatsaltedone
    Nothing out of the ordinary here. if you dont want everyone (including a government entity), dont put it into the public sphere.
  • jisamaniac
    Government has no business snooping around. Regardless of their intentions.
  • koolkat574
  • greatsaltedone
    I meant to say "if you dont want everyone to know about your business"

    But law enforcement absolutely has business snooping around. We expect police to be able to recognize crimes in action including conspiracy.

    IMO the danger comes from how they record this information. If they are creating a relational database of everyone's interactions with each other then yeah, that goes way beyond their charge. I seriously doubt this is the case, however.
  • billybobser
    Quite happy for them to scan publicly available data (lots of facebook and twitter).

    But private data, no.

    My conversations are not normally monitored, and it should not change, just because I'm on the net.

    Not to mention, the likelyhood of the US using this beyond their stated scope is huge.
  • jellico
    You mean, they don't ALREADY have a toolset to data mine these sources? Or are they waiting for approval and then, magically, they will have a toolset developed and ready for deployment within a couple of weeks?
  • coldmast
    Type F-bomb, hit enter, now you are on government watch list.
  • kanoobie
    "processing more data than humans can even chip at."

    "The application must have the ability to rapidly assemble critical open source information and intelligence that will allow to quickly vet, identity and geo-locate ," the FBI states.

    Please don't call it Skynet...