Ryan Singel
Security

DHS Data Mining Program Suspended After Evading Privacy Review, Audit Finds

A controversial Homeland Security data mining system called ADVISE that dreamed of searching through trillions of records culled from government, public and private databases analyzed personal information without the required privacy oversight, may cost more than commercially available alternatives and has been suspended until a privacy review has been completed, according to an internal audit. The […]

advise graph-tbn

A controversial Homeland Security data mining system called ADVISE that dreamed of searching through trillions of records culled from government, public and private databases analyzed personal information without the required privacy oversight, may cost more than commercially available alternatives and has been suspended until a privacy review has been completed, according to an internal audit.

The Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement program, one of twelve DHS data mining efforts, hit the trifecta of civil libertarians concerns about data mining programs – invasiveness, secrecy and ineffectiveness, according to a recent DHS Inspector General report (.pdf).

DHS hoped the data sifting tool would help analysts "detect, deter, and mitigate threats to our homeland and disseminate timely information to its homeland security partners and the American public." The idea was to build a generic toolset that could find hidden relationships in massive amounts of data and provide the tool to groups working with data sets as divergent as intelligence and newspaper reports to WMD sensor data.

Started in 2003, the program has gotten $42 million in funding through 2007.

But the data-mining program faces a troubled future, due to revelations that its tests did not simply use fake data as the DHS Science and Technology section publicly said they did.

"The pilots used live data, including personally identifiable information, from multiple sources in attempts to identify potential terrorist activity," the report said.

For its part, the DHS Privacy Office did not know that S&T had proceeded with implementation of the ADVISE pilot programs with live data, but without addressing privacy matters. In a July 6, 2006, report to the Congress, the Privacy Office stated that the ADVISE tool alone does not perform data mining. […] Unbeknownst to the Privacy Office, the ADVISE pilots had been implemented at least 18 months prior to its July 2006 report.

ADVISE is now shut down until after privacy reviews are completed.

The Science and Technology Directorate hoped its system would tap into 50 DHS databases and 100 other data sources. A DHS Workshop paper said the system would be engineered to handle 1 billion structured pieces of data and one million unstructured text messages per hour. The Inspector General found however that access to data was never lined up and that commercially available products like i2's Analyst Notebook were cheaper and more effective for small data sets.

For example, officials of Customs and Border Protection’s Office of Strategic Trade initially expressed interest in ADVISE as a potential solution to their requirement to view and process millions of pieces of trade data. Customs and Border Protection officials received a demonstration of ADVISE capabilities and attended a training session at Livermore. However, after learning the cost of ADVISE, these officials told S&T that the system was too expensive and therefore would not be a good option for them.

The Total Information Awareness program, a similar research effort started by the Darpa, the Pentagon's high-tech research arm, was largely shuttered by Congress in 2003.

But the dream of database diving to identify would-be terrorists using super-smart algorithms lives on.

See Also: