Canadian Federal Police To Develop Deep Web Crawler

Posted by: Benjamin Vitáris

August 5, 2015

The Canadian Police is investing in the development of a specific software that will search and monitor the darknet for the most hidden content and services (which is, of course, highly illegal).

The funding for the brand new deep web crawler is just a part of the Canadian government’s program. The funding source for the software is the Canadian Safety and Security Program (CSSP). The CSSP is providing funds to several projects that are in the category of public safety, policing and other military related stuff. This security research program of Canada has been successful in several projects. For example, they have developed a device that can detect drugs from their vapor, explosive detection equipment. The program has also provided funds for the training that will help the Canadian government to protect their energy infrastructure from possible cyber attacks.

We have identified the funding source for the project, however, what we have not stated is which institution will be responsible for running the research for the dark web crawler. The government has authorized the Royal Canadian Mounted Police (RCMP) and two tech firms to research the software. The Canadian government stated: “The Royal Canadian Mounted Police will lead a project to develop and implement a web-crawler to explore anonymous and dark regions of the internet and identify content of interest to national security and law enforcement communities.”

The companies, Mercur IT Solutions and Donnybrook Research and Analysis, that are involved in the research process are two firms that are specialized in technology products and are located in British Columbia. They will work in conjunction with the Canadian federal police (RCMP) on the project.

The co-CEO of Mercur IT Solutions, Richard Frank, has worked with the Canadian government on another project before. In 2014, Frank has developed a software, named Child Exploitation Network Extractor, that could tag and track images that have been posted on child pornography websites on the deep net.

Frank has stated the software, which they are developing alongside with the RCMP and Donnybrook Research and Analysis, will focus on the most extreme cases on the darknet. The program’s main goal is to track and trace things that darknet websites have in common. After that is done the software will start to build up patterns, thus allowing it to differentiate trends.

The research team is planning to find common keywords, images and themes from several deepnet websites. This will allow them to create a sitemap (just as it is in the case of clearnet websites) that will help the police and other cyber criminal divisions in the world to even monitor the most hidden darknet websites.

Richard Frank says that the automated dark web crawler is not interested in the actual websites and the possible illegal content that could be found there. “We, as the researchers, we don’t really care about the individual who’s behind it. We’re not trying to catch an individual. We’re not saying he lives at 45 Main Street.”

Knowing the usual LE intentions, the RCMP will most probably use the software for stopping or limiting the spread of darknet marketplaces, child pornography, and other websites that are offering either human trafficking or hitmen services. The crawler software could be also used by law enforcement authorities to find hidden hacker communities and websites that are offering hacking services.

However, the only question about this brand new crawler, is that how this software will affect the privacy of deepnet users? David Fraser, a noted privacy expert has been asked about this issue. He stated that the police is free to research all sections of the internet.

Fraser also states: “The complicated thing is what do they do when they get there and what do they do with what they find. If police are monitoring and tracking individual suspects online and using it to build a criminal case, they may be required to obtain a warrant. Police may also find themselves in trouble if they surveil entire subsections of the internet in the hopes of catching a handful of individuals.” He added that in the latter case „the impact on the privacy of Canadians and others is going to be affected in a way that is likely disproportionate to the law enforcement benefit.”