Meet The Hidden Web

Terminology

Also known as the Dark Web, Deep Web, Darknet or darkweb, whatever is left out of search engine indexes is located in these darknet. Despite the name used, according to NPR.org, the deepweb is made out of 96% of all content, far more webpages than the World Wide Web. What we cannot see with our "naked" eye (or in this case with traditional methods) is known to be unkown, but thanks to services such as Tor or I2P, we can actually experiment the full potential of information flow.

Now, how can we know what is indexed and what is not? Well, for the most part it is very hard to know without delving yourself into the darknet itself, but some of it can be found in the "robots.txt" file of some websites. As previously stated in one of my Blogs, the robots.txt file can be easily accessed, for example, on this website.

When connecting to the deep web, you can determine it is non-indexed because the websites are randomly assigned and have a .onion extension after their domain name; for example, DuckDuckGo's search engine website is http://3g2upl4pq6kufc4m.onion.


How to Access It

Like everything on my Blog entries, I do not condone anything illegal or foolish. Use the darknet as your own discretion. You can find horrific, ugly things as you can also find beautiful lost pieces of information.

 The way to access the .onion sites is through Tor. As previously stated in my numerous Blogs, Tor was invented by the U.S. Navy in the mid-90's and it provides a pretty anonymous access to the Internet as well as I2P and .onion sites as well (darknets). Not only people who want to hide their "activities" can use Tor but also countries like Egypt, Lybia, Afghanistan, etc who don't have a complete access to the Internet and information.  Also, a lot of criminals such as paedophiles, hitmen, cyber-criminals, cyber-bullies, drug and gun dealers access the darknets as well as black markets to sell their good in an anonymous way.  These last uses are the reasons why darknets are considered dangereous.

There are easy and fast alternatives such as the Tor Browser which takes care of the tedious install and proxy configurations but it is not guaranteed that Tor will be 100% anonymous "out of the box". Further configuration is always required.

Even though Tor had some issues with bugs, security flaws and potentially NSA's surveillance and spoofed relays to spy on people, it is not wise to judge the whole Tor project because of some rotten potatoes in the past.  A lot of flaws were and are being fixed everyday as well as other software in the market. After all, Tor is also based on software and protocols which are being fixed and improved all the time. Also, it is not safe to consider Tor as a bullet-proof for all your "hidden" activities, but it is a good choice as an extra security layer to have in your security arsenal.

Where to Look

Once you connect to the Tor Network, you can find who are in the network acting as relays.  Each relay are the people helping your connection being more secure. The entry and exit nodes (you and the server) are the only who knows about the site you are trying to visit (not the relays) and they also think your connection comes from another country.  Also, it is important to note that the only unencrypted part is from the exit relay to the destination.

All of this is only to understand a little how Tor works, but let's get to how to surf the deep web.

For starters, let's first find a starting point. To find a starting point, we need either 1) an .onion site with a list of other sites or 2) a search engine for deep web sites.  The reason why the first one is not very reliable is because the list is always out of date and the links might not work. The original one is called CoreOnion.

1) There are sites that lists, or at least, tries to list the most up to date links. Some of them are: The Hidden Wiki, Tordir, and the Onion URL Repository (You will need to be running Tor to enter to these sites).


2) You can also look at search deep web engines for .onion sites. Some of them are: DuckDuckGo, DeepSearch, and Abyss. (You will need to be running Tor to enter to these sites).

Once you have a starting point, you can surf on your own discretion. You will find a lot of information just about anything, and I mean ANYTHING.

Deepweb and Censorship

One of the reasons governments cannot shut down the deepweb entirely is because governments also use it to hide their activities and make them more anonymous in order to avoid infiltration, eavesdropping and data leak. As anything in this world: one tool which is used for the good of humanity can and will be always be used as all things evil. Even though there are tons and tons of criminal activities in the deepweb and lots of them are being shut down such as "The Silk Road v1 and v2", it is impossible to shut them all at once without bringing down the Tor network.

The Repercussion: In Numbers

Since the military, governments, navy, airforce and secret societies also use the I2P and Tor networks to hide their daily "secret" activities, it would be a total loss for all of them if they shut it down. Governments always try to keep control of the darknet by shutting down criminal content most popular sites but they re-open soon after with a new random .onion address, or better yet, a mirror somewhere else. The repercussion, however was not a lot comparing with the profit. For example, The Silk Road v2 had approximately 1 million members and was making 1.2 billion in yearly profit. When the Silk Road was shut down by the F.B.I. on November 5, 2014, they sized about 26,000 bitcoins (equivalent to 4 million U.S. dollars at that time). Bitcoins is the anonymous form of purchasing services and goods in black-markets through the darknet. Used as BTC in the stock exchange and now (by April 6, 2015) it is worth 258.19 U.S. Dollars. Suprisignly, Silk Road's operator made $80 million in commissions from its members. When the silk road re-opened its value went 3 times what it was worth both in members and financially.

Net neutrality and Last Thoughts

After the EFF winning over net neutrality on March 12, 2015, ISPs and Cable companies don't have access to a lot of their client's control but since they lost the battle (but not the war) they are finding new ways to supress their clients' browsing actions which is not included in the Net Neutrality rules. For example, Comcast is currently performing DPI (deep packet inspection) techniques to ensure they alert governments (if asked) if a customer is using Tor.  Since, they deeply analyze their customers' packets, they can determine who is using Tor and who isn't. One easy (but not bulletproof) way to avoid this is to use Tor Bridges. Since using Tor relays which are indexed from the Tor network, if Comcast (or other ISP company) has access to this list, they can easily determine who is using Tor thus blocking access to it so the customer cannot access any site through it.  By using bridges, they cannot determine if their customer is using Tor because the bridge address is not listed as "public" in the Tor network, thus they cannot discern between a Tor or non-tor connection. They just don't know what it is. Bridges are being used in highly-oppressing countries, countries such as China, Hong Kong, Lybia, Egypt, Labanon, Syria, etc. to bypass their government Firewall.  They also use Proxy Chains which intercorrelate their connections and bounce it through a series of proxies to anonymize traffic even further.

Additionally, you can use a VPN with Tor and Bridges to ensure more layers of anonymity, since by solely using Tor does NOT guarantee 100% anonimity.

With the emerge of a new, faster (even more controlled) Internet and free Internet such as Kim Dot Com's MegaNet, it is hard to conclude that Tor will be long enough to live our end of times.  Perhaps, a less centralized, non-IP address based network will be used for Freedom Fighters as the rest of the civilization will be using a faster but more controlled (and censored) Internet like the emerge of the Internet v2, which is already in progress and perhaps will be using HTTPS/2 (founded by Google).  It would be soon be a matter of speed and reliability vs privacy. The decision, hopefully, will be ours to make.


Sources:

Exploring Onionland: The tor .onion Darknet
DarKnet or DeepNet: What is it and how to access it?
Going Dark: The Internet Behind The Internet
Deep Web Links
How FBI brought down cyber-underworld site Silk Road
BTC in Dollars - Current Stock Price
EFF wins over Net Neutrality