Index the World Wide Web

A single snowflake is small, fragile, hardly visible. But taken together, snowflakes can take over entire landscapes, from remote valleys to inaccessible mountain peaks. So it is that many of us, working together, can do what only powerful organisations have achieved so far: explore, search and share the World Wide Web.


Most of us take the Web for granted. We know that it's there, that it's very big, and that we just have to open our favourite search engine to get to the information we need. But is it really so? Let's consider things point by point.

The Web is there True. Let's remind ourselves, though, that it is only there because we made it. That's right. All of us, together, wrote the billions of pages that make the browsable content of the Internet. Without us, no World Wide Web.

The Web is big That's difficult to say. It depends on what you count as valid content. You will find all sorts of figures out there, from 45 billion to 60 trillion pages. But there is one important factor: we individually only use a tiny proportion of the Web. My personal Web history for 2017 has around 20,000 unique URLs in it. Double or triple that if you are a heavy browser, the numbers remain insignificant. In fact, your yearly Web consumption would fit on your laptop (60,000 pages at an average of 3.6 MB per page are about 211 GB).

The Web is searchable The Web is only searchable because your favourite search engine lets you search (and they don't let you search everything). Imagine that for any reason, you got locked out of Google and their competitors. It would be like being a librarian with the most gigantic collection of books ever written by humankind, and no single library record, no way to even know what books there are, and what they are about.

So we collectively made the largest-ever repository of human knowledge, but we individually only have very restricted access to our creation. None of us has a map of the Web, none of us has any control over search, none of us can be sure to find what they need tomorrow. But the Web, our Web, is not that big. Can we take it into our hands?

A tool to share the Web

Phase 1 of PeARS, codename 'Orchard', is a decentralised Web search system allowing people to easily index their favourite web pages (i.e. produce a computer-readable, searchable representation of the pages' content). Its main feature is the ability to convert a small index into a greyish, unassuming picture, called a 'snow pod'. Snow pods are the mini-weapons of the indexing revolution. They can easily be shared with others by email, on social media, or through any other mean, so you can be your very own search engine, for yourself and for your friends.

For those out there who want to reach a wider audience with their pod, and who have some more advanced computer skills, we have made code available to transform pods into proper websites, directly indexable from any PeARS install. Example of official PeARS pods are Pod Zero and the Opera Pod, as pictured below.

A single snowflake is small. Many snowflakes can break mountains. We wrote the Web, single page by single page. For sure, we can read it too, single page by single page. 200 of those tiny snow pods contain someone's Web consumption for a year (and some of their friends' -- we love sharing pages we love).

We will snow

When 'it' snows, nothing is 'it'. Snow comes and goes, outside of our control. But what if 'we' snowed, in a concerted way?

Let's snow those pods. Let's post them on websites, on social media, on our cloud space, let's send them by email, let's physically hand them over on hard drives. Let's cover the landscape and remind ourselves that it is our Web.

The hashtag is #wesnow.