Two ideas on decentralized search engine
Several months ago, I wrote a search engine called Horizon, actually it consists of a centralized one and a decentralized one.
That decentralized Horizon is just a ui and the database which contains the data I crawled from zeroent, but for there’s no enough space to store all the content, I used AI to derive keywords from the content.
I also added a feature searching things in downloaded sites, this is what other search engines of ZeroNet do.
Finally the engine is done, but I’m not satisfied with that. So I continue to build a centralized search engine with Elasticsearch.
Since it’s expensive to run a elasticsearch engine, I stopped making any search engine.
Recently, I came up with IPZN, the ZeroNet runs over IPFS.
Search engines are still necessary for dWeb, but building a decentralized search engine is not easy.
I have been discovering about blockchain, in IPZN, it can be used to:
- Pay for some service
- Deposit token (coin) and prevent spam
- DAO, voting system
Yeah, combined with blockchain, but optional
With the stuff above, we can have two types of decentralized search engine：
- Tagging system, a site for tagging everything on IPZN, or even the whole IPFS.
- Searching service market, you pay for a search
For 2, we can create a token/coin, which is supplied with advertising, so users don’t need to directly pay for it.
When a user searchs, the request will be broadcasted to all available searching nodes. To prevent DDOS, user should deposit some tokens.
User can upvote for search results, and the result are mixed with the results from multiple nodes, ranking by the reputation(user upvote) of each node. Refer to existing project Steemit
So, searching can be either treated as market or steemit
Still confused how to implement it, seems not robust
Searching is different from storing file or what.
Our goal is to incentive better quality of search results, but the quality is determined by users, real human beings.
It causes a problem, how to deal with spam and bots.
For bots, we need a decentralized verifying system.
And limit maximum operations per hour a user can perform.
Decentralized verifying system
DAO and the verifiers in proof of stake mechanism give the requester a challenge (e.g. computer generated captcha), and the verifiers vote to determine accept him or not.
Tagging system can be fullly decentralized by using peer reputation and a better sorting algorithm.
To be more decentralized, we will use DPoS (Delegated Proof of Stake).
The new coin can be called SearchCoin, or other better name. It’s just like FileCoin, which incentivizes peers to store files