Two ideas on decentralized search engine

Several months ago, I wrote a search engine called Horizon, actually it consists of a centralized one and a decentralized one.

That decentralized Horizon is just a ui and the database which contains the data I crawled from zeroent, but for there’s no enough space to store all the content, I used AI to derive keywords from the content.

I also added a feature searching things in downloaded sites, this is what other search engines of ZeroNet do.

Finally the engine is done, but I’m not satisfied with that. So I continue to build a centralized search engine with Elasticsearch.

Since it’s expensive to run a elasticsearch engine, I stopped making any search engine.

Recently, I came up with IPZN, the ZeroNet runs over IPFS.

Search engines are still necessary for dWeb, but building a decentralized search engine is not easy.

I have been discovering about blockchain, in IPZN, it can be used to:

  • Pay for some service
  • Deposit token (coin) and prevent spam
  • DAO, voting system

Yeah, combined with blockchain, but optional

With the stuff above, we can have two types of decentralized search engine:

  1. Tagging system, a site for tagging everything on IPZN, or even the whole IPFS.
  2. Searching service market, you pay for a search

For 2, we can create a token/coin, which is supplied with advertising, so users don’t need to directly pay for it.

When a user searchs, the request will be broadcasted to all available searching nodes. To prevent DDOS, user should deposit some tokens.

User can upvote for search results, and the result are mixed with the results from multiple nodes, ranking by the reputation(user upvote) of each node. Refer to existing project Steemit

So, searching can be either treated as market or steemit

Still confused how to implement it, seems not robust

Edit: 8/23

Searching is different from storing file or what.

Our goal is to incentive better quality of search results, but the quality is determined by users, real human beings.

It causes a problem, how to deal with spam and bots.

For bots, we need a decentralized verifying system.

And limit maximum operations per hour a user can perform.

Decentralized verifying system

DAO and the verifiers in proof of stake mechanism give the requester a challenge (e.g. computer generated captcha), and the verifiers vote to determine accept him or not.

Edit: 8/24

Tagging system can be fullly decentralized by using peer reputation and a better sorting algorithm.

Edit: 8/25

To be more decentralized, we will use DPoS (Delegated Proof of Stake).

The new coin can be called SearchCoin, or other better name. It’s just like FileCoin, which incentivizes peers to store files

From IPFS Developers Group: IPSE is shitcoin

There’s a new stuff called IPSE for IPFS, but I think it’s just a new shitcoin, so buzzwordy.

IPSE Team, [22.08.19 19:01]
[In reply to ethersir]
Hello,it might be confusing,but what IPSE is doing right now is to make the search for IPFS files faster and more convenient. You can try it on ipse.io. If you have any questions pls let me konw.

blur HY, [22.08.19 19:53]
Hi everyone, do you know that IPSE is shit, a buzzwordy shit

Repeating IPSE is shit, LOL

ethersir, [23.08.19 04:26]
[In reply to IPSE Team]
I have my questions how is filecoin being used

ethersir, [23.08.19 04:26]
You provide no explanations other than “file consensus”

ethersir, [23.08.19 04:28]
There’s a plethora of IPFS search engines that don’t require blockchains. IPSE tries to say it’s decentralized, but the security of the IPSE data is actually dependent on the trustworthiness of the 21 EOS validators to not collude and alter the data (there’s already been cases of collusion)

Markus Lücking, [23.08.19 04:28]
[In reply to ethersir]
True

ethersir, [23.08.19 04:29]
All IPSE marketing is filled with the latest industry buzzwords, and terrible explanations as to how they’re used

ethersir, [23.08.19 04:30]
It’s literally impossible to be using filecoin consensus right now, unless you’re building your product relying on the security of consensus provided by an experimental Blockchain that’s on a testnet which is extremely bad.

That doesn’t even count for the fact that you literally can’t use filecoin consensus for anything meaningful about a search engine.

ethersir, [23.08.19 04:54]
I fully understand your platform and have actually tried it.

How does IPSE make searching more convenient? To use your search engine I need to:

  • Download scatter desktop
  • Praise to the cryptogods that I’m not going to get smited because I’m using EOS
  • Sign onto your website
  • Search for data

Theres easily at least 3 other IPFS search engines that I don’t need to sign up an account for, download third-party software, and rely on a Blockchain (EOS is not even a Blockchain, transaction database isn’t cryptographically bte).

Also you have a token incentive model for indexing which seems to be the only focus for this platform to push a coin.

False claims of retrieving data faster, and claims of making data ownership clear and other search engine don’t protect user privacy ?

Yet you are based on EOS one of the most shady and centralized platforms in the space.

Also I would love you to dive into how your encryption design efficiently and quickly retrieves data as you claim because I cannot seem to find any benchmark test.

All I have seem is an less effective way to search IPFS compared to many other platforms and pushing tokens on clients to index data and claiming it is somehow safer then every other IPFS search engine (without any evidence).

Marketing a product is fine however at least back up your claims if you are going to call other project in the space unsafe.

ethersir, [23.08.19 04:55]
[ Album ]
Tons of misinformation everywhere


Markus Lücking, [23.08.19 05:10]
[In reply to ethersir]
Any Others opiniones

blur HY, [23.08.19 09:14]
yeah, it is really shit

blur HY, [23.08.19 09:18]
shitcoin


IPSE whitepaper says IPFS will replace tcp/ip. Absolutely ridiculous.