YaCy Decentralized Web Search

[LoudLemur]

YaCy (Yet Another Cyber) is probably the best Free seach engine available to the public at the moment.

One of its great strengths is that you do the crawling yourself. You can nominate websites which you would like to index. That way you can be sure of getting uncensored information. You can make your work available to the YaCy network to help others in their searches too.

I sincerely hope that Cloudron can provide support for YaCy. The world needs proper access to information now more than ever.

For me, this would be a priority. I hope others in the Cloudron community try YaCy (it is fun to use!) and see the value of supporting it here.

[timconsidine]

@jodumont very interesting

[robi]

Yes, this would be a great thing to provide search to a growling list of self-hosted web properties.

[timconsidine]

@jodumont : intrigued so I span it up in a container

The interface seems a little dated, but hey ho, who cares really.
And it seems a little laggy (page doesn't refresh smoothly).

But it's impressive for P2P connections ...

I will play with it a little more.

[timconsidine]

@timconsidine hmmm, played a bit

I don't think it is fully stable.

Setting a language filter (to English) mostly works, but some other language entries still creep in

Setting a date order filter mostly works, but entirely, sequence is a little random

Understandably much slower than e.g. running your own searx

I entirely get the point of it in general terms, and specifically in terms of wanting to index some particular set of sites reliably. In terms of a general search engine, e.g. to replace Searx or DDG, it's not readt yet.

I have stopped the container, but not uninstalled. I may return to it later.

[LoudLemur]

@timconsidine It is great that you tried YaCy! If you mention issues to Orbiter, the maintainer, on the YaCy forum, you should receive some help pretty quickly.
https://searchlab.eu/

Orbiter is German, so that might account for some of the language issues.

If you have a niche interest and spend some time crawling sites you know are full of content, you can see remarkable improvement in the quality of YaCy search results after only a few days. It is gratifying to be able to help the network in this way. (You can verify by searching YaCy with a different instance.)

Sometimes you try a YaCy search and notice that its results were not so good on a topic and it is great to discover that even one person can do quite a lot to remedy that with a few crawls.

I encourage everybody to try YaCy and would be very keen to hear your experience.

[timconsidine]

@loudlemur : yes, I was thinking that it might take a while to build up indexes and slow searches were relate to that.

I will play a little more later.

[robi]

From their GH

You need Java 1.8 or later to run YaCy. (No Apache, Tomcat or MySQL or anything else)

That means that one could potentially hack install YaCY into any Java 1.8 Cloudron App until a package is made.

Who's up for some hacking?

[LoudLemur]

@robi Has anybody managed to deploy YaCy using Cloudron yet?

[rggl]

I love YaCy and I'd love to see it here.

I completely understand the political/ideological desire to de-middleman internet searching. It's a worthy cause but in 2022 I don't think it's the solution to private daily driver searching.

Where it excels for me is as a really neat private productivity tool for those of us who do have a pool of sites we're constantly referring to. My YaCy indexes save me tens of hours a week and let me surface things I never would manually. I don't know how you replicate this without getting into Apache Nutch which has technical skill requirements an order of magnitude higher. It's a shame it's a bit long in the tooth.

[robi]

@timconsidine did you try this in a Surfer App ?

[timconsidine]

@robi No, I installed using normal docker on separate VPS reserved for docker.

TBH didn't like it much. Just my 2p.
Applaud the concept. Understand the use case eg for @rggl
Just not very performant for me.
Uninstalled it now.

[robi]

@timconsidine Thanks for the update.

The use case I am attracted to as there are 50+ sites with hundreds of services that need a search to find them all.

Luckily the number of domains is small enough so one could manually add them to a crawler if needed, but the user experience would need to delight to be worth implementing and changing the flow of user experience.

[LoudLemur]

@JOduMonT

Has anybody else tried getting YaCy to run on Cloudron?

It would be great to help defeat censorship, if we managed to support it.

[girish]

I headed over to check what's going on with this project. Seems quite active. For some reason, they don't make releases which is why I never got any notification.

[LoudLemur]

@girish A couple of years ago, the maintainer of YaCy stated he wasn't going to do any further unpaid work to improve the project. Some improvements have happened since then, though these are largely the result of a paid project he is supporting. I haven't looked at the websited for a while but that certainly has had some big changes of late.

UPDATE!

Big changes have happened on the YaCy project website. There is a live demo of an instance now for people to try:

https://yacy.searchlab.eu/Status.html

The YaCy Grid project has got off the ground now too. (The maintainer wanted this to be the way forward.)

If you haven't visited https://searchlab.eu recently, it is worth a look

https://searchlab.eu/en/access/download/