Hey there !
I created this project for two reasons, the first is I feel like we are recently being flooded by AI generated news articles, blogs, websites and even youtube videos. This is reaching an unbearable flow, I wish to find something else. Small indie websites are amazing, and, maybe more important today, they are genuine. There is so many good indie websites and articles out there, and they are barely reachable through traditional enshittificated search engines. I wanted to have a search engine that would really help me digging into these sites.
The second reason is me trying to figure out if it's possible to make an effective yet minimal search engine. I did the smallest amount of research first before diving in, hence the name "worst search engine but in rust". Despite expecting it to fail miserably, it actually ended up working pretty fine (at least if you don't try to break it :) ).
Here are some stats about this wsebr instance :
- 923820 web pages indexed
- 1497914 tokens indexed
- 41146619 bm25 values computed
Note that this project is beyond beta, there is a lot of chance that there is some lags, that some features are missing or even some bugs... However, it should be quite safe and js-free
Amazing projects that inspired me, you should check them :
- Wander is a console to browse the small web.
- Kagi small web, amazing initiative from kagi, most of the webpage come from their github
- Tiny brutalism css is an amazing minimalist CSS framework, wsebr frontend is highly inspired of this.