Firecrawl on Cloudron - Turn any site into LLM data by web scraping

[ekevu123]

[EDITED by Mod]

• Main Page: https://www.firecrawl.dev
• Git: https://github.com/mendableai/firecrawl
• Licence: GNU Affero General Public License v3.0
• Docker: Yes https://github.com/mendableai/firecrawl/blob/main/docker-compose.yaml
• Demo: https://www.firecrawl.dev/playground?url=https%3A%2F%2Fcloudron.io&mode=scrape

---

• Summary:
  Firecrawl (https://www.firecrawl.dev) is a web scraping tool that prepares data in LLM-readable format that can be self-hosted.
  Crawl and convert any website into LLM-ready markdown or structured data. Built by Mendable.ai and the Firecrawl community. Includes powerful scraping, crawling and data extraction capabilities.

• This repository is in its early development stages. We are still merging custom modules in the mono repo. It's not completely yet ready for full self-host deployment, but you can already run it locally.

---

• Notes:
  Cloudron doesn't have a self-hosted scraper yet, so maybe this could be a good addition.
  Here is the self-hosting guide: https://github.com/mendableai/firecrawl/blob/main/SELF_HOST.md

---

• Alternative to / Libhunt link: e.g.
• Screenshots:

[LoudLemur]

Wow! The ai community is so brainy! This is a great find and how wonderful it is that people created it and released it under a Free licence!

[ekevu123]

Yeah. Unfortunately, I wasn't able to re-package it for Cloudron, although I tried. So, if someone wants to do it, we all have something new to play with

[taowang]

I want this as well!

[taowang]

When can we have this? This is extremely useful to build AI applications, since it can scrape data from nearly any website. And data is valuable these days.

[igaudette]

I want this!

[LoudLemur]

Hey, @ekevu123, thank you for this brilliant app wish! I really hope this is supported on Cloudron soon.

I have heavily edited your initial post to try and use the new template that is being developed for the App Wishlist forum.

• What do you think about the new appearance of your post?
• Is it very objectionable to have somebody mod your post like this?
• Do you have any suggestions about doing this in the future?

Thanks!

[ekevu123]

Yes, I like the template, however, I think if someone else's post has been edited by a moderator, this should be very clearly marked.

[girish]

@ekevu123 The template is posted at https://forum.cloudron.io/topic/12472/please-use-this-template-to-make-an-app-wishlist-request by @LoudLemur . Looks like a good idea to have posts (in this category) formatted a certain way. For other part of the forum, generally moderators don't edit posts (only obvious typos and language).

I am hoping people don't consider it rude if moderators edit the posts in the App Requests Category alone. Besides, the original poster gets reputation (the up arrow) anyway.

[ekevu123]

As I said, I would mark it accordingly, then it should be fine. I didn't know about the template, I will try to use it next time.

[LoudLemur]

@ekevu123 The template is a new idea. I have marked your original posted as edited by mod.