Firecrawl on Cloudron - Turn any site into LLM data by web scraping
-
wrote on Jul 22, 2024, 5:37 AM last edited by LoudLemur Sep 12, 2024, 9:44 PM
[EDITED by Mod]
- Main Page: https://www.firecrawl.dev
- Git: https://github.com/mendableai/firecrawl
- Licence: GNU Affero General Public License v3.0
- Docker: Yes https://github.com/mendableai/firecrawl/blob/main/docker-compose.yaml
- Demo: https://www.firecrawl.dev/playground?url=https%3A%2F%2Fcloudron.io&mode=scrape
-
Summary:
Firecrawl (https://www.firecrawl.dev) is a web scraping tool that prepares data in LLM-readable format that can be self-hosted.
Crawl and convert any website into LLM-ready markdown or structured data. Built by Mendable.ai and the Firecrawl community. Includes powerful scraping, crawling and data extraction capabilities. -
This repository is in its early development stages. We are still merging custom modules in the mono repo. It's not completely yet ready for full self-host deployment, but you can already run it locally.
- Notes:
Cloudron doesn't have a self-hosted scraper yet, so maybe this could be a good addition.
Here is the self-hosting guide: https://github.com/mendableai/firecrawl/blob/main/SELF_HOST.md
- Alternative to / Libhunt link: e.g.
- Screenshots:
-
wrote on Jul 22, 2024, 7:16 AM last edited by
Wow! The ai community is so brainy! This is a great find and how wonderful it is that people created it and released it under a Free licence!
-
wrote on Jul 22, 2024, 4:47 PM last edited by
Yeah. Unfortunately, I wasn't able to re-package it for Cloudron, although I tried. So, if someone wants to do it, we all have something new to play with
-
-
wrote on Aug 26, 2024, 11:31 AM last edited by
I want this as well!
-
wrote on Aug 28, 2024, 5:36 AM last edited by
When can we have this? This is extremely useful to build AI applications, since it can scrape data from nearly any website. And data is valuable these days.
-
-
wrote on Sep 11, 2024, 6:58 PM last edited by
I want this!
-
wrote on Sep 12, 2024, 9:24 AM last edited by
Hey, @ekevu123, thank you for this brilliant app wish! I really hope this is supported on Cloudron soon.
I have heavily edited your initial post to try and use the new template that is being developed for the App Wishlist forum.
- What do you think about the new appearance of your post?
- Is it very objectionable to have somebody mod your post like this?
- Do you have any suggestions about doing this in the future?
Thanks!
-
wrote on Sep 12, 2024, 7:15 PM last edited by
Yes, I like the template, however, I think if someone else's post has been edited by a moderator, this should be very clearly marked.
-
Staffwrote on Sep 12, 2024, 7:47 PM last edited by joseph Sep 12, 2024, 7:48 PM
@ekevu123 The template is posted at https://forum.cloudron.io/topic/12472/please-use-this-template-to-make-an-app-wishlist-request by @LoudLemur . Looks like a good idea to have posts (in this category) formatted a certain way. For other part of the forum, generally moderators don't edit posts (only obvious typos and language).
I am hoping people don't consider it rude if moderators edit the posts in the App Requests Category alone. Besides, the original poster gets reputation (the up arrow) anyway.
-
wrote on Sep 14, 2024, 3:39 PM last edited by
As I said, I would mark it accordingly, then it should be fine. I didn't know about the template, I will try to use it next time.
-
As I said, I would mark it accordingly, then it should be fine. I didn't know about the template, I will try to use it next time.