Help & Feedback wanted: broken link checker needs (stress-)testing

[msbt]

Hello Cloudronauts!

I’ve been building a frontend for my broken link checker over the past months as a side project and I’m getting close to releasing it to the public. The main goal was to keep it easy, fast and straightforward, while still covering the things that tend to matter in practice (broken links, redirects, some server info and basic SEO checks).

Before actually publishing it somewhere, I thought I'd ask the Cloudron community for a little help, run it against real-world setups and see where it breaks or gives misleading results (also, I ran out of ideas which sites to check). If anyone here is interested in trying it on their own sites and sharing feedback with me (either publicly or dm), that would be much appreciated.

I’m especially interested in:

• edge cases (odd redirects, timeouts, unusual setups)
• anything confusing or missing in the reports
• cases where results don’t look correct
• general feedback/bug-reports or inconsistencies
• last minute feature requests

I’ve set up a 60-day coupon (code: WE-LOVE-CLOUDRON, redeemable until the end of April) "starter plan" for this, so you can use it without any barriers (other than your email-address), after the 60 days it will fall back to a free account. If something looks off, you can send me a share link or just describe what you’re seeing.

Here's the URL: https://www.4f.at/

This is not meant as a promotion, just trying to make the tool more solid before launch. Also posting it here especially since this is a 100% Cloudron powered affair (except the runners for the build pipeline):

• custom Next.js app with PostgreSQL and Redis (sessions, rate-limiter, caching)
• Keycloak for OIDC authentication (sends out magic links for login)
• crawlers are custom Go-apps that can be easily deployed on multiple servers to scale horizontally
• source-code resides in a private Gitea instance, the build pipeline includes the docker registry app, Cloudron build service and a custom Drone-app & runners
• Umami for statistics

Thanks for the attention!

[luckow]

@msbt text color white on background white in a form, isn't the best idea.

[msbt]

@luckow thanks for that, I never checked light mode on the keycloak template Should be fixed!

[girish]

We used this tool to fix the various broken links in our docs repo - https://git.cloudron.io/docs/docs/-/commit/37fa62cd22285a1b12172fa6fca6294f235fd04e

[james]

Hello @msbt
I will look into your tool and use it a little to give feedback.

One thought that instantly crossed my mind, will there be a Cloudron Community App for this?
I can see that your aim is also to generate revenue with this, so I can understand it not being open-source.
Next thought, what if the Cloudron Community App is like a 'runner' to the global infrastructure of 4f.at?
In the sense of, if someone would use the Cloudron Community App for 4f.at, they also agree to share a certain percentage of their server resources to the global infrastructure.
This way Cloudron users can host your software themselves with full use for their needs and provide resources to the pool saving you and them money.
Something like a decentralized open infrastructure?
Implementation of that might be a big task tho.

[msbt]

@james awesome, if you run out of pages and need a bump, let me know

I thought about open-sourcing the thing from the start, but after a bit of testing I realized how many websites are tunneling through services like Cloudflare - which would instantly block traffic from bots like this one.

It was quite an ordeal to get a "signed agent" approved by CF, but I managed to get that done eventually (wrong category, but hey, it's in there ). This means every request is signed with a special signature deriving from a host key in /.well-known/...

So you would either have a lot of self-hosted bots that can't do a lot or you would swamp Cloudflares bot directory - both not ideal, that's why I opted to keep it closed and offered a generous free tier instead. Revenue is not the priority, I wanted to get it out so people can have fun with it.

The runners idea is interesting though, currently they all share the same internal API key for auth between each other, that would probably require some fiddling around. Lets iron out all the kinks first and see where we go from there