3 Node Deployment (Unified Dashboard)
-
Hey everyone,
Most cases people use a public cloud provider but I would like to have a discussion on hardware implementations , specifically multi node (3+)
I have been using the Lenovo ThinkCentre m715q AMD Ryzen 5 Pro 2400ge with 16+ GB RAM ....
They are great and usually more than enough to get the job done but if needed they actually scale really well...I will further explain but first if you would like to know more about the hardware itself the ServeTheHomeVideo Youtube has recently reviewed these types of units:
Apparently they are populate in the r/homelab community if you scroll through the pictures of setups they are very common
Regarding multi-node implementations and their viability using the mentioned type of hardware there is a production instance with 6,000 nodes: 🤯
- https://medium.com/@cfatechblog/bare-metal-k8s-clustering-at-chick-fil-a-scale-7b0607bd3541
- https://medium.com/@cfatechblog/edge-computing-at-chick-fil-a-7d67242675e2
Chick-fil-a built their own tooling but the concept is validatedSo much so there are "hyper converged hardware/software" providers pursuing this avenue: ScaleComputing - https://www.scalecomputing.com/he150
This brings me full circle to the point: 3+ Node Cloudron Deployment (6.0 - Unified Dashboard)
My thoughts/questions:
1 - Is anybody else "personally" interested in a deployment like this (hardware + cloudron)
2 - From a Service Provider prospective - Does this incentivize "edge" deployments? Or even co-location over public cloud?Love to hear everyone's thoughts and takes on the topic & really looking forward to the feature =]
+1 -
Very interesting!
I'm lucky enough to live in Jersey, where now every home has Fibre Gigabit internet with 1ms ping times to Paris & London, and I've thought a fair amount about the potential for that.
Somewhat along the lines of ownership and jurisdiction over data property and intellectual property and somewhat on federation of data to make it somewhat of a sandman beyond the risks of any location, however safe or trusted it might be today.
One the subject of data-storage alone, you can recoup some hardware and home-hosting costs with these federated storage networks, that could make a reasonable candidate for Cloudron backups storage:
I'm sure I looked at others but can't find the backups. Those communities though may have more chat on cost-efficient, reliable and energy-efficient hardware, and there's a subset of people also aiming for their home-nets to be entirely solar or other renewables powered.
-
Very interesting!
I'm lucky enough to live in Jersey, where now every home has Fibre Gigabit internet with 1ms ping times to Paris & London, and I've thought a fair amount about the potential for that.
Somewhat along the lines of ownership and jurisdiction over data property and intellectual property and somewhat on federation of data to make it somewhat of a sandman beyond the risks of any location, however safe or trusted it might be today.
One the subject of data-storage alone, you can recoup some hardware and home-hosting costs with these federated storage networks, that could make a reasonable candidate for Cloudron backups storage:
I'm sure I looked at others but can't find the backups. Those communities though may have more chat on cost-efficient, reliable and energy-efficient hardware, and there's a subset of people also aiming for their home-nets to be entirely solar or other renewables powered.
@marcusquinn Assuming you were between "New Jersey" and UK with the / but after the digital.je link I realized that's not the case lol especially with your 1ms ping to Paris and London lol!
Had to look into it and I must say Beautiful place:
(for those like my self who are unaware)Totally see your use case of wanting to have the ability to have ownership of data while worrying about location ie backup/replication given you're on an island.
I am generally familiar with Storj they are nice. They use to have a hard requirement on bandwidth capacity to be a host but that was been lifted..Also the previous Docker CEO is their current CEO or from last i saw. Looking into testing a node....Another player is https://edge.network - they have S3 like storage on the roadmap along with Compute (docker based) DNS, CDN and so on...more of a full fledged platform compared to storj but both are cool and applicable
-
Nice. We get the same speeds upstream too. Every home has fibre to the router, so there's no copper last-mile.
Don't know why the UK didn't do that, probably cost but they will have to eventually. 5G can't match the ping times so can't see that as making much difference to the average browsing experience.
I know New Jersey and the US well too from work travels, explain the Jersey thing a lot when over that side
This is a good video, we have about 30 beach cafes and bars dotted around the coast - and yes, a geographically, digitally and fiscally it's natural fortress - this one of my favourite spots:
(1m 17s Ad for Jersey)
Digital industry there is getting a lot of investment, and I try to champion the open-source cause and promote that by example.
Binance recently relocated to Jersey, I'm sure there will be others.
-
And this is the beach I was surfing in my profile pic
-
Nice. We get the same speeds upstream too. Every home has fibre to the router, so there's no copper last-mile.
Don't know why the UK didn't do that, probably cost but they will have to eventually. 5G can't match the ping times so can't see that as making much difference to the average browsing experience.
I know New Jersey and the US well too from work travels, explain the Jersey thing a lot when over that side
This is a good video, we have about 30 beach cafes and bars dotted around the coast - and yes, a geographically, digitally and fiscally it's natural fortress - this one of my favourite spots:
(1m 17s Ad for Jersey)
Digital industry there is getting a lot of investment, and I try to champion the open-source cause and promote that by example.
Binance recently relocated to Jersey, I'm sure there will be others.
@marcusquinn very interesting and beautiful place & fiber to the prem.. + growth is very appealing to. Might have to relocate one day lol
Thats cool about Binance more players like that the better for the local scene
Also that's good you are advocating for open source in the area +1 -
https://www.reddit.com/r/homelab/comments/i6t2wy/finally_comfortable_enough_with_my_cabinet_to/
Yes it is a half rack with 2 storage arrays and a desktop (on its side - lol) with a mighty fine network behind it but the focus would be on the 5 NUC's (running Docker + K8)
also that lil IP cam keeping an eye on the rack
🦾 lol +1
EDIT* not my setup =/ lol✌💙+1
-
https://www.reddit.com/r/homelab/comments/i6t2wy/finally_comfortable_enough_with_my_cabinet_to/
Yes it is a half rack with 2 storage arrays and a desktop (on its side - lol) with a mighty fine network behind it but the focus would be on the 5 NUC's (running Docker + K8)
also that lil IP cam keeping an eye on the rack
🦾 lol +1
EDIT* not my setup =/ lol@plusone-nick Nice cabling!
-
@plusone-nick Nice! I too use a thinkcenter for all my data. I think mine is an older model - M600. It's incredible how well it works. Since it's fanless, makes no sound whatsoever.
My home setup is outright amateurish compared to yours
There's 2 disks: one is an external disk for all my pictures/videos etc and the other one is for backups. My home has pretty stable internet/power, so I haven't had a need for UPS etc.
My email server is on DigitalOcean. I have couple of test cloudrons on linode as well. For my use case, it would be great I could tie these to a single unified dashboard (+auth). This is probably slightly different from other scale out use cases where the servers are generally in the same rack/data center.
-
@plusone-nick Nice! I too use a thinkcenter for all my data. I think mine is an older model - M600. It's incredible how well it works. Since it's fanless, makes no sound whatsoever.
My home setup is outright amateurish compared to yours
There's 2 disks: one is an external disk for all my pictures/videos etc and the other one is for backups. My home has pretty stable internet/power, so I haven't had a need for UPS etc.My email server is on DigitalOcean. I have couple of test cloudrons on linode as well. For my use case, it would be great I could tie these to a single unified dashboard (+auth). This is probably slightly different from other scale out use cases where the servers are generally in the same rack/data center.
@girish I recall us realizing we are using similar lenovo nodes lol & Unfortunately that is not my setup just something i found on r/homelab BUT I DO have a similar rack with:
1 Dell R610
1 Sumpermicro (BigTwin - i believe) its 4 nodes in a 2UThey are both refurbs and of course they are as loud as a jet engine lol
That is part* of the reason for this thread - Using mini and micro Nodes like the Lenovo's have been more than enough and can apply to any living situation unlike a half rack lab lol -
@plusone-nick Nice! I too use a thinkcenter for all my data. I think mine is an older model - M600. It's incredible how well it works. Since it's fanless, makes no sound whatsoever.
My home setup is outright amateurish compared to yours
There's 2 disks: one is an external disk for all my pictures/videos etc and the other one is for backups. My home has pretty stable internet/power, so I haven't had a need for UPS etc.My email server is on DigitalOcean. I have couple of test cloudrons on linode as well. For my use case, it would be great I could tie these to a single unified dashboard (+auth). This is probably slightly different from other scale out use cases where the servers are generally in the same rack/data center.
-
@girish I have the same lenovo box! (Mine might be newer)
I put an SSD, and nvme drive inside, maxed out the ram and it runs like a champ!Mine is mounted inside a cuby hole where my modem lives.
@will They are freakin amazing! lol cant wait to make a mini cloudron cluster!
-
I'd be very interested in a multi-node cloudron setup. I have some stuff I'd be able to host at home, and other stuff I'd like to run on a VPS or physical server somewhere.
Also, there's software that needs tons of storage (like nextcloud), and others that need a bunch of memory but very little storage, so I'm currently renting a server with lots of storage and memory, which could easily be switched to a cheaper dual-node setup, I think.
So yeah, eagerly awaiting that 6.0 release. -
@malvim said in 3 Node Deployment (Unified Dashboard):
have some stuff I'd be able to host at home, and other stuff I'd like to run on a VPS or physical server somewhere.
Indeed, this is exactly the use case we are looking to support. I have a similar situation where my primary cloudron is hosted at home and my mail server is on a VPS and I have another server for test deployments. I would like all this to be unified in some way.
I bring this up because I have also been looking into solutions like k8s/swarm/nomad etc but they seem more suited for horizontal scaling whereas what I think we want is just a way to manage multiple cloudron installations.
-
@malvim said in 3 Node Deployment (Unified Dashboard):
have some stuff I'd be able to host at home, and other stuff I'd like to run on a VPS or physical server somewhere.
Indeed, this is exactly the use case we are looking to support. I have a similar situation where my primary cloudron is hosted at home and my mail server is on a VPS and I have another server for test deployments. I would like all this to be unified in some way.
I bring this up because I have also been looking into solutions like k8s/swarm/nomad etc but they seem more suited for horizontal scaling whereas what I think we want is just a way to manage multiple cloudron installations.
@girish TBH k8s/swarm/nomad all have latency issues to contend with that need specific network setups unless the Cloud provider already offers K8S with a dedicated network for the nodes. Been down this road and there's usually significant performance compromises as soon as anything needs to use a database.
What I'd be more interested would be hot-swappable Master-Slave(s) setups.
So you'd have your main hardware serving everything, and use Unison for file replication and whatever database Cluster versions for the Master-Slave replication.
Some decent guides on this for Wordpress:
Then you have a hot-swap timeout (say 5 minutes or whatever is enough for normal reboots with checks & fixes).
-
@malvim said in 3 Node Deployment (Unified Dashboard):
have some stuff I'd be able to host at home, and other stuff I'd like to run on a VPS or physical server somewhere.
Indeed, this is exactly the use case we are looking to support. I have a similar situation where my primary cloudron is hosted at home and my mail server is on a VPS and I have another server for test deployments. I would like all this to be unified in some way.
I bring this up because I have also been looking into solutions like k8s/swarm/nomad etc but they seem more suited for horizontal scaling whereas what I think we want is just a way to manage multiple cloudron installations.
@girish said in 3 Node Deployment (Unified Dashboard):
nomad
Yeah, I think in this case a way to manage different cloudrons in a centralized way would be the way to go.
For me, the main objective would be to install an app from my admin interface and have it go up in a specific machine, with domains and certificates working accordingly and such.
Haven't been able to go on with the raspberry pi setup, as my provider only has NAS disks, and overlay doesn't support it, and the other storage drivers were such a pain to setup I couldn't get it to work.
Looking for a provider with a better setup (and closer to what one would have at home) to try.
Edit: I'm sorry for the raspberry pi talk, I got the threads mixed up. But yeah, my "dream" multi-node setup is basically being able to install apps on different machines from the same web/cli interface.
-
-
This appears doable by creating a new App that orchestrates 2 or more cloudron APIs in one interface.
It could be a mobile/web admin app for any number of cloudrons with features like adding apps to each, user management across all, monitoring, etc.
@robi I think they are open to design/feature requests: https://forum.cloudron.io/topic/3205/what-s-coming-in-6-0-take-2/2
✌💙+1
-
@robi I think they are open to design/feature requests: https://forum.cloudron.io/topic/3205/what-s-coming-in-6-0-take-2/2
@plusone-nick Just looking at this: https://www.scalecomputing.com Do you have any experience with them? Any recommendations for it one wanted to make a multi-node, multi-site setup with failover redundancy?
-
@plusone-nick Just looking at this: https://www.scalecomputing.com Do you have any experience with them? Any recommendations for it one wanted to make a multi-node, multi-site setup with failover redundancy?
@marcusquinn have seen them but no hands on exp unfortunately - kept more of an eye on https://www.mirantis.com/ basically open-stack/kubernetes
but then gain that's basically just https://www.proxmox.com/en/ & Kubernetes
or https://opennebula.io/ & Kubernetes
lol✌💙+1
