Cloudron as an app inside of Cloudron

Lonkle

Thanks to the Docker add-on, I feel like it might be possible to package Cloudron for Cloudron as an app.

My only motivation for doing so would be to help with development and treating it like an app would be not only cool, but super efficient for development especially with the Cloudron Build Service installed. Super quick compilation and testing.

I think it was @mehdi and @robi who expressed the possibility of running Cloudron in Cloudron but I was curious if anyone thinks it’s feasible enough for me to try to code?

BrutalBirdie

I can understand the wish for that.

I work a lot with CI/CD, even now when developing a new app for Cloudron.
My new push will be deployed to a test instance so I can test it.

Having this for the development of Cloudron it self could be beneficial.
Thinking about a growing community and more developers having the option the just give access to a cloudron instance running on a Cloudron would make the PR review faster/easier.

But I don't know if this is necessary already.

mehdi

I would theoretically be possible, I think, but honestly it would be an infrastructure nightmare to set-up. My opinion => don't go this route, use proper VMs. In addition to being much easier to accomplish, you'll also get an environment much closer to an actual cloudron, so there will be much less weird different behaviours.

Lonkle

@BrutalBirdie Not necessary, no no. But I’ve been using ssh -> nano to make all my box changes so I didn’t have to reinstall every change and it’s just really cumbersome.

I also don’t mind coding something not necessary right now if developers might find it beneficial (like I do). I was just curious if people thought I should even try to. Like, is there anything obvious that would make Cloudron just too hard to convert to an app?

BrutalBirdie

@mehdi said in Cloudron as an app inside of Cloudron:

I would theoretically be possible, I think, but honestly it would be an infrastructure nightmare to set-up. My opinion => don't go this route, use proper VMs. In addition to being much easier to accomplish, you'll also get an environment much closer to an actual cloudron, so there will be much less weird different behaviours.

Thought about the same, with the Hetzner API I can spin up a cloudron with the gitlab ci/cd to install my working changes and review my work.

Lonkle

@BrutalBirdie Do you make box changes as well?

BrutalBirdie

@Lonk said in Cloudron as an app inside of Cloudron:

@BrutalBirdie Do you make box changes as well?

No, I did not yet contribute to the cloudron "base" itself.
So I don't know what pains you are going threw

pain >

But I’ve been using ssh -> nano to make all my box changes so I didn’t have to reinstall every change and it’s just really cumbersome.

Lonkle

@mehdi said in Cloudron as an app inside of Cloudron:

I would theoretically be possible, I think, but honestly it would be an infrastructure nightmare to set-up. My opinion => don't go this route, use proper VMs. In addition to being much easier to accomplish, you'll also get an environment much closer to an actual cloudron, so there will be much less weird different behaviours.

You mentioned this before. Since this is just for testing new box code quickly with the same flow as a custom app. Do you still think it’s more work than it’s worth? Infrastructure nightmare is particularly what I’m confused about; I want to understand what you mean by that.

Lonkle

@BrutalBirdie said in Cloudron as an app inside of Cloudron:

pain >

But I’ve been using ssh -> nano to make all my box changes so I didn’t have to reinstall every change and it’s just really cumbersome.

Not gonna lie, it feels like I’m coding in the 90s whenever I use nano. You can’t even select text. Every character has to be deleted one by one. It’s faster than constantly recompiling and re-installing on Ubuntu 18.04. But slower than just about anything else.

girish

@Lonk Are you already using the hotfix script? https://git.cloudron.io/cloudron/box/#development This is what we use for development.

Lonkle

@girish said in Cloudron as an app inside of Cloudron:

@Lonk Are you already using the hotfix script? https://git.cloudron.io/cloudron/box/#development This is what we use for development.

This looks like a godsend. I just have to figure out how to format hot fixes and use this CLI tool to apply them to my virtual machine Cloudron. Thank you for showing me this!

Also, I saw in that description in your link: “front-end code click here” and it was 404 .

But using this hot fix tool I might be able to contribute to dashboard easier (like the drop down box to choose the vpnclient to connect to).

girish

@Lonk said in Cloudron as an app inside of Cloudron:

Also, I saw in that description in your link: “front-end code click here” and it was 404 .

Are you logged into GitLab? I feel like I have solved this permission/visibility issue in GitLab like a million times. I am not sure what GitLab is doing but that setting gets reset of something after each update

Lonkle

@girish I clicked on the "click here" link which lead to this:

https://git.cloudron.io/cloudron/dashboard

It asked me to log in, which I did (as Lonk), and it lead to a 404. So I tried clicking it again thinking it could have been a login redirection issue and I'm fully logged in now, and it still says 404.

robi

If you want to try and get this going, here are a few rough steps.

Install sysbox from Nestybox and validate you can spin up containers with sysbox-runc
Get a small Ubuntu 18.04 image running inside
Validate networking in and out for basic services
Install the inner Cloudron and see how to set up DNS and ports so it works for the inner and outter Cloudrons separately.
Document the kinks

Once this is running, you'll have a base container system image which is special and can run as an App.

It may be useful to set up guacamole as remote desktop access.

Feel free to chime in and add to my list.

Lonkle

@robi I plan on contributing more here so I'll try both yours and @girish's hot fix dev methods and see which one works better for me.

I genuinely appreciate you taking the time to put those steps together for me. Thank you! ️

Lonkle

@girish cloudron machine says it's not an available command. Though it's documented to be so in the npm main database. https://www.npmjs.com/package/cloudron

Do I need to load on top of my already existing cloudron cli tool?

girish

@Lonk It's a separate repo and a separate tool - https://git.cloudron.io/cloudron/cloudron-machine . If you clone and do "npm link . ", it is available in your PATH.

Lonkle

@girish said in Cloudron as an app inside of Cloudron:

@Lonk It's a separate repo and a separate tool - https://git.cloudron.io/cloudron/cloudron-machine . If you clone and do "npm link . ", it is available in your PATH.

Joel$ cloudron-machine
Usage:  [options] [command]

Options:
  -V, --version            output the version number
  -h, --help               output usage information

Commands:
  hotfix [options]         Hotfix Cloudron with latest code
  ssh [options] [cmds...]  Get a remote SSH connection

Ah, I understand, you should fix that in the npm repository description cause I was so confused for a second. Thank you very much for taking the time to write out instructions in how to get the actual tool. ️

But both the commands hotfix and ssh error out instead of displaying manuals. Are there manuals to those commands? Or maybe an example of a hotfix option and ssh option (using fake data of course)?

Lonkle

@girish Like your cloudron build --no-push hot fix that you've already uploaded to master, how could I apply that with this tool?

Lonkle

I think I get it actually. I supply it the whole cloudron installation with all the hotfixes at once and it overwrites the appropriate files with the ssh commands I give it. Still figuring out the syntax though.

Lonkle

I've only ever logged in via password but I have to use files for the ssh I give it I think. And I must have to be in the root of the cloudron box I'm hotfixing:

SSH_PASSPHRASE=sshkeypassword cloudron-machine hotfix --cloudron my.example.com --release 6.0.0 --ssh-key keyname

Lonkle

@girish I still haven't figured out what hotfix is in the documentation (I think it's a directory of cloudron files) and this app has no manuals.

Lonkle

@girish I've re-installed Cloudron. I'm assuming I run this cloudron-machine hot fix CLI before I set anything up because it goes in via SSH and does it's thing?

I just can't tell the syntax and what hotfix is in the syntax? Is it a directory of all the changed yellowtent files in which case it overwrites them? Should I start the folder structure at /yellowtent/ or /box/?

nebulon

The hotfix command currently is really only used and thus tailored to @girish and my use-case. It used to be part of the normal cli tooling, but since it requires a specific folder setup of box and dashboard repo we removed it from there.

In order to make it useful for more people, I think we have to rework that quite a bit.

Lonkle

@nebulon said in Cloudron as an app inside of Cloudron:

It used to be part of the normal cli tooling

Yeah, it's still documented in the npm repo as cloudron machine for the normal CLI.

But it would really help me with updates and applying my hot fixes (one in development made by @girish that's already in master but not released, and the other my VPN Client hot fixes until @girish is ready to work with me to integrate them into core in 2021).

Basically, @nebulon, is there anyway I can use this tool? @girish acted as if it was the perfect tool to use in my case. Because every update I have to SSH into the server and nano ../home/yellowtent/box/src/docker.js along with 3 other files to keep testing my app every time you guys update (it's treacherous, you can't even select text in nano every change has to be manually redone). Soooo, is it possible for me to use this tool to do that?

Also, @girish told me I should have permission to see the dashboard repo (the VPN client needs a dropbox in every app's dashboard config to choose which VPN Client to connect to) but it returns a 404 even when logged in (which @girish said was a bug a few posts above this one).

I figured the CLI required an exact folder format. I was guessing the entire installation of Cloudron /yellowtent/box/ pulled from the server. I then make my hot fixes (including girish's --no-push hot fix from master). And the hotfix tool took the SSH credentials and did diffs to compare and overwrite as needed.

Am I close? This tool is very undocumented but girish made it sound like it was needed for me (probably felt bad I was using nano via ssh).

girish

@Lonk Maybe you can debug a bit to figure what needs to be done/fixed. All it's doing is really ssh'ing and copying files and restarting the box service mostly.

Lonkle

Edit: Disclaimer - I don't want to waste anyone's time so in the end I will RE if need be, but I think my question should require a quick copy paste or two of the directory it needs and the syntax you use. But if that's too much, I'll RE, I don't want to bother you guys. I'm also trying to work on being more concise in written commnication and this decidedly wasn't.

@girish Well, yes, I understand that the tool is ssh-ing into the VPS for the file changes (but didn't know it only restarted box, meaning I can do this at anytime, I've been waiting on setup depending on your response in this thread).

It's just, I don't know from what directory does cloudron-machine hotfix hotfix. @nebulon mentioned I needed dashboard access for it to even work which you'd mentioned with me being logged into gitlab I should have access to (but it 404s). So, don't get me wrong, I'll RE the crap out of this entire program, I'm just trying not to do so if what nebulon says is true and I don't have access to a vital part of using the tool. I ideally wanted an example (like a copy and paste of the way you use it in the command line, but with fake data ofc). But, I understand if you don't have time for that.

You're making it sound like RE-ing wouldn't be a waste of my time then? Cause if I want to build apps and always make sure they work on the bleeding edge of your code, this would make it a lot easier to do so and I'm willing to RE. So, basically, I'm only asking your opinion on if what @nebulon sounded like he was implying that it would be impossible to use tool you recommended me due to currently only having limited access on gitlab (and the file structure necessary that cloudron-machine takes requires dashboard access as he said). Or do you think I could RE myself around that issue? Cause I haven't figured out the file directory yet but I also haven't included dashboard so maybe that's why.

girish

@Lonk Yes, the tool requires dashboard code access to work.

I have made https://git.cloudron.io/cloudron/dashboard/ public for now. I will debug why the internal setting does not work in GitLab when I find the time. Curiously, it works for some users..

For the hotfix itself, the directory structure is that the repos are checked out in same directory. Like if you have a folder "foo", then the repos are in foo/dashboard and foo/box . Then you cd into foo/box and then run the hotfix tool. That should really be it.

If you want to install a specific release, you have to check out the same tag on both the repos and run hotfix tool again. Also, there is no way to "revert" back to older release in Cloudron, you have to start afresh. We always do rolling updates.

Lonkle

@girish said in Cloudron as an app inside of Cloudron:

@Lonk Yes, the tool requires dashboard code access to work.

I have made https://git.cloudron.io/cloudron/dashboard/ public for now. I will debug why the internal setting does not work in GitLab when I find the time. Curiously, it works for some users..

For the hotfix itself, the directory structure is that the repos are checked out in same directory. Like if you have a folder "foo", then the repos are in foo/dashboard and foo/box . Then you cd into foo/box and then run the hotfix tool. That should really be it.

If you want to install a specific release, you have to check out the same tag on both the repos and run hotfix tool again. Also, there is no way to "revert" back to older release in Cloudron, you have to start afresh. We always do rolling updates.

Omg, you're the best - that was exactly what I needed. Thank you for taking the time to write out, and explain that to me. I'm sorry if I'm a burden sometimes with my verbose questions. Just know I'm grateful for your help! I honestly think this will help me always keep up to date that my current / future apps so that my apps always support the latest Cloudron changes. Sorta like testing on beta before master releases to make sure there are no bugs (once I build out a build process to have my development server always be running on master using this tool).

Anyway, let me know when you want to test "Gitlap Repo Permissions" with my account and dashboard "permissions" since we at least both know that my account specifically gets the 404. Maybe that's related to why my account is able to create new repos but unable to import a project from Github (where the current code for the OpenVPN Client is). I'll transfer it at some point, but I couldn't afford the fancy Git Desktop app (that supported any repo) and "GitHub Desktop" is free, so I've just been using that. But I will transfer it over by hand at some point. Likely, after 6.0 comes out when you have time to go over the box changes.

Lonkle

@girish I almost got this working, had to work through some issues being on OS X but the tool fully works. Pretty cool.

But, quick question, this hotfix tool expects either a compiled or uncompiled version of dashboard and I'm not sure which path is correct: if it's compiled into box the path is /dashboard/dist/dashboard_files, if it's uncompiled outside of box it's just /dashboard/dashboard_files.

Do you know happen to know off the top of your head?

Lonkle

Okay, yeah, I think it's not the compiled directory, it's just /dashboard/, I'm so close now - but check my latest, kind of hilarious error. Any ideas on this one?

Joels-Macbook-2016:box Joel$ cloudron-machine hotfix --cloudron my.example.com --ssh-key id_rsa --release 5.6.4
This script requires node 10.18.1
Unable to create version tarball.
Joels-Macbook-2016:box Joel$ node -v
v12.17.0
Joels-Macbook-2016:box Joel$

Does it mean it requires that exact node version. Oh, development.

Lonkle

@Lonk said in Cloudron as an app inside of Cloudron:

Does it mean it requires that exact node version. Oh, development.

Figured it out. It actually did mean that, switched versions and it's compiling, even comiling through the dashboard GULPFILE so I'm positive it just wanted the repo now, not the final version. This hotfix tool is almost a full compiler.

Lonkle

Got the tool fully working on macOS. Worked like a charm after I figured out everything the tool wanted.

I had to make a small commit to dashboard since macOS doesn't support the timedatectl CLI tool so I converted that command into a universal one that returns the same values on either Ubuntu and Macs. But that was literally the only thing that didn't work in the entire tool and I've already committed a change for that (locally, of course).

So, thanks for your help @girish, this will allow me to build a cloudron-master Cloudron install that's, ofc, unstable - but it will be just to test my apps in before you turn your new commits into a release to make sure my apps are always compatible with your newest changes! ️

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Cloudron as an app inside of Cloudron