SSH Access to Gitlab instance not working

[kaxline]

My ssh connection to Gitlab isn't working. I've set the SSH Port in the Cloudron app settings, but it doesn't seem to be punching through. Attempts to connect just hang with ssh git@gitlab.mydomain.com

I've tried specifying the port: ssh -Tvvv -p XXXXX git@gitlab.mydomain.com

debug1: Reading configuration data /Users/admin/.ssh/config
debug1: /Users/admin/.ssh/config line 1: Applying options for *
debug1: /Users/admin/.ssh/config line 9: Applying options for gitlab.mydomain.com
debug2: add_identity_file: ignoring duplicate key ~/.ssh/id_rsa
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 47: Applying options for *
debug1: Connecting to gitlab.mydomain.com port XXXXX.

I can ssh into the actual Cloudron server on port XXXXX with my root user.

Do I need to create a git user on Cloudron? Maybe adjust the iptables rules?

Any help is appreciated!

[BrutalBirdie]

@kaxline you can not ssh into the gitlab app, as far as I know you can not ssh into any cloudron app.

You should use the Web Terminal for ssh like access.
https://docs.cloudron.io/apps/#web-terminal

[kaxline]

@brutalbirdie Ah OK, interesting. Thanks for the answer!

[kaxline]

@BrutalBirdie So just to clarify, I can only ever use HTTPS clone urls as well?

[BrutalBirdie]

@kaxline No you can clone / push / pull via ssh.
But since the default port 22 is used by the root system cloudron assigns the gitlab app a custom ssh port.

If you install gitlab you get this window:

(alt. text: picture of the gitlab app installation showing the SSH Port configuration)

This means for my gitlab app on cloudron the custom ssh port 29418 will be used.
For example https://git.cloudron.io/cloudron is also hosted on cloudron (figures).

And if you want to clone the gitlab-app repo:

(alt. text: a screenshot from git.cloudron.io showing the custom clone url with the custom ssh port)

the custom port will be configured in the clone urls.

git clone ssh://git@git.cloudron.io:6000/cloudron/gitlab-app.git

This then will look like this on your local machine.

╭─eha@BrutalBirdie in repo: n8n-app on  master [!?] took 372ms
╰─λ git remote -v                                                                                    22:07:50
origin  ssh://git@git.cloudron.io:6000/cloudron/n8n-app.git (fetch)
origin  ssh://git@git.cloudron.io:6000/cloudron/n8n-app.git (push)

Hope this makes it a bit clearer

For more documentation about the gitlab app you can also visit the official documentation.
https://docs.cloudron.io/apps/gitlab/

[kaxline]

@brutalbirdie Thanks for the detailed response. I've actually tried all that and that led me to the issue I was trying to describe originally. Sorry I'm not being clear.

git clone ssh://git@git.cloudron.io:6000/cloudron/gitlab-app.git

Gives me an Operation timed out error.

Any ideas how to debug? Would this be on the Gitlab side or the Cloudron side?

[BrutalBirdie]

@kaxline sorry but what are you trying to achieve here?
First you write about the ssh problem with your own gitlab app, now there is a problem with the git.cloudron.io.
The Operation timed out is a different problem then your initial problem.

What is your goal exactly? Just see that clone via ssh is working?

It could have been that you ran into the automated server restart at night time that is why you got the Operation timed out.
You could have checked against that by pinging git.cloudron.io

When I try to clone the gitlab app via ssh from a random system I get this:

~/tmp # git clone ssh://git@git.cloudron.io:6000/cloudron/gitlab-app.git
Cloning into 'gitlab-app'...
The authenticity of host '[git.cloudron.io]:6000 ([45.55.2.141]:6000)' can't be established.
ECDSA key fingerprint is SHA256:5Z0PZclabDgIVpQ8VPxkR8L0SDVNylIYWUisPvYYS3A.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[git.cloudron.io]:6000,[45.55.2.141]:6000' (ECDSA) to the list of known hosts.
git@git.cloudron.io's password:

which is expected! Since this random system has no permission and/or authentication (ssh-key) in this system to clone via ssh!

Doing this via https works tho:

~/tmp # git clone https://git.cloudron.io/cloudron/gitlab-app.git
Cloning into 'gitlab-app'...
remote: Enumerating objects: 3241, done.
remote: Counting objects: 100% (320/320), done.
remote: Compressing objects: 100% (186/186), done.
remote: Total 3241 (delta 201), reused 228 (delta 134), pack-reused 2921
Receiving objects: 100% (3241/3241), 503.65 KiB | 788.00 KiB/s, done.
Resolving deltas: 100% (2134/2134), done.

If I now fork the app, then I can clone via ssh because my user account has my ssh-key.

╭─eha@BrutalBirdie in ~/Develop/cloudron took 6m
╰─λ git clone ssh://git@git.cloudron.io:6000/BrutalBirdie/gitlab-app.git                                                                                                                                   08:57:35
Cloning into 'gitlab-app'...
remote: Enumerating objects: 3241, done.
remote: Counting objects: 100% (320/320), done.
remote: Compressing objects: 100% (148/148), done.
remote: Total 3241 (delta 200), reused 282 (delta 172), pack-reused 2921
Receiving objects: 100% (3241/3241), 504.69 KiB | 1.42 MiB/s, done.
Resolving deltas: 100% (2133/2133), done.

I guess your desired outcome is to see that the ssh functionality from cloudron gitlab-app is working, but I am struggling to replicate your problem.
Maybe we should take a step back and you describe what you are trying to do and achieve.

[scooke]

Was it working before?

I searched the forum and found the following that may help you: https://forum.cloudron.io/topic/1780/cloudron-overrides-iptables-persistent/12

As well, from the official documentation: https://docs.cloudron.io/networking/#whitelist-ports.

[kaxline]

@brutalbirdie Sorry, still messing up communication despite my best efforts.

In my previous post, I only cited this command as an example:

git clone ssh://git@git.cloudron.io:6000/cloudron/gitlab-app.git

Since I didn't want to post my actual connection string, which is something like:

git clone ssh://git@gitlab.mydomain.com:XXXXX/cloudron/my-repo.git

And that's the one that is not working and has Operation timed out as the error. Again, this is not an issue with reaching git.cloudron.io. It's an issue with reaching gitlab.mydomain.com.

I'm just trying to use the ssh links for my repos hosted on my Cloudron instance of Gitlab. Nothing fancy. I just can't connect to them from my machine, even though I can ssh into the server that is running Cloudron.

Does that make sense?

[kaxline]

@scooke No, it was never working. I've always had to use the HTTPS links for repos.

Thanks for those links! I'll check them out and post back here if they work.

[girish]

@kaxline Does telnet gitlab.mydomain.com port connect? (replace port with your gitlab ssh port). I think if it doesn't connect, it's some firewall/networking issue and we should start from there.

[kaxline]

@girish Thanks for chiming in here. Indeed telnet gitlab.mydomain.com port hangs for me as well.

I white listed the port and restarted the cloudron-firewall service, but still no love.

Any other ideas of what to test/debug?

[mehdi]

@kaxline Which VPS provider are you using? Are there any other apps that you use which use external ports?

[girish]

@kaxline there's no need to white list the ports in cloudron-firewall, it's all automatic for the apps. As @mehdi asked, which VPS provider are you using? Some providers like azure/aws ec2/gce block all ports by default. So you have to add the port manually to the security group or equivalent. Other providerse like DO, linode, vultr, usually have everything open by default. But I think these days DO has added some tagging/labeling feature that can be mixed with their firewall to block ports by default for new VPS (not 100% sure about this).

[kaxline]

@girish @mehdi I'm using Netcup, so I guess I'll look into what their policy is unless you know off the top of your head. Actually started using them because people were recommending them after the last online Cloudron meetup I went to.

[girish]

@kaxline netcup should have the SSH port open. If you can write to us at support@cloudron.io, we can take a look as to why/where the SSH is getting blocked.

[girish]

Followed this up on support@, the issue was that the gitlab instance was behind Cloudflare. Cloudflare, of course, cannot proxy non-http ports. This is why the ssh+clone wasn't working. I will look into adding a warning about this when trying to install apps which require a port and a Cloudflare domain is used.

[girish]

Created https://git.cloudron.io/cloudron/box/-/issues/802