sftp is not enough

[roofboard]

@girish
hmmm breaking is a problem.
Lets start with the motivation.
I have a website which needs regular updates coming from gitlab-ci
-- Originally
I was just going to sftp the files over.
but the files are in folders with subdirectories and you cannot recursively delete using sftp.
-- Backup plan
Make special user of the host machine which can access the data folders in yellowtent. Now for this new linux user I can access just the directories which need updating.

-- I would have used my root keys
But that did not sound like a good idea I don't like the idea of storing my private key on gitlab

-- I would have installed something on the container
but that would have the issue of getting de-configured on upgrade

--maybe I can use Cloudron CLI???
can I launch web-terminal from the CLI? That would be super awesome because I can install the cli on a gitlab runner

-- It worked last night- I was able to automate pushing files to the website but if that is going to break... how is it going to break?

My ideal would be to ssh directly into the container and be able to do anything...

[BrutalBirdie]

@roofboard
I got multiple customers who run a full fledged Cloudron Gitlab with CI/CD to staging / live Cloudron servers.

Yes we use the cloudron-cli for many things in the Gitlab CI/CD.

• db dumps / imports
• pre/post and deploy
• pushing data to /app/data
• configuring domains
• and more

and if something is missing from the cli there is still the API.

But we have custom apps for each Project since the vanilla Cloudron apps are not always working with what the customers wants or has in his projects.

I can tell you it works and it works very well.
But I can't tell you how exactly since that is also how I make my money

[roofboard]

@roofboard
Yah I see it now. I should have gone for the CLI first -- reinventing the wheel over here!

[BrutalBirdie]

@roofboard happens to the best!
Better realize that now then setting up a monster to later realize there was a way easier way.

[roofboard]

@BrutalBirdie so.. how do you get around all the ci error which come from having TTY disabled in the ci runner?

[roofboard]

@BrutalBirdie Can you share some of your ci commands?

I guess I am just going to upload a zip archive and the log in with exec to decompress it...

#does not work because it does not recusively copy files

script -q -c 'cloudron login --password ${CLOUDRON_PASS} --username ${CLOUDRON_USER} my.domain.com \
cloudron push --app members.domain.com ${name} /app/data/wp-content/plugins/${name}'

#does not work because of unknown errors

script -q -c 'cloudron login --password ${CLOUDRON_PASS} --username ${CLOUDRON_USER} my.domain.com \
rsync -avz -e 'cloudron exec --app members.domain.com --' ${name} /app/data/wp-content/plugins/'

#does not work because of ttl errors

cloudron login --password ${CLOUDRON_PASS} --username ${CLOUDRON_USER} my.domain.com
cloudron push --app members.domain.com ${name} /app/data/wp-content/plugins/${name}

#This managed to work!!! errors above in the single quotes - you cannot pass vars through single quotes.

script -q -c "cloudron login --password ${CLOUDRON_PASS} --username ${CLOUDRON_USER} my.domain.com; \
cloudron push --app members.domain.com ${name}.zip /app/data/wp-content/plugins/ ; \
cloudron exec --app members.domain.com -- unzip -o /app/data/wp-content/plugins/${name}.zip -d /app/data/wp-content/plugins/ ;
sleep 3; \
cloudron exec --app members.domain.com -- rm /app/data/wp-content/plugins/${name}.zip ; \
cloudron exec --app members.domain.com -- chown -R www-data:www-data /app/data/wp-content/plugins/${name} "

[girish]

@roofboard Have you seen this doc page already - https://docs.cloudron.io/packaging/cli/ ?

All the way down, there is a CI/CD section, which gives an example of using --token instead of username/password.

Pushing directory should also work - https://docs.cloudron.io/packaging/cli/#pushing-a-file

I have never used the script command. What does it do?

[girish]

@roofboard said in sftp is not enough:

#does not work because of ttl errors

This is expected. Don't use exec in automation . Only for interactive tty or to run commands for which you want a result. For file management, use push and pull .

[roofboard]

@girish

the script command is like an inline .sh
however I was not able to use the push command because it was not getting the subdirectories.

What I would really like to see is an example of using cloudron exec in combination with rsync. I was not able to get that working but rsync would be the ideal way to push and pull files from the server.

[girish]

@roofboard depending on your use case, you can also just use rsync+ssh directly no? The app data is in /home/yellowtent/appsdata/<appid>/data .

[roofboard]

@girish the issue with using it directly is that I do not want to expose my root keys. Honestly I have done that in the past, but with setting up automation on the server I want it's access to be scoped. I mean we kinda have something with cloudron exec, it allows me to do almost anything. However I am having big trouble running rsync through it. That would be a game changer.

I mean... I could rsync from the server to my runner - but that just sounds crazy.

[girish]

@roofboard that makes sense. I would personally not throw in a ssh key into a CI/build system either

I guess you are looking for a cloudron sync or something from the CLI tool. I guess this is possible to implement. Can you open a feature request at https://forum.cloudron.io/category/97/feature-requests ?