How to automate migrating/importing apps from Cloudron to Cloudron via API

[robi]

There are two existing Cloudrons, A & B, each with their own separate domains and apps, and Cloudron B needs to retire, moving all its apps to A.

Since this isn't a full Cloudron restore, but a more individual app restore to a different Cloudron, times 100+; it would seem this needs some automating unless there is a better way.

Hence the ask, what might be best at this time before we have native Multi-Cloudron ability to flick apps across servers with a few clicks.

An alternate idea would be to rewire Cloudron A backup to B's backup location, so it can read and restore directly from there all relevant apps, but that is not exactly how the backup UI works as intended. Individual app backups are only visible from already installed apps, which isn't the case from A.

If this was one app import, I wouldn't be asking, yet this is 2 orders of magnitude more apps.

Please advise.

[nebulon]

If the goal is to move mostly everything, but only switch over one-by-one app, maybe you could restore the whole cloudron with dry-run option and then cut over DNS one-by-one? https://docs.cloudron.io/backups/#dry-run

Otherwise, I guess writing a small script against the Cloudron API will come in handy.

[jdaviescoates]

@nebulon said in How to automate migrating/importing apps from Cloudron to Cloudron:

maybe you could restore the whole cloudron with dry-run option and then cut over DNS one-by-one? https://docs.cloudron.io/backups/#dry-run

But how could one restore onto a VPS with an already existing Cloudron with lots of other apps already there?

[robi]

@nebulon :
@jdaviescoates is right.. Cloudron A needs preserving.

Are you aware of anyone writing such a script?

[jdaviescoates]

I will also nees to do this soon, so also interested...

[fbartels]

Quite some time ago I did some exploration around the backup part of the Cloudron api in https://blog.9wd.eu/posts/cloudron-migration/. I could imagine looking into this. Are you offering a bounty?

[robi]

@fbartels Thanks for your thoughtful response Felix!

This brings to mind another idea that Cloudron could have a 3rd party API plugin feature integration that would allow for such customizations just as Wordpress plugins do.

For me this does clarify the idea that a tool/script should be run from the destination Cloudron A to pull an app from the App backups of Cloudron B.

This would also allow for devs to share their pre-packaged Apps for testing and more (using group/domain visibility), similar to the Github fork feature, by sharing the backup configuration.

The next bit would be to iterate that import process for a list of App IDs provided.

If this were to be polished later as a plugin/feature, remote clone of an app across Cloudrons would be a few clicks away.

Another external option would be to use N8N to automate the process in a more visual workflow.

[robi]

@girish Looking at the Cloudron API docs, there doesn't seem to be an import path as there is in the UI.

The Restore path doesn't seem to be the right thing to use as it expects a local backup id.

How does the UI do the import via the API?

[robi]

Here's a draft outline of running this from Cloudron A via N8N:

Remote Cloudron B
GET
https://my.example.com/api/v1/apps
https://my.example.com/api/v1/apps/{appid}
use appStoreId, subdomain, domain
https://my.example.com/api/v1/apps/{appid}/backups
pick latest
https://my.example.com/api/v1/backups
https://my.example.com/api/v1/settings/backup_config
merge backup config with app backup config??

Local Cloudron A
POST
https://my.example.com/api/v1/apps/install
use appStoreId, subdomain, domain; get id
??? import merged remote backup config
https://my.example.com/api/v1/apps/{appid}/restore
use payload from remote app backup config???

What would the import calls looks like? @girish

[roofboard]

Here is the start of an N8N workflow.
Copy and paste all the JSON into your N8N instance it will detect the JSON and nodes.
We are using query auth here because we do not know how to do the Bearer Token.

And we are having trouble figuring out how to tell the destination cloudron to pull a backup from a remote location.

{
  "meta": {
    "instanceId": "8d8e1b7ceae09105ea1231dc6c31045b9d36dc713f8e22970f6eacf9f1f4d996"
  },
  "nodes": [
    {
      "parameters": {},
      "id": "4be687c9-4a4b-4be3-a8fc-d1405a66fa95",
      "name": "When clicking \"Execute Workflow\"",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [
        680,
        320
      ]
    },
    {
      "parameters": {
        "url": "https://my.demo.cloudron.io/api/v1/apps",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpQueryAuth",
        "options": {}
      },
      "id": "33c66a41-753b-491a-bf03-8683f86b95c5",
      "name": "PullAppsFromSource",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.1,
      "position": [
        900,
        320
      ],
      "credentials": {
        "httpHeaderAuth": {
          "id": "11",
          "name": "RobCloudron-Demo"
        },
        "httpQueryAuth": {
          "id": "12",
          "name": "Query Auth account"
        }
      }
    },
    {
      "parameters": {
        "jsCode": "// Loop over input items and add a new field called 'myNewField' to the JSON of each one\nfor (const item of $input.all()) {\n  item.json.myNewField = 1;\n}\n\nreturn $input.all();"
      },
      "id": "a1d203d0-4d01-421a-9d88-7299f8f515ce",
      "name": "ManipulateResponse",
      "type": "n8n-nodes-base.code",
      "typeVersion": 1,
      "position": [
        1120,
        320
      ]
    },
    {
      "parameters": {
        "url": "=https://my.demo.cloudron.io/api/v1/apps/{{ $json.apps[0].id }}/backups",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpQueryAuth",
        "options": {}
      },
      "id": "9c925efa-458f-4599-8172-7d9fafc3ce23",
      "name": "PullBackupsFromSource",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.1,
      "position": [
        1320,
        320
      ],
      "credentials": {
        "httpHeaderAuth": {
          "id": "11",
          "name": "RobCloudron-Demo"
        },
        "httpQueryAuth": {
          "id": "12",
          "name": "Query Auth account"
        }
      }
    }
  ],
  "connections": {
    "When clicking \"Execute Workflow\"": {
      "main": [
        [
          {
            "node": "PullAppsFromSource",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "PullAppsFromSource": {
      "main": [
        [
          {
            "node": "ManipulateResponse",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "ManipulateResponse": {
      "main": [
        [
          {
            "node": "PullBackupsFromSource",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

[robi]

@roofboard There's an example of how to use a bearer token here: https://forum.cloudron.io/post/65043

Looks like this in N8N for single use:

To reuse it in all nodes, this should work:
https://community.n8n.io/t/how-to-authorize-using-bearer-token/25674