Tar Backups timing out on too large part number

[girish]

@adrw It seems parallel multi-part copy will work per https://dzone.com/articles/amazon-s3-parallel-multipart . Looks like a good change to make.

[adrw]

@girish The only advanced features I see for the tar backups is the memory, none of the parallel copy or upload/download like I see when I'm in rsync mode.

[adrw]

@girish Thanks! That's what I was looking for, I'll give that a shot.

[girish]

@adrw Ah indeed. When doing tar backup, there is only one file to upload and copy. Nothing to do in parallel (apart from the multi-part copy which I think can be hardcoded).

[adrw]

@girish Could the copy that happens at the end of the task (I think from the snapshot folder to the timestamped one) be done in parallel? It seems to be done serially right now which contributes to the longer task time to some extent.

[adrw]

@girish Could this timeout be configurable in a future release?

[girish]

@adrw I made the timeout now 24 hours. The timeout is really just there to kill "stuck" backups. This can usually only happen when there is a bug in our code.

[adrw]

@girish Great! Thanks again for your help debugging this and adding more configuration. Huge help for larger backups like mine.