SyncConfiguration for Pulling all Streams from a Remote Repository

[ruediste]

The stream patterns in the SyncConfiguration are evaluated against the locally available streams. This leads to the following problem:

From @Polve:

... my problem was precisely to know in advance which remote streams there are to backup. Since the remote server is managed by other people, my goal is to use the backup server to automatically backup all remote streams, even new ones never initialized before.

So I created a little batch file that leverage the "list" command to know the remote streams and add them to the local repo if not already present. The code is similar to this:

cd <myrepo>
for stream in `ssh root@$serverName "cd /repo; btrbck list" | tail -n +2` ; do
  if [ ! -d $stream ]; then
    btrbck create $stream
  fi
done

It would be nice to have this kind of behaviour directly in btrbck itself ....

[Polve]

👍

[ruediste]

I just had a quick look at the sources to get you started. In the SnapshotTransferService, streams get transferred one-by-one. So I guess it is easier to create an additional functionality which separately retrieves the stream list from a remote repository.

Suggestion: add a flag to the list command to switches it to a machine-readable mode, in which it outputs

<START TAG>
<stream name>
<stream name>
....
<END TAG>

with tags like "BTRBCK stream list start".

Then you can create a StreamListTransferService with a listStreams() method similar to SnapshotTransferService.pull() and use it from CliMain.cmdProcess() to retrieve the stream name list.

I'd always read the remote stream names when pulling...

[ruediste]

Refinement: Instead of a plain text stream name list, create a DTO with the stream name list...