Syncing Files From Training Server

For larger training runs, we typically would use a dedicated remote training server to run.

This short document notes down a way to easily synchronize training checkpoints from the server down to local workspace.

To do this, we will use this VSCode extension:

GitHub - Natizyskunk/vscode-sftp: Super fast sftp/ftp extension for VS CodeGitHub

After download, create a sftp.json configuration under the .vscode/ directory and add these configurations:

{
    "name": "Your Remote Training Server",
    "host": "server.ip.address",
    "port": 22,
    "protocol": "sftp",
    "context": "./logs",
    "remotePath": "/PATH/TO/THE/WORKSPACE/logs",
    "username": "YOUR-USERNAME",
    "ignore": [],
    "uploadOnSave": false,
    "useTempFile": false,
    "openSsh": false,
    "privateKeyPath": "~/.ssh/id_ed25519"
}

Now, you can run "> SFTP: Sync Remote -> Local" command to fetch all the training logs and checkpoints.