How to Transfer file to multiple server at a same time ?

1. Using Shell Scripting and scp/rsync

You can create a shell script to loop through a list of servers and transfer the file to each server.

How It Works

• Write a list of server addresses.

• Use a loop in your script to iterate over each server and execute the file transfer command (e.g., scp or rsync).

Example Script

bashCopyEdit#!/bin/bash

# File to transfer
FILE_TO_TRANSFER="/path/to/local/file"

# List of servers
SERVERS=("server1.example.com" "server2.example.com" "server3.example.com")

# Remote directory where the file will be uploaded
REMOTE_DIR="/path/to/remote/directory"

# SSH username
USER="your_username"

for SERVER in "${SERVERS[@]}"; do
    echo "Transferring $FILE_TO_TRANSFER to $SERVER..."
    scp "$FILE_TO_TRANSFER" "$USER@$SERVER:$REMOTE_DIR" &
done

# Wait for all background transfers to complete
wait
echo "File transfer to all servers completed."

Pros:

• Simple and effective for small-scale deployments.

• Works with standard tools (scp, rsync).

Cons:

• Transfers are done individually, not truly simultaneous (although backgrounding processes can speed it up).

• Not optimized for very large infrastructures.

---

2. Using Parallel SSH (pssh)

Parallel SSH is a tool that allows you to run commands (including file transfers) on multiple servers simultaneously.

How It Works

Install pssh on your local machine and provide it with a file containing the list of server addresses.

Command to Install pssh

bashCopyEditsudo apt install pssh   # On Ubuntu/Debian
sudo yum install pssh   # On CentOS/RHEL

Steps to Transfer Files

1. Create a file (servers.txt) containing the list of server addresses:

    CopyEditserver1.example.com
    server2.example.com
    server3.example.com
   

2. Use the pscp (parallel SCP) command:

    bashCopyEditpscp -h servers.txt -l your_username -r /path/to/local/file /path/to/remote/directory
   

Pros:

• True parallelism.

• Scales well for medium-sized deployments.

Cons:

• Requires installation of pssh.

• Error handling can be tricky.

---

3. Ansible

Ansible is an automation tool designed for managing infrastructure, and it includes modules for file transfers.

How It Works

• Define your servers in an inventory file.

• Use the copy module or the synchronize module to transfer files.

Steps to Transfer Files

1. Install Ansible:

    bashCopyEditsudo apt install ansible  # On Ubuntu/Debian
    sudo yum install ansible  # On CentOS/RHEL
   

2. Create an inventory file (inventory):

    csharpCopyEdit[all]
    server1.example.com
    server2.example.com
    server3.example.com
   

3. Use the ansible command:

    bashCopyEditansible all -i inventory -m copy -a "src=/path/to/local/file dest=/path/to/remote/directory"
   

Pros:

• Scalable for large infrastructures.

• Easy integration into other tasks and automation workflows.

• Handles error reporting and retries automatically.

Cons:

• Requires Ansible installation and setup.

• Initial learning curve for Ansible.

---

4. Using rsync with SSH Multiplexing

rsync is a powerful tool for syncing files across servers. Using SSH multiplexing, you can speed up the process by reusing the same SSH connection.

How It Works

• Set up an SSH config file for multiplexing.

• Use rsync to copy files to multiple servers.

Example

bashCopyEdit#!/bin/bash

FILE="/path/to/local/file"
DEST="/path/to/remote/directory"
SERVERS=("server1.example.com" "server2.example.com" "server3.example.com")

for SERVER in "${SERVERS[@]}"; do
    rsync -avz -e ssh "$FILE" "$USER@$SERVER:$DEST" &
done

wait
echo "File sync completed."

Pros:

• Efficient for large files or directories.

• Supports partial file transfers (resumes broken transfers).

Cons:

• Requires setup for multiplexing.

• Slightly complex compared to scp.