Linux copy next ‘n’ number of files from one folder to another in bash/python script

Welcome to Programming Tutorial official website. Today - we are going to cover how to solve / find the solution of this error Linux copy next ‘n’ number of files from one folder to another in bash/python script on this date .

We have a 17k files with name like file1.csv, file2.csv, file3.csv …file17000.csv. All this files should be copied from one folder to another. The goal is create Linux bash or python script to copy all this files divided by ‘n’ number of csv files every 5 minute, and prevent copying ‘n’ number of files that already copied.

The idea is:

copy file1.csv file2.csv file3.csv  file4.csv  file5.csv to destination_dir
sleep for 300 seconds
copy file6.csv file7.csv file8.csv file9.csv file10.csv to destination_dir
sleep for 300 seconds
...
copy file16996.csv file16997.csv file16998.csv file16999.csv file17000.csv to destination_dir

For small number of files we’ve been used below script to copy files between 2 ranges:

#!/bin/bash
source_dir='/source_dir'
target_dir='/target_dir'
echo "beginning number:$1"
echo $1
echo "finite number:$2"
echo $2
for f in $(eval ls $source_dir/file{$1..$2}.csv);
do
cp $f $target_dir
done

Can anyone suggest how to correctly point in the script to use next ‘n’ number of csv files

Any advice and suggestions will be greatly appreciated.

Answer

Finally, we’ve done it by updating script provided by Marcel. We added while function to read list of files within array, and it works as we expected:

#!/bin/bash
all_csv_files=()
source_dir='/source_dir'
target_dir='/target_dir'
while IFS=  read -r -d $''; do
    all_csv_files+=("$REPLY")
done < <(find $source_dir -name "file*.csv" -print0)

echo ${#all_csv_files[@]}

batch_size=5
sleep_break=60
file_counter=0

echo Found ${#all_csv_files[@]} files

for f in "${all_csv_files[@]}"
do
    cp $f $target_dir
    echo $f
    let file_counter++
    if [ $file_counter == $batch_size ]
    then
        echo "Take a break $(date)"
        file_counter=0
        sleep $sleep_break
    fi
done
echo Done

Thank you all for suggestion!