Bandwidth throttling when creating archives with tar

Bandwidth throttling when creating archives with tar

Introduction

If you need to create a large archive and want to spare your disk big IO operations then use the pv or cstream utility.

Say hello to PV

What is PV?

This is the definition from the author: “pv - Pipe Viewer - is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion.”

Installation

Install the pv utility.

apt-get install pv

Create the archive with tar and pipe it to pv which will do the bandwidth throttling.

tar -cpzf - web/ | pv -L 1m > ./web.tar.gz

CSTREAM as the alternative to PV utility

Alternative way is to use cstream utility.

Installation

apt-get install cstream

Usage

Run command with bandwidth limit of 777k

tar cj file.txt | cstream -t 777k | tar xj -C /tmp/
  • https://www.ivarch.com/programs/pv.shtml