curl and wget are the most popular command-line tools (with curl being a library) for downloading remote content from the CLI. On one side, wget is simple to use as it gets, downloading web content using the command: curl <some_remote_file_url>.
On the other side, curl is more than that. curl supports multiple protocols including HTTP, HTTPS, FTP, SFTP, SMTP, SMB, SCP and others. You could find the major differences here. Nonetheless, the topic for today is to learn how easily we could download remote content with curl.
Prerequisites
- Shell environment
Installation
### Debian-based distros
apt install -y curl
### RHEL-based distros
yum install -y curl
### CentOS8
dnf install -y curl
### macOS
brew install curl
Verify installation.
curl --version
Output:
curl 7.77.0 (x86_64-apple-darwin21.0) libcurl/7.77.0 (SecureTransport) LibreSSL/2.8.3 zlib/1.2.11 nghttp2/1.42.0
Release-Date: 2021-05-26
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL UnixSockets
Basic usage
Step 1. Let’s take for example the DevCoops site robots.txt file and download it.
curl -O https://devcoops.com/robots.txt
-O: Saves the file locally with the same name as the remote one. If you want to save the remote file on the local machine using custom name, use -o instead. For instance:
curl -o devcoops_robots.txt https://devcoops.com/robots.txt
Step 2. Verify the downloaded robots.txt file.
cat robots.txt
Expected output:
User-agent: *
Sitemap: https://devcoops.com/sitemap.xml
Now, if you have noticed, most of the third-party software that comes with curl installations, includes a remote bash script called install.sh and use redirection (pipe) to run the script. Let’s take the Node Version Manager nvm curl command installation as an example:
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
Note(s): Since -o specifies output file, the second dash - after the -o, points to the standard output, which means that the install.sh file will be downloaded and redirected as a standard input to the bash invocation.
Conclusion
As always, hope this was convenient enough. Feel free to leave a comment below and if you find this tutorial useful, follow our official channel on Telegram.