Secure Copy Protocol implemented in Go

Overview

Production-ready Secure Copy Protocol (aka: SCP) implemented in Go with well documentation and neat dependency.

Introduction

Secure Copy Protocol uses Secure Shell (SSH) to transfer files between host on a network.

There is no RFC that defines the specifics of the protocol. This package simply implements SCP against the OpenSSH's scp tool, thus you can directly transfer files to/from *uinx system within your Go code, as long as the remote host has OpenSSH installed.

Features

• Copy file from local to remote.
• Copy file from remote to local.
• Copy from buffer to remote file. (e.g: copy from bytes.Reader)
• Copy from remote file to buffer. (e.g: copy to os.Stdout)
• Recursively copy directory from local to remote.
• Recursively copy directory from remote to local.
• Set permission bits for transferred files.
• Set timeout/context for transfer.
• Preserve the permission bits and modification time at transfer.
• No resources leak. (e.g: goroutine, file descriptor)
• Low memory consuming for transferring huge files.
• TODO:
  • Transfer speed limit.
  • Performance benchmark/optimization for lots of small files.
• Won't support:
  • Copy file from remote to remote.

Install

go get github.com/povsister/scp

Example usage

This package leverages golang.org/x/crypto/ssh to establish a SSH connection to remote host.

Error handling are omitted in examples!

Copy a file to remote

// Build a SSH config from username/password
sshConf := scp.NewSSHConfigFromPassword("username", "password")

// Build a SSH config from private key
privPEM, err := ioutil.ReadFile("/path/to/privateKey")
// without passphrase
sshConf, err := scp.NewSSHConfigFromPrivateKey("username", privPEM)
// with passphrase
sshConf, err := scp.NewSSHConfigFromPrivateKey("username", privPEM, passphrase)

// Dial SSH to "my.server.com:22".
// If your SSH server does not listen on 22, simply suffix the address with port.
// e.g: "my.server.com:1234"
scpClient, err := scp.NewClient("my.server.com", sshConf, &scp.ClientOption{})

// Build a SCP client based on existing "golang.org/x/crypto/ssh.Client"
scpClient, err := scp.NewClientFromExistingSSH(existingSSHClient, &scp.ClientOption{})

defer scpClient.Close()


// Do the file transfer without timeout/context
err = scpClient.CopyFileToRemote("/path/to/local/file", "/path/at/remote", &scp.FileTransferOption{})

// Do the file copy with timeout, context and file properties preserved.
// Note that the context and timeout will both take effect.
fo := &scp.FileTransferOption{
    Context: yourCotext,
    Timeout: 30 * time.Second, 
    PreserveProp: true,
}
err = scpClient.CopyFileToRemote("/path/to/local/file", "/path/at/remote", fo)

Copy a file from remote

// Copy the file from remote and save it as "/path/to/local/file".
err = scpClient.CopyFileFromRemote("/path/to/remote/file", "/path/to/local/file", &scp.FileTransferOption{})

// Copy the remote file and print it in Stdout.
err = scpClient.CopyFromRemote("/path/to/remote/file", os.Stdout, &scp.FileTransferOption{})

Copy from buffer to remote as a file

// From buffer
buffer := []byte("something excited")
reader := bytes.NewReader(buffer)

// From fd
// Note that its YOUR responsibility to CLOSE the fd after transfer.
reader, err := os.Open("/path/to/local/file")
defer reader.Close()


// Note that the reader must implement "KnownSize" interface except os.File
// For the content length must be provided before transfer.
// The last part of remote location will be used as file name at remote.
err := scpClient.CopyToRemote(reader, "/path/to/remote/file", &scp.FileTransferOption{})

Recursively copy a directory to remote

// recursively copy to remote
err := scpClient.CopyDirToRemote("/path/to/local/dir", "/path/to/remote/dir", &scp.DirTransferOption{})

// recursively copy to remote with timeout, context and file properties.
// Note that the context and timeout will both take effect.
do := &scp.DirTransferOption{
    Context: yourContext,
    Timeout: 10 * time.Minute,
    PreserveProp: true,
}
err:= scpClient.CopyDirToRemote("/path/to/local/dir", "/path/to/remote/dir", do)

Recursively copy a directory from remote

// recursively copy from remote.
// The content of remote dir will be save under "/path/to/local".
err := scpClient.CopyDirFromRemote("/path/to/remote/dir", "/path/to/local", &scp.DirTransferOption{})

Something you need to know

SCP is a light-weighted protocol which implements file transfer only. It does not support advanced features like: directory listing, resume from break-point.

So, it's commonly used for transferring some small-size, temporary files. If you heavily depend on the file transfer, you may consider using SFTP instead.

Another thing you may notice is that I didn't put context.Context as the first argument in function signature. Instead, it's located in TransferOption. This is intentional because it makes the API also light-weighted.

License

MIT License