ftz - A small file transfer tool
2021-03-04
I created a new tool for conveniently transferring files between two machines.
I created a new tool called ftz (File Transfer Zig) which allows to fetch or upload files onto another computed connected to network.
ftz Github Repository (and downloads)
What is ftz?
ftz (all lower case) is a file transfer tool that allows a client to connect to a server and fetch a file via a ftz url. It is also possible to upload files to a ftz server.
The tool and protocol is comparable to TFTP, but is based on TCP and provides an additional data integrity check.
To use ftz, just download an executable for your OS or build it yourself, then you can do the following:
# open the current directory for read/write access ftz host . # fetch file named "demo" from a ftz server "remote.machine" ftz get ftz://remote.machine/demo # upload the file "mode" to a ftz server "other.machine" with the name "demo" ftz put mode ftz://other.machine/demo
ftz has several command line options to adjust the port used for hosting as well as allow/deny get and put requests. You can find all usage options by invoking "ftz help" or checking out the GitHub link above.
Why another tool?
This question is answered easily: I didn't find a tool that fitted my requirements:
- Requires no install (so anything bigger is ruled out)
- Requires no admin right (rules out tftp and other low-port protocols)
- Has built-in validation (rules out netcat and similar)
- Allows control over uploads/downloads (rules out any non-scriptable web server)
- Is cross-platform compatible
So as I could find no tool that satisfied my needs, I created a new one. It appeared to be easy and I actually got from the idea to a production-ready implementation in less than 24 hours.
The protocol
ftz uses a very simple tcp based protocol, defaulting to port 17457. A new transfer is initiated by a client by connecting to the server and sending a request line.
Fetching a file
The client opens the tcp connection and sends "GET " followed by a "/"-separated path to the file. The path can be either relative or absolute and will be normalized by the server. The path is then terminated by a CR LF sequence. After this, the server will respond with 32 byte containing the plain text hex-encoded MD5 checksum of the file in hexadecimal notation, followed by a CR LF sequence. After this sequence, the server will send the contents of the file and terminate the connection.
Example transfer for a file called "cat.jpg":
Client: "GET cat.jpg" CR LF Server: "e1961be6e1370d615415a4354e08e131" CR LF Server: <FILE DATA>
Here, a text in quotes will be sent as verbatim ASCII. CR is the ASCII carriage return character (0x0D), LF is the line feed character (0x0A). <FILE DATA> is the binary contents of the file. The length of this data is determined by how much data the server sends. When the connection is closed, the client should verify that the MD5 checksum actually fits the transferred checksum by the server. If not, the received file should be deleted/discarded and a error message should be shown.
Upload a file
The client opens the tcp connection and sends "PUT " followed by a "/"-separated path to the file and a CR LF sequence. The same rules as in "fetching a file" apply for the path. After the path was sent, the client will then send the hex-encoded MD5 checksum of the file, followed by the CR LF sequence. After this, the data of the file is sent, the connection is then terminated. The server can abort the transfer by just closing the TCP connection.
Example transfer for a file called "cat.jpg":
Client: "GET /cat.jpg" CR LF Client: "e1961be6e1370d615415a4354e08e131" CR LF Client: <FILE DATA>
The same notation style is in "fetching a file" is applied.
Design Rationale
Here are some bullet points that went through my mind while designing the tool and the protocol:
- ftz has no authentication as it is meant to be used in a private network or between a host and virtual machines.
- A checksum is used to guarantee file integrity as the transfer is ended by terminating the connection.
- The MD5 checksum was chosen as it is a fairly small (16 byte) checksum and is widely available.
- ftz uses a ASCII/UTF-8 based encoding for the file names as those are the most widespread encodings in the internet
- An ASCII representation of the headers and checksum are chosen to simplify debuggability
- The protocol was designed to be easily implementable with common unix tools like md5sum and netcat.
- There is no option to list remote contents
- Upload and download are implemented to use separate directories so the user can decide where the files go
- ftz will not follow symlinks and thus allows no escape from the hosted folder, even by accident.
- ftz allows to restrict access to both only uploads or only downloads, thus providing a basic safety against unwanted access
Known bugs
- When a connection is not closed, it blocks all other pending connections until the currently active one will be closed.