inCrypt - in-place crypto conversion

inCrypt is a command-line tool that applies encryption to a block device (or a regular file used as such). It allows a data payload to be converted between raw(unencrypted), dm-crypt plain encryption and dm-crypt LUKS encryption.

inCrypt was written as a learning tool and has served its purpose in that regard. Before using it, consider it an unfinished pre-alpha grade piece of work that only considers its happy path.

This document describes how to use inCrypt and explains how it works by practical examples that can be followed from a bash command-line.

inCrypt may be downloaded from here (MIT License).

! known issue mbuffer doesn't reliably do its job. See here.

Usage

inCrypt converts the data payload on a device. It converts from one format to another.

$ incrypt device to [cryptsetup args]
$ incrypt device from to [cryptsetup args] 
$ incrypt device plain luks yes|no [cryptsetup args] 

In the first form the encryption mode of the device will be auto-detected, if possible. If autodetection isn't possible then the second form must be used (this will be the case for devices that are encrypted with plain-mode encryption).

In the second form the encryption mode of the device is explicitly specified as from and no auto-detection is performed.

In these forms the device will be converted to the encryption mode given by to, which can be one of the supported encryption modes listed below.

In the specific case of converting from luks to plain, a fourth argument is mandatory and specifies whether the data should be re-encrypted: yes causes re-encryption and allows a new passphrase to be selected; no does not re-encrypt but exports the existing LUKS master key as a key file. There is no way to recover a passphrase for this key.

Any remaining arguments are passed verbatim to cryptsetup when it is used on plain devices; they are not used for LUKS devices. In the absence of any such arguments, the following defaults are applied:

--cipher aes-xts-plain64 --key-size=512 --hash sha512

Supported encryption modes

  • raw - no encryption
  • plain - dm-crypt plain mode encryption
  • luks - dm-crypt luks mode encryption

Space Requirements

Converting into LUKS format adds a header in front of the data. If the volume is a regular file then the resulting file will be larger. If it is a physical block device then there must be sufficient free space outside any contained filesystem to allow the header to be accommodated. This may require a filesystem on a block device to be shrunk - inCrypt does not do this for you.

The amount of additional space that is required is determined by the size of the LUKS header and any block aligment performed by cryptsetup. 2Mib (4096 blocks) is typical.

Conversely, converting from LUKS frees up such space which will become unused free space at the end of the volume. Any filesystem could be enlarged to fill this space but inCrypt does not do this for you.

Converting between raw and plain requires no additional space.

Implementation Notes

The Linux kernel's dm-crypt module underlies both the plain and LUKS encryption modes. It encrypts 512-byte logical disk blocks and there is a 1:1 mapping between raw and encrypted blocks.

dm-crypt is a device mapper for encryption. It creates a virtual block device that can be used transparently. Any reads/writes to the virtual device are applied to the underlying raw device via a cryptography layer.

There is no difference between how the plain and LUKS modes encrypt data. Both encrypt using a key that is appropriate to the chosen cipher. The difference is that LUKS embeds that key, which it calls the master key, in an encrypted header up to eight times, each encrypted with a different key (these keys are known as LUKS keys).

The LUKS keys are derived from end-user passprhrases using PBKDF2, a password-based key derivation function that uses a salt and applies a hashing algorithm multiple times.

In LUKS-mode, users enter a passphrase to unlock the device and the master key is generally unknown by the user.

Plain-mode users need to supply the master key to unlock the device (as well as details about the encryption cipher). This can be supplied as a key-file or as a passphrase and hashing algorithm (Plain mode does not use a salt or PBKDF2).

To accommodate the header, the data area of a LUKS device is offset from the beginning of the device. Plain-mode devices have no header and the raw and encrypted blocks align.

inCrypt works by creating device mappers and uses dd to copy between them. It also uses dd to move the data payload as required to add or remove a LUKS header. It uses mbuffer to allow overlapping read/write areas on the same device.

To create a LUKS header, inCrypt uses a temporary file and then copies it into place afterwards. If converting from plain mode, the existing key is used as the master key when creating the LUKS header. This allows conversion from plain to LUKS without re-encrypting. It also means that the original passphrase can still be used to access the data as a plain-mode device if it is known.

It opens devices with cryptsetup (which requires passphrase entry) and then uses dmsetup to extract required parameters, such as the encryption key, cipher and offset.

Hashes, Keys and Key Files

dm-crypt uses a symmetric cipher to perform encryption and decryption that is is specified as a tuple comprising its name, key type and operating mode. For example: aes-xts-plain64.

The key type defines the length of the key, the secret token that the cipher uses to encrypt. The key has a predefined length (a number of bits) as required by the cipher and is binary data. It is usually supplied in a key file or derived from a pass-phrase.

xts requires a 512-bit key that is split into two 256-bit keys; That is 512 bits (64 bytes) of data. XTS means XEX-based tweaked-codebook mode with ciphertext stealing.

The operating mode determines how the encryption of one block affects the encryption of subsequent blocks; it can be considered as part of the cipher. plain64 is the appropriate mode for XTS.

Key file interpretation is context sensitive. When a key file is given to cryptsetup to open a LUKS-mode volume, it is treated the same as a passphrase: it is hashed using the same key derivation function.

! if using a key file to store a passphrase, ensure that it doesn't contain a trailing newline unless it is part of the passphrase.

However, when a key file is given to cryptsetup to open a plain-mode volume, the required number of bytes are read from it and used as-is; it is not treated as a passphrase. If the key file contains more bytes than necessary then the remaining bytes are ignored - there is no gain (beyond obfuscation) to be achieved by a key file that is longer than the key length.

cryptsetup allows an offset to be specified and this allows the required bytes to be read from within a larger file instead of from the beginning of the file.

A passphrase is a sequence of bytes that is hashed to prodce a key of the desired length (e.g a 512-bit key can be produced with a SHA-512 hash). Pass-phrases are typically human-readable because they are usually typed in at a prompt but they can also be read from a file, also called a key-file, and can therefore also be binary data.

! the fact that a key or passphrase is human-readable is only of consequence to the human end-user. To cryptsetup, it's just binary data.

! The hash function is only used to generate a key from a passphrase. It is not used for encryption.

Practical Examples

This section presents practical examples to demonstrate how inCrypt works. The examples use bash code extracts from inCrypt and can be used to perform manual conversions in a similar fashion.

To begin, define a helper function that converts a string of hexadecimal characters into a binary byte string. The examples will use this to output keys:

hex2bin() {
  local b=''
  local i=0
  for ((i=0; i<${#1}; i+=2));do b+=\x${1:$i:2};done
  printf "$b"
}

The examples assume some variables:

  • raw_device is set to the file or device
  • cryptsetup_args contains plain-mode arguments: --cipher aes-xts-plain64 --key-size=512 --hash sha512
  • tmp is a reusable tempoary file tmp=$(mktemp)

A test file can be created to try the examples on:

raw_device=testfile
head -c 10M /dev/urandom > "${raw_device}"
mkfs.ext4 "${raw_device}"

To put some content into the test file

mount ${raw_device} /mnt
ls /bin > /mnt/some_data
umount /mnt
Raw to plain

Converting raw to plain is straightforward: create a plain-mode device mapper and copy from the raw device onto it.

cryptsetup open "${raw_device}" incrypt --type plain ${cryptsetup_args}
dd if="${raw_device}" ${dd_skip} | mbuffer | dd conv=notrunc of="/dev/mapper/incrypt"
cryptsetup close incrypt
Raw to LUKS

Converting raw to LUKS is a little more complex because a header must be inserted in front of the data. First create the header:

head -c 2M /dev/urandom > "$tmp"
cryptsetup luksFormat $cryptsetup_args "$tmp"
hdr=$(mktemp -u)
cryptsetup luksHeaderBackup "$tmp" --header-backup-file "$hdr"

Next, get the required payload offset from the header and move the raw data blocks. You can hash the data before and afterwards to confirm that it is intact:

sha1sum "${raw_device}"

Perform the conversion:

offset=$(cryptsetup luksDump $hdr | sed -nr 's/^Payload offset:\s(.*)/\1/p')
dd if="${raw_device}" | mbuffer -s 512 -b ${offset} -P 100 | dd of="${raw_device}" seek="${offset}" conv=notrunc

The arguments to mbuffer ensure that no data is overwritten before it is read. Next, write the header:

dd if="${hdr}" of="${raw_device}" conv=notrunc

Before encryption re-hash the data and compare with the hash made earlier to confirm that it is intact:

dd if="${raw_device}" skip="${offset}" | sha1sum

Now, encrypt the data: open the LUKS device and copy from the raw device into it:

cryptsetup open "${raw_device}" incrypt
dd if="${raw_device}" skip="${offset}" | mbuffer | dd conv=notrunc of=/dev/mapper/incrypt
cryptsetup close incrypt

This use of mbuffer is solely to visualise progress; this is useful when converting large volumes.

Plain to LUKS

Converting from plain mode to LUKS uses a method similar to conversion from raw to LUKS but it differs in two ways: the plain key is inserted into the header and no encryption is performed because the data is already encrypted. Begin by extracting the plain-mode cipher and key:

cryptsetup open "${raw_device}" incrypt --type plain ${cryptsetup_args}
dmsetup table --target crypt --showkey /dev/mapper/incrypt > $tmp
cryptsetup close incrypt
cipher=$(awk '{print $4}' $tmp)
master_key=$(awk '{print $5}' $tmp)
master_key_size=$(( ${#master_key} * 4 ))
master_key_file=$(mktemp)
( hex2bin $master_key ) > "${master_key_file}"

Use that information to create the LUKS header

head -c 2M /dev/urandom > "$tmp"
cryptsetup luksFormat "$tmp" --master-key-file "$master_key_file" --key-size "$master_key_size" --cipher "$cipher"
hdr=$(mktemp -u)
cryptsetup luksHeaderBackup "$tmp" --header-backup-file "$hdr"

Then hash the data before performing the conversion and writing the header:

sha1sum "${raw_device}"
offset=$(cryptsetup luksDump $hdr | sed -nr 's/^Payload offset:\s*(.*)/\1/p')
dd if="${raw_device}" | mbuffer -s 512 -b ${offset} -P 100 | dd of="${raw_device}" seek="${offset}" conv=notrunc
dd if="${hdr}" of="${raw_device}" conv=notrunc

And, finally, cross check the data hash:

dd if="${raw_device}" skip="${offset}" | sha1sum

Plain or LUKS to Raw

Converting to raw from both plain and LUKS volumes is done the same way, by creating an appropriate device mapper and copying its contents onto the raw device:

cryptsetup isLuks "${raw_device}" && cryptsetup open "${raw_device}" incrypt || cryptsetup open "${raw_device}" incrypt --type plain ${cryptsetup_args}
dd if=/dev/mapper/incrypt | mbuffer | dd of="${raw_device}" conv=notrunc
cryptsetup close incrypt

LUKS to Plain

Conversion from LUKS to plain mode can be achieved in two ways. The easiest way is to re-encrypt the data by copying from a LUKS device mapper onto a plain device mapper. The other way is to extract the current master key and move the data payload to overwrite the LUKS header.

re-encrypt data

Re-encrypting allows a new passphrase to be chosen and derives a new encryption key. To re-encrypt from LUKS to plain, create two device mappers and copy between them:

cryptsetup open "${raw_device}" incrypt-luks
cryptsetup open "${raw_device}" incrypt-plain --type plain ${cryptsetup_args}
dd if=/dev/mapper/incrypt-luks  | mbuffer | dd conv=notrunc of=/dev/mapper/incrypt-plain
cryptsetup close incrypt-luks
cryptsetup close incrypt-plain
move data and extract key

This method retains the existing encryption key. Begin by extracting the key and data payload offset and then use them to move the data:

cryptsetup open "${raw_device}" incrypt
dmsetup table --target crypt --showkey /dev/mapper/incrypt > $tmp
cryptsetup close incrypt
dd_skip="skip=$(awk '{print $8}' $tmp)"
master_key=$(awk '{print $5}' $tmp)
( hex2bin $master_key ) > my_key.bin
dd if="${raw_device}" ${dd_skip} | mbuffer | dd conv=notrunc of="${raw_device}"

Miscellanea

To sha1sum a device:

  • whole device (raw or encrypted) sha1sum ${raw_device}"
  • data area of LUKS device dd if="${raw_device}" skip="${offset}" | sha1sum

Mounting a device read/write immediately modifies it and means that its sha1sum will change. This doeesn't happen if it is mounted read-only.

To directly mount LUKS data if the master key passphrase is known:

cryptsetup open "${raw_device}" incrypt --type plain ${cryptsetup_args} --offset "${offset}"

or append --key-file my_key.bin if the master key is in the file my_key.bin

To view the device mapper table of an unlocked device:

dmsetup table --target crypt --showkey /dev/mapper/incrypt
shrinking an ext4 filesystem

Open the LUKS device mapper and get the current size in 512-byte blocks :

cryptsetup open "${raw_device}" incrypt --type plain ${cryptsetup_args}
 blockdev --getsz /dev/mapper/incrypt

Calculate new size in filesystem 1024 byte blocks = (current size - $offset) / 2

 new_size=$(( ($(blockdev --getsz /dev/mapper/incrypt) - $offset)) / 2)

Perform the resize:

e2fsck -f /dev/mapper/incrypt
resize2fs /dev/mapper/incrypt "${new_size}"

Finally:

cryptsetup close incrypt
Some useful commands:
  • Display a binary key file: xxd key-file.bin
  • Check header size in blocks ls -l $hdr
  • mount plaintext: mount -o loop testfile /mnt

Further Reading