LUKS with Serpent an SHA512, CBC ESSIV

LUKS with Serpent an SHA512, CBC ESSIV

I am trying to setup my new Gentoo system (I believe this is not relevant,
as this is not Gentoo specific question) with LUKS while trying to use:
cat smth | cryptsetup luksFormat /dev/sda10 -c serpent-cbc-essiv:sha512 -s
512
For some reason it fails while saying:
device-mapper: reload ioctl on failed: Invalid argument
Failed to setup dm-crypt key mapping for device /dev/sda10.
Check that kernel supports serpent-cbc-essiv:sha512 cipher (check syslog
for more info).
Failed to write to key storage.
Also dmesg says:
[ ... ] device-mapper: table: 253:1: crypt: Error decoding and setting key
[ ... ] device-mapper: ioctl: error adding target to table
I did load both sha512, cbc and serpent with modprobe and I see them
loaded by doing lsmod. Also I do see them when I do cat /proc/crypto.
Weird thing is that it works with
cryptsetup luksFormat /dev/sda10 -c serpent-cbc-essiv:sha256 -s 256
Furthermore, it works with sha512 when I pipe no passphrase to LUKS.
I did try to google both stderr messages of LUKS and the dmesg. The only
suggestions I could find is that I was missing the -s parameter, which I
wasn't.
Do any of you have any idea as to how to solve this problem?
Thank you,
Chester