Using SHA1 and RSA with java.security.Signature vs. MessageDigest and Cipher
JavaEncryptionCryptographyRsaDigital SignatureJava Problem Overview
I'm trying to understand what the Java java.security.Signature class does. If I compute an SHA1 message digest, and then encrypt that digest using RSA, I get a different result to asking the Signature class to sign the same thing:
// Generate new key
KeyPair keyPair = KeyPairGenerator.getInstance("RSA").generateKeyPair();
PrivateKey privateKey = keyPair.getPrivate();
String plaintext = "This is the message being signed";
// Compute signature
Signature instance = Signature.getInstance("SHA1withRSA");
instance.initSign(privateKey);
instance.update((plaintext).getBytes());
byte[] signature = instance.sign();
// Compute digest
MessageDigest sha1 = MessageDigest.getInstance("SHA1");
byte[] digest = sha1.digest((plaintext).getBytes());
// Encrypt digest
Cipher cipher = Cipher.getInstance("RSA");
cipher.init(Cipher.ENCRYPT_MODE, privateKey);
byte[] cipherText = cipher.doFinal(digest);
// Display results
System.out.println("Input data: " + plaintext);
System.out.println("Digest: " + bytes2String(digest));
System.out.println("Cipher text: " + bytes2String(cipherText));
System.out.println("Signature: " + bytes2String(signature));
Results in (for example):
>Input data: This is the message being signed
>Digest: 62b0a9ef15461c82766fb5bdaae9edbe4ac2e067
>Cipher text: 057dc0d2f7f54acc95d3cf5cba9f944619394711003bdd12...
>Signature: 7177c74bbbb871cc0af92e30d2808ebae146f25d3fd8ba1622...
I must have a fundamental misunderstanding of what Signature is doing - I've traced through it, and it appears to be calling update on a MessageDigest object, with the algorithm set to SHA1 as I would expect, then getting the digest, then doing the encryption. What's making the results differ?
EDIT:
Leonidas made me check whether the signature scheme is supposed to do what I think it does. There are two types of signature defined in the RFC:
The first of these (PKCS1) is the one I describe above. It uses a hash function to create a digest, and then encrypts the result with a private key.
The second algorithm uses a random salt value, and is more secure but non-deterministic. The signature produced from the code above does not change if the same key is used repeatedly, so I don't think it can be PSS.
EDIT:
Here's the bytes2string method I was using:
private static String bytes2String(byte[] bytes) {
StringBuilder string = new StringBuilder();
for (byte b : bytes) {
String hexString = Integer.toHexString(0x00FF & b);
string.append(hexString.length() == 1 ? "0" + hexString : hexString);
}
return string.toString();
}
Java Solutions
Solution 1 - Java
OK, I've worked out what's going on. Leonidas is right, it's not just the hash that gets encrypted (in the case of the Cipher class method), it's the ID of the hash algorithm concatenated with the digest:
DigestInfo ::= SEQUENCE {
digestAlgorithm AlgorithmIdentifier,
digest OCTET STRING
}
Which is why the encryption by the Cipher and Signature are different.
Solution 2 - Java
To produce the same results:
MessageDigest sha1 = MessageDigest.getInstance("SHA1", BOUNCY_CASTLE_PROVIDER);
byte[] digest = sha1.digest(content);
DERObjectIdentifier sha1oid_ = new DERObjectIdentifier("1.3.14.3.2.26");
AlgorithmIdentifier sha1aid_ = new AlgorithmIdentifier(sha1oid_, null);
DigestInfo di = new DigestInfo(sha1aid_, digest);
byte[] plainSig = di.getDEREncoded();
Cipher cipher = Cipher.getInstance("RSA/ECB/PKCS1Padding", BOUNCY_CASTLE_PROVIDER);
cipher.init(Cipher.ENCRYPT_MODE, privateKey);
byte[] signature = cipher.doFinal(plainSig);
Solution 3 - Java
A slightly more efficient version of the bytes2String method is
private static final char[] hex = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'};
private static String byteArray2Hex(byte[] bytes) {
StringBuilder sb = new StringBuilder(bytes.length * 2);
for (final byte b : bytes) {
sb.append(hex[(b & 0xF0) >> 4]);
sb.append(hex[b & 0x0F]);
}
return sb.toString();
}
Solution 4 - Java
Erm, after understanding your question: are you sure that the signature-method only creates a SHA1 and encrypts it? GPG et al offer to compress/clear sign the data. Maybe this java-signature-alg also creates a detachable/attachable signature.
Solution 5 - Java
Taking @Mike Houston's answer as pointer, here is a complete sample code that does Signature and Hash and encryption.
/**
* @param args
*/
public static void main(String[] args)
{
try
{
boolean useBouncyCastleProvider = false;
Provider provider = null;
if (useBouncyCastleProvider)
{
provider = new BouncyCastleProvider();
Security.addProvider(provider);
}
String plainText = "This is a plain text!!";
// KeyPair
KeyPairGenerator keyPairGenerator = null;
if (null != provider)
keyPairGenerator = KeyPairGenerator.getInstance("RSA", provider);
else
keyPairGenerator = KeyPairGenerator.getInstance("RSA");
keyPairGenerator.initialize(2048);
KeyPair keyPair = keyPairGenerator.generateKeyPair();
// Signature
Signature signatureProvider = null;
if (null != provider)
signatureProvider = Signature.getInstance("SHA256WithRSA", provider);
else
signatureProvider = Signature.getInstance("SHA256WithRSA");
signatureProvider.initSign(keyPair.getPrivate());
signatureProvider.update(plainText.getBytes());
byte[] signature = signatureProvider.sign();
System.out.println("Signature Output : ");
System.out.println("\t" + new String(Base64.encode(signature)));
// Message Digest
String hashingAlgorithm = "SHA-256";
MessageDigest messageDigestProvider = null;
if (null != provider)
messageDigestProvider = MessageDigest.getInstance(hashingAlgorithm, provider);
else
messageDigestProvider = MessageDigest.getInstance(hashingAlgorithm);
messageDigestProvider.update(plainText.getBytes());
byte[] hash = messageDigestProvider.digest();
DigestAlgorithmIdentifierFinder hashAlgorithmFinder = new DefaultDigestAlgorithmIdentifierFinder();
AlgorithmIdentifier hashingAlgorithmIdentifier = hashAlgorithmFinder.find(hashingAlgorithm);
DigestInfo digestInfo = new DigestInfo(hashingAlgorithmIdentifier, hash);
byte[] hashToEncrypt = digestInfo.getEncoded();
// Crypto
// You could also use "RSA/ECB/PKCS1Padding" for both the BC and SUN Providers.
Cipher encCipher = null;
if (null != provider)
encCipher = Cipher.getInstance("RSA/NONE/PKCS1Padding", provider);
else
encCipher = Cipher.getInstance("RSA");
encCipher.init(Cipher.ENCRYPT_MODE, keyPair.getPrivate());
byte[] encrypted = encCipher.doFinal(hashToEncrypt);
System.out.println("Hash and Encryption Output : ");
System.out.println("\t" + new String(Base64.encode(encrypted)));
}
catch (Throwable e)
{
e.printStackTrace();
}
}
You can use BouncyCastle Provider or default Sun Provider.
Solution 6 - Java
I have a similar problem, I tested adding code and found some interesting results. With this code I add, I can deduce that depending on the "provider" to use, the firm can be different? (because the data included in the encryption is not always equal in all providers).
Results of my test.
Conclusion.- Signature Decipher= ???(trash) + DigestInfo (if we know the value of "trash", the digital signatures will be equal)
IDE Eclipse OUTPUT...
Input data: This is the message being signed
Digest: 62b0a9ef15461c82766fb5bdaae9edbe4ac2e067
DigestInfo: 3021300906052b0e03021a0500041462b0a9ef15461c82766fb5bdaae9edbe4ac2e067
Signature Decipher: 1ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff003021300906052b0e03021a0500041462b0a9ef15461c82766fb5bdaae9edbe4ac2e067
CODE
import java.security.InvalidKeyException;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.NoSuchProviderException;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.Signature;
import java.security.SignatureException;
import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.NoSuchPaddingException;
import org.bouncycastle.asn1.x509.DigestInfo;
import org.bouncycastle.asn1.DERObjectIdentifier;
import org.bouncycastle.asn1.x509.AlgorithmIdentifier;
public class prueba {
/**
* @param args
* @throws NoSuchProviderException
* @throws NoSuchAlgorithmException
* @throws InvalidKeyException
* @throws SignatureException
* @throws NoSuchPaddingException
* @throws BadPaddingException
* @throws IllegalBlockSizeException
*///
public static void main(String[] args) throws NoSuchAlgorithmException, NoSuchProviderException, InvalidKeyException, SignatureException, NoSuchPaddingException, IllegalBlockSizeException, BadPaddingException {
// TODO Auto-generated method stub
KeyPair keyPair = KeyPairGenerator.getInstance("RSA","BC").generateKeyPair();
PrivateKey privateKey = keyPair.getPrivate();
PublicKey puKey = keyPair.getPublic();
String plaintext = "This is the message being signed";
// Hacer la firma
Signature instance = Signature.getInstance("SHA1withRSA","BC");
instance.initSign(privateKey);
instance.update((plaintext).getBytes());
byte[] signature = instance.sign();
// En dos partes primero hago un Hash
MessageDigest digest = MessageDigest.getInstance("SHA1", "BC");
byte[] hash = digest.digest((plaintext).getBytes());
// El digest es identico a openssl dgst -sha1 texto.txt
//MessageDigest sha1 = MessageDigest.getInstance("SHA1","BC");
//byte[] digest = sha1.digest((plaintext).getBytes());
AlgorithmIdentifier digestAlgorithm = new AlgorithmIdentifier(new
DERObjectIdentifier("1.3.14.3.2.26"), null);
// create the digest info
DigestInfo di = new DigestInfo(digestAlgorithm, hash);
byte[] digestInfo = di.getDEREncoded();
//Luego cifro el hash
Cipher cipher = Cipher.getInstance("RSA","BC");
cipher.init(Cipher.ENCRYPT_MODE, privateKey);
byte[] cipherText = cipher.doFinal(digestInfo);
//byte[] cipherText = cipher.doFinal(digest2);
Cipher cipher2 = Cipher.getInstance("RSA","BC");
cipher2.init(Cipher.DECRYPT_MODE, puKey);
byte[] cipherText2 = cipher2.doFinal(signature);
System.out.println("Input data: " + plaintext);
System.out.println("Digest: " + bytes2String(hash));
System.out.println("Signature: " + bytes2String(signature));
System.out.println("Signature2: " + bytes2String(cipherText));
System.out.println("DigestInfo: " + bytes2String(digestInfo));
System.out.println("Signature Decipher: " + bytes2String(cipherText2));
}
Solution 7 - Java
Code below (taken from my blog article - http://todayguesswhat.blogspot.com/2021/01/manually-verifying-rsa-sha-signature-in.html ) is hopefully helpful in understanding what is present in a standard SHA with RSA signature. This should work in standard Oracle JDK and does not require Bouncy Castle libraries. It is using the sun.security classes to process the decrypted signature contents - you could just as easily manually parse.
In the example below, the message digest algorithm is SHA-512 which produces a 64 byte (512-bit) checksum.
SHA-1 would be pretty similar - but producing a 20-byte (160-bit) checksum.
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.MessageDigest;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.security.Signature;
import java.util.Arrays;
import javax.crypto.Cipher;
import sun.security.util.DerInputStream;
import sun.security.util.DerValue;
public class RSASignatureVerification
{
public static void main(String[] args) throws Exception
{
KeyPairGenerator generator = KeyPairGenerator.getInstance("RSA");
generator.initialize(2048);
KeyPair keyPair = generator.generateKeyPair();
PrivateKey privateKey = keyPair.getPrivate();
PublicKey publicKey = keyPair.getPublic();
String data = "hello oracle";
byte[] dataBytes = data.getBytes("UTF8");
Signature signer = Signature.getInstance("SHA512withRSA");
signer.initSign(privateKey);
signer.update(dataBytes);
byte[] signature = signer.sign(); // signature bytes of the signing operation's result.
Signature verifier = Signature.getInstance("SHA512withRSA");
verifier.initVerify(publicKey);
verifier.update(dataBytes);
boolean verified = verifier.verify(signature);
if (verified)
{
System.out.println("Signature verified!");
}
/*
The statement that describes signing to be equivalent to RSA encrypting the
hash of the message using the private key is a greatly simplified view
The decrypted signatures bytes likely convey a structure (ASN.1) encoded
using DER with the hash just one component of the structure.
*/
// lets try decrypt signature and see what is in it ...
Cipher cipher = Cipher.getInstance("RSA");
cipher.init(Cipher.DECRYPT_MODE, publicKey);
byte[] decryptedSignatureBytes = cipher.doFinal(signature);
/*
sample value of decrypted signature which was 83 bytes long
30 51 30 0D 06 09 60 86 48 01 65 03 04 02 03 05
00 04 40 51 00 41 75 CA 3B 2B 6B C0 0A 3F 99 E3
6B 7A 01 DC F2 9B 36 E6 0D D4 31 89 53 A3 D9 80
6D AE DD 45 7E 55 45 01 FC C8 73 D2 DD 8D E5 B9
E0 71 57 13 41 D0 CD FF CA 58 01 03 A3 DD 95 A1
C1 EE C8
Taking above sample bytes ...
0x30 means A SEQUENCE - which contains an ordered field of one or more types.
It is encoded into a TLV triplet that begins with a Tag byte of 0x30.
DER uses T,L,V (tag bytes, length bytes, value bytes) format
0x51 is the length = 81 decimal (13 bytes)
the 0x30 (48 decimal) that follows begins a second sequence
https://tools.ietf.org/html/rfc3447#page-43
the DER encoding T of the DigestInfo value is equal to the following for SHA-512
0D 06 09 60 86 48 01 65 03 04 02 03 05 00 04 40 || H
where || is concatenation and H is the hash value.
0x0D is the length = 13 decimal (13 bytes)
0x06 means an OBJECT_ID tag
0x09 means the object id is 9 bytes ...
https://docs.microsoft.com/en-au/windows/win32/seccertenroll/about-object-identifier?redirectedfrom=MSDN
taking 2.16.840.1.101.3.4.2.3 (object id for SHA512 Hash Algorithm)
The first two nodes of the OID are encoded onto a single byte.
The first node is multiplied by the decimal 40 and the result is added to the value of the second node
2 * 40 + 16 = 96 decimal = 60 hex
Node values less than or equal to 127 are encoded on one byte.
1 101 3 4 2 3 corresponds to in hex 01 65 03 04 02 03
Node values greater than or equal to 128 are encoded on multiple bytes.
Bit 7 of the leftmost byte is set to one. Bits 0 through 6 of each byte contains the encoded value.
840 decimal = 348 hex
-> 0000 0011 0100 1000
set bit 7 of the left most byte to 1, ignore bit 7 of the right most byte,
shifting right nibble of leftmost byte to the left by 1 bit
-> 1000 0110 X100 1000 in hex 86 48
05 00 ; NULL (0 Bytes)
04 40 ; OCTET STRING (0x40 Bytes = 64 bytes
SHA512 produces a 512-bit (64-byte) hash value
51 00 41 ... C1 EE C8 is the 64 byte hash value
*/
// parse DER encoded data
DerInputStream derReader = new DerInputStream(decryptedSignatureBytes);
byte[] hashValueFromSignature = null;
// obtain sequence of entities
DerValue[] seq = derReader.getSequence(0);
for (DerValue v : seq)
{
if (v.getTag() == 4)
{
hashValueFromSignature = v.getOctetString(); // SHA-512 checksum extracted from decrypted signature bytes
}
}
MessageDigest md = MessageDigest.getInstance("SHA-512");
md.update(dataBytes);
byte[] hashValueCalculated = md.digest();
boolean manuallyVerified = Arrays.equals(hashValueFromSignature, hashValueCalculated);
if (manuallyVerified)
{
System.out.println("Signature manually verified!");
}
else
{
System.out.println("Signature could NOT be manually verified!");
}
}
}