C #: OAEP decoding error adds strange problem

Question

I'm currently working on a class that encrypts large amounts of text with a randomly generated encryption key, encrypted with an X509 certificate, using a smart card, using RSACryptoServiceProvider to perform encryption and decryption of the master key. However, when I have the add-on option fOEAP set to true, every time I decrypt I get the error "OAEP decoding error" during decryption. I checked the key size and it is within acceptable limits. And I went through the breakpoints to make sure that the Base64 line that is returned from the encryption function is exactly the same as the encrypted Base64 line that returns to the decryption function when the file is downloaded again.

The key pair is definitely correct, as it works fine without OAEP. And I also checked the text encoding.

EDIT: Turns out it could be a smart card issue, when I tried decryption with a local X509 certificate, decryption succeeded.

EDIT: this is a decryption code that does not work:

string TestString = "Hello World!"; X509Certificate2 cert = DRXEncrypter.GetCertificate("Select a test certificate", "Select a certificate to use for this test from the local store."); string key = DRXEncrypter.GenerateEncryptionKey(214); Console.WriteLine("Encryption Key: " + key); string encrypted = DRXEncrypter.EncryptBody(TestString, key); Console.WriteLine("Encrypted Body: " + encrypted); string cryptokey = DRXEncrypter.EncryptWithCert(cert, key); Console.WriteLine("Encrypted Decryption Key: " + cryptokey); string decrypted = DRXEncrypter.DecryptBody(encrypted, cryptokey, cert); Console.WriteLine("Decrypted Body: " + decrypted); Console.WriteLine("Output String: " + decrypted + "."); 

Here is the code from the cryptography provider class I wrote. I’m stuck with this issue for hours, so it would be great if someone could help me.

 using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using System.Security.Cryptography; using System.Security.Cryptography.X509Certificates; using System.IO; namespace CoreDRXEditor { public class DRXEncrypter { private byte[] Salt = Encoding.ASCII.GetBytes("81PO9j8I1a94j"); private string EncryptionKey; private const bool UseOAEP = true; public DRXEncrypter(string EncryptionKey) { this.EncryptionKey = EncryptionKey; } public static string EncryptBody(string body, string encryptionkey) { // Use the plaintext master key to encrypt the body. DRXEncrypter enc = new DRXEncrypter(encryptionkey); // Encrypt the body. return enc.Encrypt(body); } public static int GetMaxKeySize(X509Certificate2 cert) { RSACryptoServiceProvider csp = cert.PublicKey.Key as RSACryptoServiceProvider; return csp.KeySize; } public static string DecryptBody(string body, string encryptionkey, X509Certificate2 cert) { // Decrypt the encrypted encryption key with the certificate. string DecryptedKey = Convert.ToBase64String(DecryptWithCert(cert, encryptionkey)); // Create a new DRXEncrypter using the decrypted encryption key to decrypt the body. DRXEncrypter enc = new DRXEncrypter(DecryptedKey); // Return the decrypted body. return enc.Decrypt(body); } public static string GenerateEncryptionKey(int KeyLength) { using (RandomNumberGenerator rng = new RNGCryptoServiceProvider()) { byte[] CryptoBytes = new byte[KeyLength]; rng.GetBytes(CryptoBytes); return Convert.ToBase64String(CryptoBytes); } } public static X509Certificate2 GetCertificate(string title, string message) { X509Store cstore = new X509Store(StoreLocation.CurrentUser); cstore.Open(OpenFlags.ReadOnly); X509CertificateCollection certs = X509Certificate2UI.SelectFromCollection(cstore.Certificates, title, message, X509SelectionFlag.SingleSelection); if (certs.Count == 1) { X509Certificate2 mcert = certs[0] as X509Certificate2; return mcert; } else { return null; } } public static string EncryptWithCert(X509Certificate2 cert, string PlainText) { RSACryptoServiceProvider csp = cert.PublicKey.Key as RSACryptoServiceProvider; byte[] PlainBytes = Convert.FromBase64String(PlainText); // This converts the plain text into a byte array and then encrypts the raw bytes. byte[] CryptoBytes = csp.Encrypt(PlainBytes, UseOAEP); // This converts the encrypted bytes into a Base64 string. string ReturnString = Convert.ToBase64String(CryptoBytes); return ReturnString; } public static byte[] DecryptWithCert(X509Certificate2 cert, string EncryptedText) { RSACryptoServiceProvider csp = cert.PrivateKey as RSACryptoServiceProvider; //CspParameters csps = new CspParameters(); byte[] EncryptedBytes = Convert.FromBase64String(EncryptedText); // This converts the encrypted, Base64 encoded byte array from EncryptWithCert() to a byte[] and decrypts it. byte[] CryptoBytes = csp.Decrypt(EncryptedBytes, UseOAEP); return CryptoBytes; } public string Encrypt(string PlainText) { RijndaelManaged Algorithm = null; string Output = null; try { Rfc2898DeriveBytes PrivateKey = new Rfc2898DeriveBytes(this.EncryptionKey, this.Salt); Algorithm = new RijndaelManaged(); Algorithm.Key = PrivateKey.GetBytes(Algorithm.KeySize / 8); Algorithm.Padding = PaddingMode.PKCS7; ICryptoTransform Encryption = Algorithm.CreateEncryptor(Algorithm.Key, Algorithm.IV); using (MemoryStream msa = new MemoryStream()) { msa.Write(BitConverter.GetBytes(Algorithm.IV.Length), 0, sizeof(int)); msa.Write(Algorithm.IV, 0, Algorithm.IV.Length); using (CryptoStream csa = new CryptoStream(msa, Encryption, CryptoStreamMode.Write)) { using (StreamWriter swa = new StreamWriter(csa)) { swa.Write(PlainText); } } Output = Convert.ToBase64String(msa.ToArray()); } } finally { if (Algorithm != null) { Algorithm.Clear(); } } return Output; } public string Decrypt(string EncryptedText) { RijndaelManaged Algorithm = null; string Output = null; try { Rfc2898DeriveBytes PrivateKey = new Rfc2898DeriveBytes(this.EncryptionKey, this.Salt); byte[] KeyBytes = Convert.FromBase64String(EncryptedText); using (MemoryStream msb = new MemoryStream(KeyBytes)) { Algorithm = new RijndaelManaged(); Algorithm.Key = PrivateKey.GetBytes(Algorithm.KeySize / 8); Algorithm.IV = ReadByteArray(msb); Algorithm.Padding = PaddingMode.PKCS7; ICryptoTransform Decryption = Algorithm.CreateDecryptor(Algorithm.Key, Algorithm.IV); using (CryptoStream csb = new CryptoStream(msb, Decryption, CryptoStreamMode.Read)) { using (StreamReader srb = new StreamReader(csb)) { Output = srb.ReadToEnd(); } } } } finally { if (Algorithm != null) { Algorithm.Clear(); } } return Output; } public static string Sha512(string ToHash) { using (SHA512 SHA = new SHA512Managed()) { byte[] HashByte = Encoding.UTF8.GetBytes(ToHash); byte[] HashBytes = SHA.ComputeHash(HashByte); string Hash = System.Text.Encoding.UTF8.GetString(HashBytes, 0, HashBytes.Length); return Hash; } } public static string Base64Encode(string data) { byte[] str = Encoding.UTF8.GetBytes(data); return Convert.ToBase64String(str); } public static string Base64Decode(string data) { byte[] str = Convert.FromBase64String(data); return Encoding.UTF8.GetString(str); } private byte[] ReadByteArray(Stream st) { byte[] Length = new byte[sizeof(int)]; st.Read(Length, 0, Length.Length); byte[] Buffer = new byte[BitConverter.ToInt32(Length, 0)]; st.Read(Buffer, 0, Buffer.Length); return Buffer; } } } 

Answer 1

I argued with this today using smart cards (or, more precisely, Yubikey Neo with the included PIV smart card applet); using this code:

 var encryptor = (RSACryptoServiceProvider)c.PublicKey.Key; var decryptor = (RSACryptoServiceProvider)c.PrivateKey; var encrypt = encryptor.Encrypt(bytes, RSAEncryptionPadding.Pkcs1); var decrypt = decryptor.Decrypt(encrypt, RSAEncryptionPadding.Pkcs1); 

I found that what I use to populate matters. If I use PKCS1 padding, everything works. If I use OaepSHA1, I get an Error while decoding [...] error. If I use something else (e.g. OaepSHA256), I get a Not supported error.

I can only conclude that my smart card does not support OAEP SHA1, but padding with PKCS # 1 is fine.

Even if this answers what you already know, it can be useful as a different date for any other using smart cards.

Answer 2

Make sure the key size is not too small or too large.

See comments from MSDN

RSACryptoServiceProvider supports key sizes from 384 bits to 16384 bits in 8-bit increments if you have a Microsoft Enhanced Cryptography Provider installed. It supports key sizes from 384 bits to 512 bits in 8-bit increments, if you have a Microsoft Base cryptography provider installed.

Thus, you may need to fill in short lines of keys with a few bytes to get the minimum key length

Answer 3

Well, I was able to verify this and from what I see, I have problems with some certificates. I am not sure why some certificates work and others do not. It would be nice to know why some certificates fail in this case?

In any case, I created a new self-signed certificate using the "Manage File Encryption Certificates" windows and used that certificate, and everything seems to work.

The output from your code.

 Encryption Key: aUc/GXWDoh2LktaEGeCJfju1dHP118yD/fzfT0iJLuhOq2QeyGpG6m3aBHaxvdH0ufeXRHbMjmlmPgIL/bhABzkT2C5Oa6ZhY3IFXb5t7JXZ3AtUunvtNAnRyFJ7MzklrSZGgQ vF67DSNfIVE17doKt6j6mkCpSco56ooZCrOs2Mp3vSXqNjvjiwMEfQbk41aYUNVNVNlBGhdNQCIZIAKezQCUpWqzn2II27FIDfqDIEW4ieyzpXC05GzUlGXDxFOiFUPk3n0Y94vgeF8AlCD74eyZtz WQ== Encrypted Body: EAAAANS/W7+GGRbT1q5NCYvZlDZYtxaA8g55HzUqP5qxhenn Encrypted Decryption Key: vc/tcsApmY1503BFi7oSu/RDvZivA1Ed58KJuLoEC6eE8q0BIa6ye2JvtXyxkVbzzL0MA51pZ2ZhMIsfCnBsEDjCgy+JLTZTGM1Mv+em9frFUKb0zHbICnPUa/3H yd1yOWsdn5ws19QN2dzC6eau+ExhT2T/vyZO4Nf9NdHKnB8n2yB1rrQ/T+N2EYCNH/AVPDAsme6JG7k9Od2XIipBXMyCgXgWYZmQusq+JQjA9d3c4CrQYcg/ERF+K3oZv/gPicBkAR5taxwSxAajGg bpkJNsbhTMHTN9bOn333qZ6ojlo5e882baZXuZWPr9qtj1b7ONoOyuSx/OvGKjt93BQg== Decrypted Body: Hello World! Output String: Hello World!. 

Hope that helps