PEM_read_RSAPrivateKey returns "Illegal Seek" when decrypted using OpenSSL libs in C

Question

This problem drove me crazy! :) I generate a pair of public / private keys, but when I go to download the private key, I get an error message. I am using C. Note. I use the password "password" on the keys, it is set as #define, but I tried to put it manually as a string without a difference. This is how I generate the keys:

FILE *fp; OpenSSL_add_all_algorithms(); RSA *rsa=NULL; unsigned char seed[KEYSIZE]; int i; //Seed PRNG srand(time(NULL)); RAND_bytes(seed, KEYSIZE - 1); //Generate a key if ((rsa=RSA_generate_key(KEYSIZE,65537,NULL,NULL)) == NULL) ERR_get_error(); //Write the public key fp = fopen(pubkey,"w"); if (!PEM_write_RSA_PUBKEY(fp, rsa)) { printf("Error writing public key\n"); exit(1); } fclose(fp); //Write the private key fp = fopen(privkey,"w"); if (!PEM_write_RSAPrivateKey(fp, rsa, EVP_des_ede3_cbc(), NULL, 0, NULL, PASSWORD)) { printf("Error writing private key\n"); exit(1); } fclose(fp); 

This succeeds and I get the files in the appropriate places and they look good. I have also successfully encrypted / decrypted these files on the command line, so I know that they work!

This is how I upload the private key:

 static void decrypt(int locale) { FILE *key; RSA *rsa; key = fopen(PRIVATEKEY, "r"); if (key == NULL) perror("Error reading private key"); rsa = PEM_read_RSAPrivateKey(key, NULL, NULL, PASSWORD); // THIS BREAKS! if (rsa == NULL) perror("Private Key not valid"); // I always get this error :( if (rsa != NULL) RSA_free(rsa); fclose(key);} 

Any suggestions? I went through the rsa.c source file and this is basically exactly how they do it there, except that they use BIO instead of freads. I do not understand why my method should not work!

Here is the relevant strace output section:

 open("/opt/evoting/keys/priv/mix1.priv", O_RDONLY) = 15 fstat64(15, {st_mode=S_IFREG|0644, st_size=1751, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb773c000 read(15, "-----BEGIN RSA PRIVATE KEY-----\n"..., 4096) = 1751 dup(2) = 16 fcntl64(16, F_GETFL) = 0x2 (flags O_RDWR) fstat64(16, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 2), ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb773b000 _llseek(16, 0, 0xbfba00d0, SEEK_CUR) = -1 ESPIPE (Illegal seek) write(16, "Private Key not valid: Resource "..., 56Private Key not valid: Resource temporarily unavailable ) = 56 close(16) = 0 munmap(0xb773b000, 4096) = 0 close(15) = 0 munmap(0xb773c000, 4096) = 0 

In addition, perror generates a "resource unavailable" error. I'm not sure about that, but I reset my hard and soft ulimits to be really tall, and he did nothing. I copied the function to my own file and compiled it, and it does not generate an error (although it still does not read the key).

Answer 1

Yeah! Well, I just learned a lot about fixing this problem, and hopefully it helps someone else along the line. Most of my information came from this topic: http://readlist.com/lists/openssl.org/openssl-users/2/10340.html

Basically, I didn't need to use perror and use ERR_print_errors_fp(stderr) instead. This gave me the actual error message:

 3077973640:error:0906B072:lib(9):func(107):reason(114):pem_lib.c:530: 

Then I used the command: openssl errstr 0906B072 and got:

 error:0906B072:PEM routines:PEM_get_EVP_CIPHER_INFO:unsupported encryption 

Which basically led me to create my key pair, where I pointed out CBC: EVP_des_ede3_cbc (), which is not supported for RSA. So you go!

Answer 2

I found a solution - you just call the OpenSSL_add_all_ciphers() function before

OpenSSL_add_all_ciphers() adds all encryption algorithms to the table, including password-based encryption algorithms.

Encryption and digest search functions are used in many parts of the library. If the table is not initialized, some functions will behave badly and complain that they cannot find algorithms. This includes the PEM, PKCS # 12, SSL, and S / MIME libraries. This is a common request on the OpenSSL mailing lists.

Calling OpenSSL_add_all_algorithms () links in all algorithms: as a result, the statically linked executable can be quite large. If this is important, you can simply add the necessary ciphers and digests.

http://www.openssl.org/docs/crypto/x509.html#