Why does GCC create a shared object instead of an executable binary according to the file?

Question

I have a library that I create. All of my objects are compiled and linked in series when I run any of the following: ar rcs lib/libryftts.a $^

gcc -shared $^ -o lib/libryftts.so

in my makefile. I can also successfully install them in /usr/local/lib When I test the file with nm, all functions are there. My problem is that when I run gcc testing/test.c -lryftts -o test && file ./test or gcc testing/test.c lib/libryftts.a -o test && file ./test it says:

test: ELF 64-bit LSB shared object instead of test: ELF 64-bit LSB executable , as I expected. What am I doing wrong?

Answer 1

What am I doing wrong?

Nothing.

It looks like your GCC is configured to create -pie by default. These binaries are indeed shared libraries (of type ET_DYN ), except that they run just like a regular executable.

So, you should just run your binary and (if it works) not to worry about it.

Or you could link your binary with gcc -no-pie ... and it should create a non- PIE ET_EXEC type ET_EXEC , for which file would indicate an ELF 64-bit LSB executable .

Answer 2

file 5.36 says it clearly

file 5.36 actually prints it clearly if the executable is PIE or not, as shown at: https://unix.stackexchange.com/questions/89211/how-to-test-whether-a-linux-binary-was-compiled Positionally Independent Code / 435038 # 435038

For example, the PIE executable shows how:

 main.out: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, not stripped 

and not -pie like:

 main.out: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped 

This function was introduced in 5.33, but it performed a simple check on chmod +x . Before that, he simply printed a shared object for PIE.

In 5.34, it was supposed to start checking more specialized ELF metadata DF_1_PIE , but due to an implementation error on commit 9109a696f3289ba00eaa222fd432755ec4287e28, it actually broke things and showed GCC PIE executables as shared objects .

The error was fixed in 5.36 when commit 03084b161cf888b5286dbbcd964c31ccad4f64d9 .

This error is present, in particular, in Ubuntu 18.10, in which there is file 5.34.

It does not show up when associating assembler code with ld -pie due to a match.

For a breakdown of the source code, see the “Analyzing the source file 5.36” section of this answer.

Linux kernel 5.0 determines whether ET_DYN based ASLR can be used

The main reason for file "confusion" is that both PIE executables and shared libraries are position-independent and can be placed in randomized memory locations.

In fs / binfmt_elf.c, the kernel accepts only these two types of ELF files:

 /* First of all, some simple consistency checks */ if (interp_elf_ex->e_type != ET_EXEC && interp_elf_ex->e_type != ET_DYN) goto out; 

Then only for ET_DYN it set load_bias to the value, load_bias from zero. Then load_bias determines the ELF offset: How is the text section address of the PIE executable on Linux determined?

 /* * If we are loading ET_EXEC or we have already performed * the ET_DYN load_addr calculations, proceed normally. */ if (loc->elf_ex.e_type == ET_EXEC || load_addr_set) { elf_flags |= elf_fixed; } else if (loc->elf_ex.e_type == ET_DYN) { /* * This logic is run once for the first LOAD Program * Header for ET_DYN binaries to calculate the * randomization (load_bias) for all the LOAD * Program Headers, and to calculate the entire * size of the ELF mapping (total_size). (Note that * load_addr_set is set to true later once the * initial mapping is performed.) * * There are effectively two types of ET_DYN * binaries: programs (ie PIE: ET_DYN with INTERP) * and loaders (ET_DYN without INTERP, since they * _are_ the ELF interpreter). The loaders must * be loaded away from programs since the program * may otherwise collide with the loader (especially * for ET_EXEC which does not have a randomized * position). For example to handle invocations of * "./ld.so someprog" to test out a new version of * the loader, the subsequent program that the * loader loads must avoid the loader itself, so * they cannot share the same load range. Sufficient * room for the brk must be allocated with the * loader as well, since brk must be available with * the loader. * * Therefore, programs are loaded offset from * ELF_ET_DYN_BASE and loaders are loaded into the * independently randomized mmap region (0 load_bias * without MAP_FIXED). */ if (elf_interpreter) { load_bias = ELF_ET_DYN_BASE; if (current->flags & PF_RANDOMIZE) load_bias += arch_mmap_rnd(); elf_flags |= elf_fixed; } else load_bias = 0; 

I confirm this experimentally at: What is the -fPIE option for position-independent executables in gcc and ld?

breakdown of behavior file 5.36

Having studied how file works from its source. We conclude that:

• if Elf32_Ehdr.e_type == ET_EXEC
  • print executable
• otherwise if Elf32_Ehdr.e_type == ET_DYN
  • if DT_FLAGS_1 dynamic section entry is present
    • if DF_1_PIE set to DT_FLAGS_1 :
      • pie executable
    • yet
      • print shared object
  • yet
    • if the file is executable by a user, group or others
      • pie executable
    • yet
      • print shared object

And here are a few experiments that confirm that:

 Executable generation ELF type DT_FLAGS_1 DF_1_PIE chdmod +x file 5.36 --------------------------- -------- ---------- -------- -------------- -------------- gcc -fpie -pie ET_DYN yyy pie executable gcc -fno-pie -no-pie ET_EXEC nny executable gcc -shared ET_DYN nny pie executable gcc -shared ET_DYN nnn shared object ld ET_EXEC nny executable ld -pie --dynamic-linker ET_DYN yyy pie executable ld -pie --no-dynamic-linker ET_DYN yyy pie executable 

Tested on Ubuntu 18.10, GCC 8.2.0, Binutils 2.31.1.

A complete test example for each type of experiment is described at:

• gcc -pie and gcc -no-pie : What is the -fPIE option for position-independent executables in gcc and ld?

  Keep in mind that -pie is the default starting with Ubuntu 17.10, which is related: are 32-bit absolute addresses no longer allowed on x86-64 Linux?

• gcc -shared (shared .so library): https://github.com/cirosantilli/cpp-cheat/tree/b80ccb4a842db52d719a16d3716b02b684ebbf11/shared_library/basic

• ld Experiments: How to create a statically linked position-independent ELF executable on Linux?

ELF type and DF_1_PIE defined respectively using:

 readelf --file-header main.out | grep Type readelf --dynamic main.out | grep FLAGS_1 

file 5.36 source code analysis

The key file for analysis is magic / Magdir / elf .

This magic format defines file types depending only on byte values ​​in fixed positions.

The format itself is documented at:

 man 5 magic 

Therefore, at the moment, you will need the following documents:

• http://www.sco.com/developers/devspecs/gabi41.pdf ELF standard in the ELF header section
• http://www.cirosantilli.com/elf-hello-world/#elf-header my ELF format introduction and breakdown

At the end of the file we see:

 0 string \177ELF ELF !:strength *2 >4 byte 0 invalid class >4 byte 1 32-bit >4 byte 2 64-bit >5 byte 0 invalid byte order >5 byte 1 LSB >>0 use elf-le >5 byte 2 MSB >>0 use \^elf-le 

\177ELF is 4 magic bytes at the beginning of each ELF file. \177 is octal for 0x7F .

Then, comparing with the Elf32_Ehdr structure from the standard, we see that byte 4 (5th byte, the first after the magic identifier) ​​defines the ELF class:

 e_ident[EI_CLASSELFCLASS] 

and some of its possible meanings:

 ELFCLASS32 1 ELFCLASS64 2 

In the original file we have:

 1 32-bit 2 64-bit 

both 32-bit and 64-bit lines are file which are output to standard output!

So now we are looking for a shared object in this file, and we are led to:

 0 name elf-le >16 leshort 0 no file type, !:mime application/octet-stream >16 leshort 1 relocatable, !:mime application/x-object >16 leshort 2 executable, !:mime application/x-executable >16 leshort 3 ${x?pie executable:shared object}, 

So this elf-le is a kind of identifier that is included in the previous part of the code.

Byte 16 exactly corresponds to the ELF type:

 Elf32_Ehdr.e_type 

and some of its meanings:

 ET_EXEC 2 ET_DYN 3 

Therefore, ET_EXEC always printed as executable .

However, ET_DYN has two possibilities depending on ${x :

• pie executable
• shared object

${x asks: is the executable file or not a user, group or other? If yes, show pie executable , otherwise a shared object .

This extension is executed in the varexpand function in src/softmagic.c :

 static int varexpand(struct magic_set *ms, char *buf, size_t len, const char *str) { [...] case 'x': if (ms->mode & 0111) { ptr = t; l = et - t; } else { ptr = e; l = ee - e; } break; 

There is, however, another hack! In the dodynamic function src/readelf.c dodynamic , if DT_FLAGS_1 dynamic section DT_FLAGS_1 entry ( PT_DYNAMIC ), then permissions in st->mode are canceled by the presence or absence of the DF_1_PIE flag:

 case DT_FLAGS_1: if (xdh_val & DF_1_PIE) ms->mode |= 0111; else ms->mode &= ~0111; break; 

The error in 5.34 is that the source code was written as:

  if (xdh_val == DF_1_PIE) 

this means that if another flag was set, which GCC makes by default due to DF_1_NOW , the executable DF_1_NOW is a shared object .

This flag entry is not described in the ELF standard, so it must be an extension of Binutils.

This flag does not make sense in the Linux kernel 5.0 or glibc 2.27, so it seems to me that it is informative enough to indicate whether the file is a PIE or not.