Im is working on tuning performance on a high-performance, high-performance data engine that ultimately serves the end-user web interface. In particular, the part delegated to me revolves around the characteristics of multi-threaded file I / O and mapping data memory to the local cache. When writing test applications to isolate the time poles of time, several questions were opened. The code was minimized to make only an open call to the system file (open(O_RDONLY)) I hope that the result of this query will help us understand the fundamental low-level system processes so that we can understand the complete predictive (or at least relational) time model. Suggestions are always welcome. Weve seems to have hit a time barrier and would like to understand the behavior and determine if this barrier can break.
Testing program:
- It is written in C compiled using the gnu C compiler, as follows;
- Minimum written to isolate detected problems with one system file "open ()";
- Configurable to run the requested number of pthreads at the same time;
- loads a list of 1000 text files ~ 8K in size;
- creates threads (simply) without changing the attributes;
- each thread makes multiple consecutive open () calls in the next available file from a predefined list of files until the file list is exhausted in such a way that one stream should open all 1000 files, 2 threads should theoretically open 500 files (until proved) etc.);
We run tests several times, parametrically changing the number of threads, file sizes, and whether we find files on a local or remote server. There are a few questions.
Observed results (opening deleted files):
- File open time for the first time exceeds (as expected, due to file caching);
- Running a test application with a single thread to download all deleted files takes X seconds;
- , 1 # , (nX ).
- > #CPUs , , , , , #CPUs ( ?).
- (, 25 ) , .
- .
( , ):
- ( );
- , LOW 4-5 , , CPU, ;
- ( ) , ( , № 5 ).
, , Local open 0,01 , 100 1 . , 8x 8 , 9+ . , 8 . , roundtrip, , . , , , 100 . , , 8.
, , , :
- ;
- ,
- ?
- - , 8-10 ?
- "open()" ?
- ;
- open(), / ? ?
- open() , , ;
- , 1000 , /?
- open() , stat() ed? () , () ?
:
Red Hat Enterprise Linux Server release 5.4 (Tikanga)
8-CPUS, each with characteristics as shown below:
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 23
model name : Intel(R) Xeon(R) CPU X5460 @ 3.16GHz
stepping : 6
cpu MHz : 1992.000
cache size : 6144 KB
physical id : 0
siblings : 4
core id : 1
cpu cores : 4
apicid : 1
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall lm constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm
bogomips : 6317.47
clflush size : 64
cache_alignment : 64
address sizes : 38 bits physical, 48 bits virtual
power management:
GNU C compiler, version:
gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-46)