Why is this simple libpqxx code skipping memory?

Question

I recently wrote an application that accesses Postgres DB via libpqxx, and it does not tolerate memory well. Even this simple test program, based on the example of http://pqxx.org/devprojects/libpqxx/doc/4.0/html/Reference/a00001.html , runs like no tomorrow.

(Edit: I added the commit () and clear () calls in response to the suggestions. The same leak.)

#include <iostream> #include <pqxx/pqxx> #include <string> #include <stdio.h> int main() { try { pqxx::connection c("user=postgres"); int i = 0; while(true) { pqxx::work w(c); pqxx::result r = w.exec("SELECT 1"); w.commit(); i++; if ( i % 1000 == 0 ) printf( "Cycle %d\n", i ); r.clear(); } //while } //try catch (const std::exception &e) { std::cerr << e.what() << std::endl; return 1; } //catch } //main 

After approximately 75,000 loop cycles, the top one shows 206 MB of virtual memory usage, and it continues to rise. I ran a similar test program with 5000 cycles through valgrind and got the following:

 ==1647== 13,732,155 (219,868 direct, 13,512,287 indirect) bytes in 4,997 blocks are definitely lost in loss record 12 of 12 ==1647== at 0x40060D5: operator new(unsigned int) (vg_replace_malloc.c:214) ==1647== by 0x404C0A9: pqxx::result::result(pg_result*, int, std::string const&, int) (in /usr/lib/libpqxx-4.0.so) ==1647== by 0x40309EF: pqxx::connection_base::make_result(pg_result*, std::string const&) (in /usr/lib/libpqxx-4.0.so) ==1647== by 0x4036D65: ??? (in /usr/lib/libpqxx-4.0.so) ==1647== by 0x405EFD6: pqxx::transaction_base::DirectExec(char const*, int) (in /usr/lib/libpqxx-4.0.so) ==1647== by 0x40416EA: pqxx::dbtransaction::do_exec(char const*) (in /usr/lib/libpqxx-4.0.so) ==1647== by 0x40618FA: pqxx::transaction_base::exec(std::string const&, std::string const&) (in /usr/lib/libpqxx-4.0.so) ==1647== by 0x80498F8: main (dbtest.cpp:21) 

Any idea what is going on? It is very difficult to accept that a widely used library like libpqxx will have such a serious error, so what can I do wrong here?

Configuration Details:

• OS: Linux 2.6.18-238.el5
• gcc version 4.4.0
• libpqxx 4.0
• postgres 9.2

( Final edit: it was easier for me to replace libpqxx with libpq than to continue investigating a memory leak. )

Answer 1

I could not reproduce your problem.

Neither with libpqxx-2.6.9, nor with libpqxx-4.0.1 (currently the current stable version).

• Debian Linux 2.6.32
• gcc version 4.4.5
• libpq5 8.4.17
• libpqxx 4.0.1
• libstdc ++ 6.4.4
• postgres 8.4

Memory usage is permanent.

$ valgrind --tool = memcheck run.sh
[...]
Cycle 35000
36000 cycle
^ C == 18420 ==
== 18420 == HEAP SUMMARY:
== 18420 == used on exit: 0 bytes in 0 blocks
== 18420 == total heap usage: 0 allocs, 0 frees, 0 bytes allocated
== 18420 ==
== 18420 == All heap blocks were freed - no leaks possible

I would check if you have a problem with the various library settings that you use. For example, you refer to libpqxx-4.0, but later, when you run the program, you use a different version.

$ ldd client
linux-gate.so.1 => (0xb7776000)
libpqxx-4.0.so => ​​not found
libstdc ++. so.6 => /usr/lib/libstdc++.so.6 (0xb7669000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb764a000)
libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7503000) libpthread.so.0 => /lib/i686/cmov/libpthread.so.0 (0xb74ea000)
libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb74c4000)
/lib/ld-linux.so.2 (0xb7777000)

As I downloaded and installed libpqxx-4.0.1 manually, I must explicitly set LD_LIBRARY_PATH since I did not set it below /usr/local/ . If different versions are installed in /usr/local a cleanup must be performed before this.

In your case (using ldd ) libpqxx should point to the version you compiled and installed. Make sure that only one libpqxx is installed on your system.

I used this Makefile

CPPFLAGS + = -I / home / dev / data / src / cpp / libpqxx-4.0.1 / install / include
LDFLAGS + = -L / home / dev / data / src / cpp / libpqxx-4.0.1 / install / lib -lpqxx -lstd ++

client: client.o

and libpqxx was created using

./configure --prefix = / home / dev / data / src / cpp / libpqxx-4.0.1 / install --enable-shared
to do
make install

and I launched the program using

LD_LIBRARY_PATH = $ LD_LIBRARY_PATH: /home/dev/data/src/cpp/libpqxx-4.0.1/install/lib client

Answer 2

Probably pqxx :: result is a memory leak. Try adding:

 r.clear(); 

It will at least reduce leakage.

 #include <iostream> #include <pqxx/pqxx> #include <string> #include <stdio.h> int main() { try { pqxx::connection c("user=postgres"); int i = 0; while(true) { pqxx::work w(c); pqxx::result r = w.exec("SELECT 1"); i++; if ( i % 1000 == 0 ) printf( "Cycle %d\n", i ); r.clear(); } //while } //try catch (const std::exception &e) { std::cerr << e.what() << std::endl; return 1; } //catch } //main 

Answer 3

Try declaring pqxx :: result r whitout while. And call it inside how you do it. Because for each while loop, you create a pqxx :: result object. Thus, you will create only one object for the entire cycle to charge and clean in each cycle:

 #include <iostream> #include <pqxx/pqxx> #include <string> #include <stdio.h> int main() { try { pqxx::connection c("user=postgres"); int i = 0; pqxx::result r; while(true) { pqxx::work w(c); r = w.exec("SELECT 1"); w.commit(); i++; if ( i % 1000 == 0 ) printf( "Cycle %d\n", i ); r.clear(); } //while } //try catch (const std::exception &e) { std::cerr << e.what() << std::endl; return 1; } //catch } //main 

Or even better, create a method to call in every time loop, so when each method call completes, the temporary object will disappear.

 main() { ... while() { ... method_to_call_pqxx(); } }