What causes a Python segmentation error?

Question

I am implementing the Kosaraju Strong Connected Component (SCC) graphical search algorithm in Python.

The program works fine on a small data set, but when I run it on a super-large graph (more than 800,000 nodes), it says “Segmentation Error”.

What could be the reason for this? Thank!

---

Additional information: First I got this error when working with a super-large data set:

"RuntimeError: maximum recursion depth exceeded in cmp" 

Then I reset the recursion limit using

 sys.setrecursionlimit(50000) 

but got a "segmentation error"

Believe me, this is not an infinite loop; it works correctly with respect to relatively small data. Perhaps the program has run out of resources?

Answer 1

This happens when the python extension (written in C) tries to access memory out of reach.

You can trace it as follows.

• Add sys.settrace to the very first line of code.

• Use gdb as described by Mark in this answer . At the command line

   gdb python (gdb) run /path/to/script.py ## wait for segfault ## (gdb) backtrace ## stack trace of the c code 

Answer 2

I understand that you have solved your problem, but for others reading this thread, here is the answer: you need to increase the stack that your operating system allocates for the python process.

The way to do this depends on the operating system. In linux, you can check the current value with the ulimit -s command, and you can increase it with ulimit -s <new_value>

Try doubling the previous value and continuing doubling if it doesn't work until you find one that does or is out of memory.

Answer 3

Segmentation error is common, there are many possible reasons for this:

• Little memory
• Bad RAM memory
• Retrieving a huge data set from a database using a query (if the size of the extracted data exceeds the size of the swap mem)
• wrong request / error code
• with a long loop (multiple recursion)

Answer 4

Updating ulimit for my SCC implementation in Kosaraju by fixing segfault for both Python (Python segfault .. who knew!) And C ++ implementations.

For my MAC, I found out the maximum possible through:

 $ ulimit -s -H 65532