What is the most efficient way to identify a very scarce network matrix in Julia?

Question

I have data for a very large network, which is pretty sparse. I was wondering what would be the most efficient storage method and easy access to the two nodes.

Obviously, with N nodes, preserving the N * N matrix is ​​not so efficient in terms of the space that I save. Therefore, I thought that perhaps keeping the adjacency list as shown below:

Array(Vector{Int64}, N_tmp) 

Where N_tmp <= N, since many nodes may not have any connections.

Could you please help me if there are better ways or maybe packages that are better in terms of memory and access?

Answer 1

In LightGraphs.jl, we use adjacency lists (mostly vector vectors) to store neighbors for each node. This provides very good memory usage for large sparse schedules, which allows us to scale up to hundreds of millions of nodes on commercial equipment, while providing quick access that exceeds the native allowed matrix data structure for most graphics operations.

You might be wondering if LightGraphs will meet your needs directly.

Edit with additional information: we save a sorted list of neighbors - this gives us success when creating an edge, but makes it much faster for later searches.