How to increase memory for processing large large Lua tables

Question

I have a Lua function that, given n, generates all permutations of the series from 1 to n and saves each unique series as a table in the container table.

The size of this generated table grows very quickly (and certainly so). Around the time I try to execute n = 11, the script will run for a few seconds before coming to "lua: not enough memory". I have 16 gigabytes of physical memory, but viewing the performance monitor in the Windows task manager allows me to see how ram will be consumed at runtime, and before the script appears, it ends up to about 20% with a memory error.

I found this post that looks the way I need it: process memory in Lua

Since I run my script with Lua.exe, I assume that I will limit myself to how much memory Windows allocates for Lua.exe. Can I increase this amount? Can I use a C # wrapper just to run a Lua script (the idea is that it will have a higher / less limited memory allocation)? Or am I looking in the wrong direction?

---

Answer 1

Do you need to save all permutations in advance? You can generate them on the fly instead.

Example:

local function genPerm(self, i) local result = {} local f = 1 for j = 1, self.n do f = f * j table.insert(result, j) end for j = 1, self.n-1 do f = f / (self.n + 1 - j) local k = math.floor((i - 1) / f) table.insert(result, j, table.remove(result, j+k)) i = i - k * f end return result end local function perms(n) return setmetatable({n=n}, {__index=genPerm}) end local generator = perms(11) for _, i in ipairs {1, 42, 1000000, 39916800} do print(table.concat(generator[i], ',')) end 

Answer 2

In the same vein as the Finn answer, here is another permutation generator:

 local function perms(a,lo,hi,f) if lo>hi then f(a) end for i=lo,hi do a[lo],a[i]=a[i],a[lo] perms(a,lo+1,hi,f) a[lo],a[i]=a[i],a[lo] end end local function gperms(n,f) local a={} for i=1,n do a[i]=i end perms(a,1,#a,f) end local function show(a) for i=1,#a do io.write(a[i],' ') end io.write('\n') end gperms(4,show) 

Answer 3

Perhaps you can use a memory mapped file on the C ++ Lua side, for which you could provide an API through LuaBridge .

Update 1: NoSQL database could be an alternative to a memory mapped file