The platform I'm working on has pretty tight memory limits, and I'm trying to find a way to parse large JSON strings without loading up to a few hundred bytes in memory. The JSON string is stored in a file on a much larger chip (flash memory).
There are two things that I cannot find a good solution for:
- Access a specific value by specifying a "path", for example
foo["bar"][2].
(And if the value turns out to be an array / object, then we should only return the fact that it is an array / object and, perhaps, it is also empty or not.) - Iterate over any object / array in JSON.
Therefore, I mainly need functions that, when called, process json step by step and save only those parts that we really need in order to continue parsing.
I do not think for an interface that could be something like exampleJson["aa"].2.["gg], but I was able to approach this: exampleJson["aa"].2.["gg"](). This will call a function call, which can then easily access {'aa', 2, 'gg'} and read / parse json from the file.
This is my code so far, but I really don't know how to proceed:
https://repl.it/HfwS/2
-- Looks complicated, but is pretty simple. Using meta tables we create a json interface that can almost be accessed as if it was a lua table.
-- E.g. example["aa"][2]["gg"]() ; the only difference is that we have to use parentheses at the end
-- The problematic part starts where it says `THIS IS WHERE THE JSON PARSING WOULD HAPPEN`
json = {}
setmetatable(json, {
__call = function(path)
local jsonFile = _file.open(filePath)
local fileLen = jsonFile:stat().size
local patternTable = {} -- Will store `{'aa',2,'gg'}` for `example.['aa'].[2]['gg']()`
local fakeJson = {}
setmetatable(fakeJson, {
__index = function (t, k)
patternTable[
return fakeJson
end;
__call = function()
-- THIS IS WHERE THE JSON PARSING WOULD HAPPEN --
-- The patternTable contains {'aa',2,'gg'} at this point
-- Loop through the json file char by char
local valueToReturn = ''
local filePos = 0
for i=1, fileLen do
jsonFile:seek("set", filePos)
local currentChar = jsonFile:read(1) -- read character at current position
filePos = filePos + 1
-- print(currentChar)
-- Now the question is, how do we parse the json?
print('Magic to parse the json')
-- valueToReturn = ?
end
patternTable = {} -- Reset the patternTable
return valueToReturn
end;
})
return fakeJson
end;
})
local fakeParsedJson = json('example.json')
local value = fakeParsedJson["aa"][2]["gg"]() -- Notice the `()` in the end
print(value)
source
share