String iteration in Vimscript or Parse JSON file

Question

So, I am creating a vim script that should load and parse the JSON file into a local graph of objects. I searched, and I could not find any native way to handle the JSON file, and I do not want to add any dependencies to the script. So I wrote my own function to parse the JSON string (obtained from the file), but it is very slow. At the moment, I repeat each character in the file as follows:

let len = strlen(jsonString) - 1 let i = 0 while i < len let c = strpart(jsonString, i, 1) let i += 1 " A lot of code to process file.... " Note: I've tried short cutting the process by searching for enclosing double-quotes when I come across the initial double quotes (also taking into account escaping '\' character. It doesn't help endwhile 

I also tried this method:

 for c in split(jsonString, '\zs') " Do a lot of parsing .... endfor 

For reference, a file with ~ 29,000 characters takes about 4 seconds to process, which is unacceptable.

Is there a better way to iterate over a string in a vim script?

Or even better, did I miss the native function for parsing JSON?

Update:

I did not request any dependencies because I:

• I did not want to deal with them.
• I really would like to get some ideas for a better way to do this if someone else is not working.
• Sometimes I just like to do something manually, although the problem has already been solved.

I'm not against plugins or dependencies at all, I'm just curious. So the question is.

I ended up creating my own function to parse a JSON file. I created a script that could parse the package.json file associated with the node.js modules. Because of this, I could rely on a fairly consistent format and stop processing whenever I get the information I need. This usually cuts large chunks of the file, since most developers put the largest fragment of the file, its "readme" section, at the end. Since the package.json file is strictly defined, I left the process somewhat fragile. He suggested the root dictionary { } and is actively looking for specific entries. You can find the script here: https://github.com/ahayman/vim-nodejs-complete/blob/master/after/ftplugin/javascript.vim#L33 .

Of course, this does not answer my question. This is only a solution to my unique problem. I will wait a few days for new answers and pick up the best one before the distribution ends (the alarm is already set on my phone).

Answer 1

If you are looking for the right solution without addiction, this is one of the worst things you can do. The eval() option mentioned by @MarcWeber is one of the fastest, but has its drawbacks:

• Using an eval protection solution . I mentioned in a comment that makes it no longer the fastest. In fact, after you use this, it makes eval() slower by more than an order of magnitude (0.02 s versus 0.53 in my test).
• He does not respect surrogate couples.
• It cannot be used to make sure that you have the correct JSON: it accepts some lines (for example, "\<Co>" ) that are not JSON lines, and allows you to use commas.
• It does not give normal error messages. It does not work well if you use vam#VerifyIsJSON , which I mentioned in page 1.
• It is not possible to load floating point values, such as 1e10 (vim requires numbers to look like 1.0e10 , but numbers like 1e10 are allowed : mark "and / or" in the first paragraph).

. All of the above (except the first) also applies to vim-addon-json-encoding , mentioned by @MarcWeber, because it uses eval. There are other possibilities:

• The fastest and most correct one is to use python: pyeval('json.loads(vim.eval("varname"))') . Not faster than eval, but fastest among other features. (0.04 in my test: about two times slower than eval() )

  Please note that here I am using pyeval() . If you need a solution for a version of vim that does not have this functionality, it will no longer be one of the fastest.

• Use the json.vim plugin. It has the advantages of a slightly better error report compared to the unsuccessful vam#VerifyIsJSON , slightly worse compared to eval() , and it loads floating point numbers correctly. It can be used to check strings (it does not accept "\<Ca>" ), but it loads the trailing comma lists just fine. It does not support surrogate pairs. This is also very slow: in the test I used (it uses long lines with a length of 279702), it takes 11.59s to load. Json.vim is trying to use python if possible, though.
• For best error information, you can take yaml.vim and clear YAML support, leaving only JSON (I once did the same thing for pyyaml, though in python: see markjon library used in powerline : this is pyyaml ​​minus the stuff YAML plus classes with labels). But this option is even slower than json.vim and should be used only if the main thing you need is an error message: 207 seconds to load the same long line with a length of 279702.

Note that the only option that satisfies both the “no dependencies” and “no python” requirements is equal to eval() . If you are not okay with your flaws, you need to throw away one or both of these requirements. Or copy-paste code. Although, if you consider speed, there are only two candidates left: eval() and python: if you want to quickly parse json, you really should use C, and only these solutions spend the most time on functions written in C.

Most other interpreters (ruby / perl / TCL) do not have the pyeval() equivalent, so they will be slower even if their JSON implementation is written in C. Some others (lua / racket (mzscheme)) have pyeval() equivalent, but for example , luaeval('{}') means that you will need to add an additional step to explicitly and recursively convert objects to dictionaries and vim lists (for example, luaeval('vim.dict({})') ), which will affect performance. I can not say anything about mzeval() , but I have never heard that someone really uses a racket (mzscheme) with vim.

Answer 2

Despite the fact that Vim's origin dates back very often, its internal representation of string () is eval (), which is close to JSON, which will probably work if you don't need special characters.

You can find an implementation here that even supports true / false / null if you want: https://github.com/MarcWeber/vim-addon-json-encoding

It is better to use this library (vim-addon-manager makes it easy to install dependencies). Now it depends on your data, is it enough.

Now Benjamin Klein has sent your question to vim_use, so I answer. Better and faster answers come when you subscribe to the Vim mailing list. Go to vim.sf.net and follow the community link. You cannot expect the Vim community to clear stackoverflow.

I added the keywords "json" and "parsing" to this little code that can be found more easily.

If this solution does not work for you, you can try many links: h if_ * or write an external script that retrieves the information you are looking for, or turns JSON into a Vim dictionary view that can be read by eval (), avoiding special characters, which you care about correctly.

Answer 3

The simplest solution with the least number of dependencies is to simply use json_decode vim json_decode .

 let dict = json_decode(jsonString)