KnockoutJS foreach blocks the main thread

Question

When I have a large dataset in my Model view, and I use foreach to foreach through an array of objects to render each object as a row inside a table, KnockoutJS blocks the main thread until it can display, which sometimes takes minutes (!).

Here is a jsFiddle example using a dataset containing 2000 objects containing url and a code . The actual data will have longer URLs in some cases and 4 more columns (only in this example.) I also added some simple styles, since adding styles also seems to slow down the process a bit during the process.

Warning: your browser may break

http://jsfiddle.net/DESC3/7/

Answer 1

I suggest that if you have such large data sets, you should try an alternative solution. For example, slickGrid makes large data arrays in a more efficient way, creating only HTML elements for the actually visible data. We used this for large datasets, and it works well.

Answer 2

How about something like that. Let's say you have viewModel.items = ko.observableArray() that you want to display.

• You have a separate invisible array of all your data: var itemsToRender = functionThatReturnsLargeArray() .
• Put some of your data from itemsToRender into your observable array. Let's say just 50 elements.
• Continue adding elements to the observed array piecemeal inside the setTimeout callback.

---

NOTE 1. You can add some time tracking to the setTimeout callback and increase / decrease the number of elements added at each iteration. Your goal is to keep every callback time below 50-100 milliseconds so that your application still feels responsive.

 var batchSize = 50; // default number of items rendered per iteration var batchOffset = 0; function render(items, itemsToRender, done) { setTimeout(function () { var startTime = new Date().getTime(); items.pushAll(itemsToRender.slice(batchOffset, batchSize)); batchOffset += batchSize; // at this point Knockout rendered next batchSize items from itemsToRender var endTime = new Date().getTime(); // update batchSize for next iteration batchSize = batchSize * 50 / (endTime - startTime); // 50 milliseconds batchSize = Math.min(itemsToRender.length, batchOffset + batchSize); if (batchSize > 0) render() else done(); // callback if you need one }, 0); } /* I haven't actually tested the code */ 

Another batch size upgrade strategy may be based on a targeted FPS. Suppose you want to achieve a refresh rate of 60 frames per second and therefore 60 calls per setTimeout in 1000 milliseconds. It will take a little longer to process the entire collection. You can also use requestAnimationFrame instead of setTimeout and see how this will work.

EDIT : Built-in throttling added in Knockout JS 1.3 (currently in beta, but looks pretty stable).

---

NOTE 2: If some other data in the view is dependent on viewModel.items , you can still map it to the original itemsToRender array. Say, for example, that you want to show the number of elements in a collection. If you use viewModel.items().length , you will end up changing the size value in the user interface, while more elements will get renderred. To avoid this, you can first define your size snapping as dependentObservable based on itemsToRender , not viewModel.items . After you have rendered all the elements, you can rearrange them to viewModel.items if you think it is necessary.