How to make a Sliding window model for data flow design?

Question

we have a situation where the stream (data from the sensor data or the click stream on the server) comes with a sliding window algorithm, we must store the last (say) 500 data samples in memory. These samples are then used to create histograms, aggregations, and capture information about anomalies in the input data stream.

please tell me how to make such a sliding window.

Answer 1

If you are asking how to store and maintain these values ​​in a sliding window mode, consider this simple example, which stores traces of the current average value of the last 10 values ​​of a random data stream:

WINDOW_SIZE = 10; x = nan(WINDOW_SIZE,1); %# init counter = 0; stats = [NaN NaN]; %# previous/current value %# prepare figure SHOW_LIM = 200; hAx = axes('XLim',[1 SHOW_LIM], 'YLim',[200 800]); hLine = line('XData',1, 'YData',nan, 'EraseMode','none', ... 'Parent',hAx, 'Color','b', 'LineWidth',2); %# infinite loop! while true val = randi([1 1000]); %# get new value from data stream x = [ x(2:end) ; val ]; %# add to window in a cyclic manner counter = counter + 1; %# do something interesting with x stats(1) = stats(2); %# keep track of the previous mean stats(2) = nanmean(x); %# update the current mean %# show and update plot set(hLine, 'XData',[counter-1 counter], 'YData',[stats(1) stats(2)]) if rem(counter,SHOW_LIM)==0 %# show only the last couple of means set(hAx, 'XLim', [counter counter+SHOW_LIM]); end drawnow pause(0.02) if ~ishandle(hAx), break, end %# break in case you close the figure end 

---

Update

The EraseMode=none property EraseMode=none been deprecated and removed in recent versions. Use animatedline instead of a similar function.