What data structure is reasonable for time series data in Java?

Question

I'm new to Java, so I'm not sure which data structure choice would be good here. I will store the data of the accelerometer, gyroscope and magnetometer (9 values) in the list, which will be used later for smoothing, displaying and processing some signals.

My idea is to create a MyObject that has ten members: a timestamp and nine orientation / movement values, they all float. Then I save the data in an ArrayList<MyObject> . Is this a good idea or am I missing something?

The list will contain no more than 100 thousand values.

Answer 1

Use TreeMap to get started to improve your search performance.

Treemap

Note (in docs):

This implementation provides a guaranteed log (n) time value for containsKey, get, put, and remove operations.

Answer 2

Creating your own class is the right way. Now for storing objects of this class ...

If you need a direct search (for example, "give me the 5th object") on objects, when you have everything, use an ArrayList . But if you just iterate over all the objects in order, you should consider a linked list .

The ArrayList class uses a primitive array, which will grow as needed to accommodate the elements you add to it. When it develops its internal structure, it needs to select a new array and copy all the original values. This can be expensive (especially with 100K elements!). You can give it its original size to give it more time before it needs to grow; but if you don’t need so much space, the internal ArrayList can waste a large chunk of memory.

Adding items to a linked list is practically worthless, because "growth" is not needed; it just adds another node to the list. But you cannot search for items by their index. You will need to start from the first element and iterate over the list into the desired element.

Answer 3

You might want to use the file output stream to output your data directly, instead of storing it in some data structure:

 output = new BufferedWriter(new FileWriter("output.csv")); while(dataSource.stillHasData()) output.println(dataSource.getData().toString()); output.close(); 

Using BufferedWriter ensures that the program does not need to wait for the disk to write before it can take the following data, so this will be acceptable for collecting live data (usually).

Then you should do something similar in your data class:

 public String toString(){ StringBuilder buf = new StringBuilder(); buf.append(timeStamp); str.append(','); // ... // append all the other data return buf.toString(); } 

In this case, the advantage is that you can import it into programs such as excel or really just about any program that you will use to process the data.