Basic Java Application Data Warehouse

Question

I am working on a (essentially) calendar application written in Java, and I need a way to store calendar events. This is the first "real" application that I wrote, unlike simple projects (usually for classes), which either do not store information between program sessions, or store it as text or .dat files in the same directory as the program, so I have some very simple questions about data storage.

• How should event objects and other data be stored? (.dat files, some kind of database, etc.)
• Where should they be stored?
• I assume that you should not load all objects into memory when starting the program and do not update them on the hard disk until the program closes. So what should I do?

If there is some kind of textbook (or several textbooks) that covers the answers to my questions, references to them will be quite acceptable answers.

(I know that there are several similar questions that have already been asked, but none of them I could find an address for beginners.)

EDIT: As I said in one of the comments, on the whole I am interested in using it as an opportunity to learn how to do things “correctly” (reasonably scalable, fairly standard) way, even if there are simpler solutions that will work in this basic case.

Answer 1

For a quick solution, if your data structures (and, of course, the way they are accessed) are simple enough, read and write data to files using your own format (for example, binary, XML, ...) or possibly standard formats such as iCalendar may be more suitable for your problem. Libraries such as iCal4J can help you with this.

Given the more general aspects of your question, this is a more general topic, but you can read about databases (relational or not). Whether you want to use them or not, it will depend on the total complexity of your application.

In Java, you can use multiple relational databases using JBDC . This should allow you to connect to the relational database ( SQL ) of your choice. Some of them work in their own server application (for example, MS SQL, Oracle, MySQL, PostgreSQL), but some of them can be embedded in your Java application, for example: JavaDB (version of Apache Derby DB), Apache Derby DB , HSQLDB , H2 or SQLite.

These embedded SQL databases will essentially store data on files on the same computer that the application is running in (in a format specific to them), but allow data to be used using SQL queries. Benefits include a certain structure of your data (which you create when developing tables and possible restrictions) and (with the support of the engine) the ability to handle parallel access through operations . This can be useful even in a desktop application.

This may mean a learning curve if you need to learn SQL, but it should save you from having to process the details of determining your own file format. Providing a structure to your data through SQL (often known by other developers) may be better than defining your own data structures, which you still have to save and read from your own files.

In addition, if you want to directly access objects without knowing about SQL, you might be interested in Object-Relational Mapping such as Hibernate . Their goal is to hide SQL data from you, being able to store / load objects directly. Not everyone likes them, and they also have their own learning curve (which may entail learning some details about how SQL works). Their pros and cons could be discussed in detail (there are questions about this on StackOverflow or even DBA.StackExchange).

There are also other forms of databases, such as XML databases or Semantic-Web / RDF databases, that may or may not suit your needs.

Answer 2

How should event objects and other data be stored? (.dat files, some kind of database, etc.)

It depends on the size of the data that you need to save (and load), and if you want to be able to perform queries on your data or not.

Where should they be stored?

A file in the user directory (or in a subdirectory of the user directory) is a good choice. Use System.getProperty("user.home") to get it.

I assume that it is not good to load all objects into memory when the program starts and does not update them on the hard drive while the program is closing. So what am I doing instead?

This may be the right thing if the amount of data is not so large that it will consume too much memory. I do not think this would be a problem for a simple calendar application. If you do not want to do this, save the events in the database and run the queries only to load the events that should be displayed.

Answer 3

A simple sequential file is enough. Basically, each line in your file represents a record or, in your case, an event. Separate each field in your records with a field separator, something like a pipe ( | ) character. Remember to save each entry in the same format, for example:

 date|description|etc 

Thus, you can read each line in the file as a record, extract the fields by dividing the line by the delimiter character ( | ), and use the data.

Saving data in the same folder as your application should be fine.

The best way to find objects (for the most part) is to determine if the amount of data you are storing is large enough to have consequences for user memory. Based on your description, this program should be fine.

Answer 4

The correct answer depends on the details, but you probably want to write your events to the database. There are some good free databases, such as MySQL and Postgres, so you can (relatively) easily grab one and play with it.

Learning how to use a database is a big question, more than I'm going to answer in a forum. (I could recommend that you read my book, A Smarter Approach to Database Design, but making such a shameless plugin in a forum would be sticky!)

Basically, however, you want to read data from the database when you need it, and update it when it changes. Do not read everything at startup and write it all down when turned off.

If the amount of data is small and rarely changes, all this is stored in memory, and writing it to a flat file is simpler and faster. But most applications do not fit this description.