Should I use a text file or database?

Question

So, I am compiling an RSS parser that will process the RSS feed, filter it, and then load the associated items. Suppose that the downloaded files are legitimate torrent files.

Now I need to keep a record of the files that I have already downloaded, so they no longer run.

I already have work with SQLite (create a database if it does not exist, insert a line if the select statement returns nothing), but the resulting jar file is 2.5MB + (due to sqlite libs).

I think that if I use a text file, I can shorten the jar file to a few hundred kilobytes.

I could keep a list of the names of the downloaded files - one per line - and read the entire file in memory, search if the file exists, etc.

A few questions that I know know:

• Tell me, do you upload 10 files per day, will the text file method be completed with too many resources?
• In general, which is faster

Anyway, what do you guys think? I could use some tips here as I am still new to programming and doing it as a hobby :)

Answer 1

If you need to track only a few details (e.g. file name), you can use a plain text file for sure.

Using BufferedReader , you should achieve good performance.

Answer 2

Theoretically, a database (either relational or NoSQL is better. But if the size of the distribution is critical for you, using a file system may be preferable.

The only problem here is the performance of data access (either for writing or reading). Probably think about the following approach. Do not use a single file. Instead, use a directory containing several files. The file name will contain the key (or keys), which allows you to access the data, just like the key on the map. In this case, you can access the data relatively quickly and easily.

Probably take a look at XStream. They have an implementation of the Map, which is implemented as described above: it saves records on disk, each record in a separate file.