Need help choosing a database server

Question

Good afternoon everyone.

Recently, I was tasked with developing an application to automate some aspects of stock trading. While working on the initial architecture, a database dilemma arose. I need a fast database engine that can handle huge amounts of data coming in very quickly. I am quite experienced in general programming, but I have never encountered the task of developing a database architecture with a heavy load. I developed a simple MSSQL database schema with several many-to-many relationships during one of my projects, but that is.

I am looking for some tips on choosing the most suitable database engine and some pointers to various manuals or books that describe the development of a database with a heavy load.

Project features are as follows:

• OS: Windows NT Family (Server 2008/7)
• Primary platform: .NET with C #
• Database structure: one table for storing primary elements and two or three tables with foreign keys for the first table for storing additional information.
• SELECT database requirements: ultrafast selection is required using foreign keys and a combination of a foreign key and one of the columns (presumably DATETIME)
• INSERT database requirements: the faster the better :)

If there will be a significant increase in performance, some parts can be written in C ++ with managed interfaces for the rest of the system.

So again: given everything I just printed, give me some tips on what is the best database for my project. References and links to some manuals and books on this subject are also welcome.

EDIT: I will need to insert 3-5 rows into 2 tables about once every 30-50 milliseconds, and I will need to do SELECT with 0-2 WHERE clauses at the same speed.

Answer 1

Technology doesn't matter. It's about equipment.

Superfast selection of extraneous keys and a combination of a foreign key and one of the columns are required (supposedly DateTime)

So, you need something like a super-micro computer system of 72 disks filled with SSD and 256 gigabyte memory as a cache. Just like that. Add processors if necessary.

The problem is that it SHOULD, but at the end of the database refer to IO. Point.

Alternatively, you can use the click flow database (KD +) and pay $ 30,000 for the equipment they request for $ 20,000. For starters, the smallest software license.

I know this sounds complicated, but the speed of the database — unless you do stupid things that kill performance — is hardware. PRettym uch budget IOPS (independent IOPS per second). Not about software.

Answer 2

This list is just the basic requirements. You can use almost any database for this, although I would not recommend MySQL. This can slow down a bit when you do a lot of inserts because it clears the query cache. Other DBMSs do better.

But then again. Three tables ... you could almost do it in Access.

What you did not specify is the number of expected reads and inserts. Our poor little MySQL database runs millions of queries per day for our site. Only a small part of this is insertions and updates (DML statements), so MySQL handles this very well, but the number of queries, their complexity and the amount of data are more important than the number of tables.

Answer 3

If you already have experience with Sql Server, I would suggest sticking to it in one form or another. In addition, Sql Server and .NET have a pleasant synergistic effect (obviously) because they are both Microsoft products.

Answer 4

If this is not necessarily SQL, perhaps a system like Cassandra is a good option. Such databases are used by high-frequency websites, such as social networks. I do not know how it is installed under the Windows server.