Managing a lot of data and images inside

Question

my question is similar to another friend posted here ... we are trying to develop an application that supports perhaps terabytes of information based on the land registry in Paraguay with images and normal data.

The problem is that we want to minimize the cost of the operation to a minimum, because it is like competition between companies, and for this reason we want to use a free database ... I read a lot about this, but I'm still confused. We must understand that the people who are going to use it are government people, so the database must be easily managed at the same time.

What would you advise me?

Tanu is very much

Answer 1

MySQL and even SQLite already have spatial indexes, so there are no problems.

You can use the BLOB field to store data files, but it is usually better (and easier to optimize) to store as files. So that the files associated with database records, you can put the full path (or URL) in the varchar field or save the image in the path calculated by the record identifier.

To easily scale to a multi-terabyte storage, plan to use multiple servers from the start. If the data is read - basically, an easy way is to store the images on different hosts, each with a static HTTP server, and each image is written to the database. then put the webapp interface for the database, where the URLs for each image directly point to the corresponding storage server. Thus, you can continue to add storage without creating a bottleneck on the “central” server.

Answer 2

Postgresql , SQL Server 2008 and Any recent version of Oracle has spatial indexing, table partitioning and BLOBs and can act as the background database of a large geographic database. You can also check two open source GIS applications: GRASS and QGIS , which can help Do what you want with less changes than writing a custom application. Both can use Postgresql and other databases.

In terms of support, any commercial or open source database will need the attention of a competent database administrator if you want it to work well on terabyte databases. I do not think that you will leave with a model of net end-user support - attempts to do this are unlikely to work.

Answer 3

It looks like image files will be a significant amount of your storage. Do not store them in the database, just store the file location data in the database.

(If you want to access over the Internet, try Amazon Storage . It's not free, it's very cheap, and they handle scalability for you.)

Answer 4

Another cautionary note about using B / C / LOB, as I was bitten by an exponential increase in the database, while saving internally with the database.

How about saving GIS maps on a separate server and just saving the LAT / LONG "form" in an area without a database. GIS can be updated separately at the cost of storing images in the main database.

Less to admin. Less backup costs.

Answer 5

Although you do not meet your criteria for free, I highly recommend that you use SQL Server 2008 because of the two Gfeatures in this version that might help:

• FILESTREAM - allows you to store your binary images in the file system, and not in the database itself. This will make your database more manageable, while allowing you to query data in the usual way.

• TYPES OF GEOGRAPHICAL DATA - support for geospatial (lat / long) data types is likely to be very valuable for your decision.

Good luck

Answer 6

Use ESRI Image Server. You do not need a database to work with images. It is very easy to use. It also works with files and quickly and processes many image formats. In addition, it processes images on the fly and supports many clients. AutoCAD, Microstation, ArcMap, ArcIMS, ArcServer ... etc.

Image server