What is the best way to keep almost static data for a web application?

I am creating a web application in python. Part of this application works with data that can be described as follows:

Symbol     Begin Date      End Date
AAPL       Jan-1-1985      Dec-27-2010
...

The data is somewhat static - they will be updated periodically, that is: new records can be added, and the "End date" field can be updated for all records.

Now, the question is: given the more or less static nature of the data set, what is the best way to store and work with it? "Job" means fetching random strings, hopefully more than a few times per second.

I can do this with an XML file, with SQL DB or SQLite, with a JSON object file and some kind of python object in memory.

What are the pros and cons of different solutions? I will be grateful for the explanations and for the extreme situation (for example, until the XML file up to 10 times / sec becomes the best after SQL DB).

Update: Thanks for all the answers! Just a small update: currently the set is about 3 thousand lines. It can grow to, say, 15 thousand lines per year. Access pattern: updates are regular, once a day, for a full set; therefore, both the addition of rows and the updating of the end date will be performed immediately. Fetching a random string is valid by character, can be performed several times per second.

+3
source share
6 answers

Python , , . , .

data = {
  "AAPL":       ("Jan-1-1985",      "Dec-27-2010"),
...
}

pprint.pprint, .

. , , script,

import random, string, pprint

def randsym():
    res =[]
    for i in range(4):
        res.append(random.choice(string.uppercase))
    return ''.join(res)

months = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 
          'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']
days = range(1,29)
years = range(1980,2010)
def randdate():
    return "%s-%s-%s" % (random.choice(months),
                         random.choice(days),
                         random.choice(years))

data = {}
for i in range(15000):
    data[randsym()] = (randdate(), "Dec-27-2010")

with open("data.py", "w") as f:
    f.write("data=")
    f.write(pprint.pformat(data))

, from data import data.

+3

, XML . CSV JSON . , (.. , ), , . , , .

0

? - . python dict, (csv ?) -, - .

? python , (csv ?) -, -, .

, - , . , .

0

, , - . read-many-write-once , :

  • SQLite
  • MySQL
  • /

, , .. ? , , ad hoc.

0

XML, . , , , " " (.. AAPL ). , CSV, :

import csv

myList = []
myReader = csv.reader(open("your_file.csv", "rb"))
for row in reader:
    myList.append(row)
...do stuff...
myWriter = csv.writer(open("your_file.csv", "wb"))
myWriter.writerows(myList)

, , SQL, . SQL (MySQL, MSSQL, Postgre ..) SQL CSV XML.

" " CSV, , . , - , , SQL .

0

: SQL , . SQL, , , (98%) .

CSV: , (12 ) . - , ramdrive, . , . , , , SSD. CSV SSD, 1000 /, . .

Ifs , - .

0

Source: https://habr.com/ru/post/1782350/


All Articles