All geek questions in one place

How to convert a nested dictionary to dataframe?

I have a nested dictionary. This is some Nasdaq data. Like this:

{'CLSN':     
 Date        Open  High   Low  Close  Volume  Adj Close                                                
 2015-12-31  1.92  1.99  1.87   1.92   79600       1.92
 2016-01-04  1.93  1.99  1.87   1.93   39700       1.93
 2016-01-05  1.89  1.94  1.85   1.90   50200       1.90,
 'CCC':    
 Date            Open       High        Low      Close  Volume  Adj Close                                                              
 2015-12-31  17.270000  17.389999  17.120001  17.250000  177200  16.965361
 2016-01-04  17.000000  17.219999  16.600000  17.180000  371600  16.896516
 2016-01-05  17.190001  17.530001  17.059999  17.450001  417500  17.162061,
}

To help you understand, this is the key followed by the values , and the values are the dataframe

Before asking, I tried the path pd.Panel(nas)['CLSN'], so I'm sure its value is a dataframe. But the way pd.Panel(nas).to_frame().reset_index()doesn’t help me at all! It displays an empty data block with thousands of columns that are populated with the stock name.

Now this is bothering, I want the dataframe to look like this:

index  Date      Open       High       Low       Close      Volume     Adj Close                                            CLSN 2015-12-31  1.92       1.99       1.87       1.92       79600.0   1.92
CLSN 2016-01-01   NaN       NaN        NaN        NaN        NaN       NaN
ClSN 2016-01-04  1.93       1.99       1.87       1.93       39700.0   1.93  
CCC  2015-12-31  17.270000  17.389999  17.120001  17.250000  177200.0  16.965361
CCC  2016-01-04  17.000000  17.219999  16.600000  17.180000  371600.0  16.896516
CCC  2016-01-05  17.190001  17.530001  17.059999  17.450001  417500.0  17.162061

Of course, I can use a loop forto get each frame of stock data, but it kills me to join everyone.

Do you have a better idea? Extremely want to know!

MaxU: print(nas['CLSN'].head()) :

            Open  High   Low  Close  Volume  Adj Close
Date                                                  
2015-12-31  1.92  1.99  1.87   1.92   79600       1.92
2016-01-04  1.93  1.99  1.87   1.93   39700       1.93
2016-01-05  1.89  1.94  1.85   1.90   50200       1.90
2016-01-06  1.86  1.89  1.77   1.78   62100       1.78
2016-01-07  1.75  1.80  1.75   1.77  117000       1.77
+4
2

UPDATE:

, Date ( ):

:

In [70]: d2
Out[70]:
{'CCC':                  Open       High        Low      Close  Volume  Adj Close
 Date
 2015-12-31  17.270000  17.389999  17.120001  17.250000  177200  16.965361
 2016-01-04  17.000000  17.219999  16.600000  17.180000  371600  16.896516
 2016-01-05  17.190001  17.530001  17.059999  17.450001  417500  17.162061,
 'CLSN':             Open  High   Low  Close  Volume  Adj Close
 Date
 2015-12-31  1.92  1.99  1.87   1.92   79600       1.92
 2016-01-04  1.93  1.99  1.87   1.93   39700       1.93
 2016-01-05  1.89  1.94  1.85   1.90   50200       1.90}

:

In [73]: pd.Panel(d2).swapaxes(0, 2).to_frame().reset_index(level=0).sort_index()
Out[73]:
            Date       Open       High        Low      Close    Volume  Adj Close
minor
CCC   2015-12-31  17.270000  17.389999  17.120001  17.250000  177200.0  16.965361
CCC   2016-01-04  17.000000  17.219999  16.600000  17.180000  371600.0  16.896516
CCC   2016-01-05  17.190001  17.530001  17.059999  17.450001  417500.0  17.162061
CLSN  2015-12-31   1.920000   1.990000   1.870000   1.920000   79600.0   1.920000
CLSN  2016-01-04   1.930000   1.990000   1.870000   1.930000   39700.0   1.930000
CLSN  2016-01-05   1.890000   1.940000   1.850000   1.900000   50200.0   1.900000

Date :

In [74]: pd.Panel(d2).swapaxes(0, 2).to_frame().sort_index()
Out[74]:
                       Open       High        Low      Close    Volume  Adj Close
Date       minor
2015-12-31 CCC    17.270000  17.389999  17.120001  17.250000  177200.0  16.965361
           CLSN    1.920000   1.990000   1.870000   1.920000   79600.0   1.920000
2016-01-04 CCC    17.000000  17.219999  16.600000  17.180000  371600.0  16.896516
           CLSN    1.930000   1.990000   1.870000   1.930000   39700.0   1.930000
2016-01-05 CCC    17.190001  17.530001  17.059999  17.450001  417500.0  17.162061
           CLSN    1.890000   1.940000   1.850000   1.900000   50200.0   1.900000

OLD answer - , Date ( ) :

In [59]: pd.Panel(d).swapaxes(0, 2).to_frame().reset_index('major', drop=True).sort_index()
Out[59]:
            Date   Open   High    Low  Close  Volume Adj Close
minor
CCC   2015-12-31  17.27  17.39  17.12  17.25  177200   16.9654
CCC   2016-01-04     17  17.22   16.6  17.18  371600   16.8965
CCC   2016-01-05  17.19  17.53  17.06  17.45  417500   17.1621
CLSN  2015-12-31   1.92   1.99   1.87   1.92   79600      1.92
CLSN  2016-01-04   1.93   1.99   1.87   1.93   39700      1.93
CLSN  2016-01-05   1.89   1.94   1.85    1.9   50200       1.9

d - nested dictionary:

In [60]: d
Out[60]:
{'CCC':         Date       Open       High        Low      Close  Volume  Adj Close
 0 2015-12-31  17.270000  17.389999  17.120001  17.250000  177200  16.965361
 1 2016-01-04  17.000000  17.219999  16.600000  17.180000  371600  16.896516
 2 2016-01-05  17.190001  17.530001  17.059999  17.450001  417500  17.162061,
 'CLSN':         Date  Open  High   Low  Close  Volume  Adj Close
 0 2015-12-31  1.92  1.99  1.87   1.92   79600       1.92
 1 2016-01-04  1.93  1.99  1.87   1.93   39700       1.93
 2 2016-01-05  1.89  1.94  1.85   1.90   50200       1.90}
+3

, pandas.concat - , :

In [8]: data = dict(A=pd.DataFrame([[1,2], [3,4]], columns=['X', 'Y']),
                    B=pd.DataFrame([[1,2], [3,4]], columns=['X', 'Y']),)

In [9]: data
Out[9]: 
{'A':    X  Y
 0  1  2
 1  3  4, 
 'B':    X  Y
 0  1  2
 1  3  4}

In [10]: pd.concat(data)
Out[10]: 
     X  Y
A 0  1  2
  1  3  4
B 0  1  2
  1  3  4
+2

Source: https://habr.com/ru/post/1674413/

More articles:


All Articles