Understanding LDA Transformed Corpus at Gensim

I tried to examine the contents of the BOW and LDA corps [BOW Corpus] (transformed by the LDA model trained on this corps, for example, with 35 topics) I found the following output:

DOC 1 : [(1522, 1), (2028, 1), (2082, 1), (6202, 1)]  
LDA 1 : [(29, 0.80571428571428572)]  
DOC 2 : [(1522, 1), (5364, 1), (6202, 1), (6661, 1), (6983, 1)]  
LDA 2 : [(29, 0.83809523809523812)]  
DOC 3 : [(3079, 1), (3395, 1), (4874, 1)]  
LDA 3 : [(34, 0.75714285714285712)]  
DOC 4 : [(1482, 1), (2806, 1), (3988, 1)]  
LDA 4 : [(22, 0.50714288283121989), (32, 0.25714283145449457)]  
DOC 5 : [(440, 1), (533, 1), (1264, 1), (2433, 1), (3012, 1), (3902, 1), (4037, 1), (4502, 1), (5027, 1), (5723, 1)]  
LDA 5 : [(12, 0.075870715371114297), (30, 0.088821329943986921), (31, 0.75219107156801579)]  
DOC 6 : [(705, 1), (3156, 1), (3284, 1), (3555, 1), (3920, 1), (4306, 1), (4581, 1), (4900, 1), (5224, 1), (6156, 1)]  
LDA 6 : [(6, 0.63896110435842401), (20, 0.18441557445724915), (28, 0.09350643806744402)]  
DOC 7 : [(470, 1), (1434, 1), (1741, 1), (3654, 1), (4261, 1)]  
LDA 7 : [(5, 0.17142855723258577), (13, 0.17142856888458904), (19, 0.50476192150187316)]  
DOC 8 : [(2227, 1), (2290, 1), (2549, 1), (5102, 1), (7651, 1)]  
LDA 8 : [(12, 0.16776844589094803), (19, 0.13980868559963203), (22, 0.1728575716782704), (28, 0.37194624921210206)]  

Where, DOC N is a document from the BOW LDA N corpus is a DOC N conversion by this LDA model

Do I correctly understand the output for each converted LDA N document as the topics to which document N belongs? Through this understanding, I can see that some documents, such as 4, 5, 6, 7, and 8, relate to more than one topic, such as DOC 8, relate to topics 12, 19, 22, and 28 with corresponding probabilities.

LDA N , , - , , ?

+4
1

, LDA gensim . , LDA[corpus] , ( ).

document belongs to ONE topic , . LDA , *. , , () 50% , .

(*) , LDA[corpus] , , . ,

theta, _ = lda.inference(corpus)
theta /= theta.sum(axis=1)[:, None]
+3

Source: https://habr.com/ru/post/1539517/


All Articles