Getting comments with python libclang

Question

In the following header file, I would like to get the corresponding +reflect comment on the class and member variable:

 #ifndef __HEADER_FOO #define __HEADER_FOO //+reflect class Foo { public: private: int m_int; //+reflect }; #endif 

Using python bindings for libclang and the following script:

 import sys import clang.cindex def dumpnode(node, indent): print ' ' * indent, node.kind, node.spelling for i in node.get_children(): dumpnode(i, indent+2) def main(): index = clang.cindex.Index.create() tu = index.parse(sys.argv[1], args=['-x', 'c++']) dumpnode(tu.cursor, 0) if __name__ == '__main__': main() 

Gives me this result:

 CursorKind.TRANSLATION_UNIT None CursorKind.TYPEDEF_DECL __builtin_va_list CursorKind.CLASS_DECL type_info CursorKind.CLASS_DECL Foo CursorKind.CXX_ACCESS_SPEC_DECL CursorKind.CXX_ACCESS_SPEC_DECL CursorKind.FIELD_DECL m_int 

The problem is that there are no comments. Are they devoid of preprocessor? Is there any way to prevent this?

Answer 1

To do this, you need to get tokens, not cursors. If I ran this script in the file above:

 import sys import clang.cindex def srcrangestr(x): return '%s:%d:%d - %s:%d:%d' % (x.start.file, x.start.line, x.start.column, x.end.file, x.end.line, x.end.column) def main(): index = clang.cindex.Index.create() tu = index.parse(sys.argv[1], args=['-x', 'c++']) for x in tu.cursor.get_tokens(): print x.kind print " " + srcrangestr(x.extent) print " '" + str(x.spelling) + "'" if __name__ == '__main__': main() 

I get the following:

 TokenKind.PUNCTUATION test2.h:1:1 - test2.h:1:2 '#' TokenKind.IDENTIFIER test2.h:1:2 - test2.h:1:8 'ifndef' TokenKind.IDENTIFIER test2.h:1:9 - test2.h:1:21 '__HEADER_FOO' TokenKind.PUNCTUATION test2.h:2:1 - test2.h:2:2 '#' TokenKind.IDENTIFIER test2.h:2:2 - test2.h:2:8 'define' TokenKind.IDENTIFIER test2.h:2:9 - test2.h:2:21 '__HEADER_FOO' TokenKind.COMMENT test2.h:4:1 - test2.h:4:11 '//+reflect' TokenKind.KEYWORD test2.h:5:1 - test2.h:5:6 'class' TokenKind.IDENTIFIER test2.h:5:7 - test2.h:5:10 'Foo' TokenKind.PUNCTUATION test2.h:6:1 - test2.h:6:2 '{' TokenKind.KEYWORD test2.h:7:5 - test2.h:7:11 'public' TokenKind.PUNCTUATION test2.h:7:11 - test2.h:7:12 ':' TokenKind.KEYWORD test2.h:8:5 - test2.h:8:12 'private' TokenKind.PUNCTUATION test2.h:8:12 - test2.h:8:13 ':' TokenKind.KEYWORD test2.h:9:9 - test2.h:9:12 'int' TokenKind.IDENTIFIER test2.h:9:13 - test2.h:9:18 'm_int' TokenKind.PUNCTUATION test2.h:9:18 - test2.h:9:19 ';' TokenKind.COMMENT test2.h:9:20 - test2.h:9:30 '//+reflect' TokenKind.PUNCTUATION test2.h:10:1 - test2.h:10:2 '}' TokenKind.PUNCTUATION test2.h:10:2 - test2.h:10:3 ';' TokenKind.PUNCTUATION test2.h:12:1 - test2.h:12:2 '#' TokenKind.IDENTIFIER test2.h:12:2 - test2.h:12:7 'endif' 

For me, should be enough for the job.

Answer 2

You need to modify the cindex.py script and set the following function.

 class Cursor(Structure): def getRawComment(self): return conf.lib.clang_Cursor_getRawCommentText(self) 

also add this to the right place in cindex.py

 ("clang_Cursor_getRawCommentText", [Cursor], _CXString, _CXString.from_result), 

I had to make my comments using

  /*! * +reflect */ though 

Answer 3

Yes, all comments are removed by the preprocessor. You can see that by running clang -E mycode.c > mycode.i , which will provide you with the mycode.i file with all the preprocessing, but no comment.

Perhaps you can do something using #pragma or something that is not without and is ignored by the compiler.