parsing - Can I use python ast module for this? -


I would like to write a program that will modify dragon programs like this:

Change

"Some lexical string%"% some

functioncall ("some literal string%")%

thanks to something,

This can be easy - optimization of examples in Docs, 

  import CStringIO import tokenize class overview (object): def __init __ (self, s): self._t = tokenize.generate_tokens (cStringIO.StringIO) .readline) self.lookahead = Agal (Self._t, none) DEF __iter__ (self): return self def next (self): result = self.lookahead If there is no result: increased StopIteration self.lookahead = next (self._t, none) return result Def charts (s): Toks = Proceedings (s) Results = [] toktype, tokvalue, to _, _, _ in Toks: if toktype == tokenize.STRING: p = toks.lookahead if pk none and p [ 0] == is not tokenize.OP and pk [1] == '%': result.extend ([(tokenize.NAME, 'functioncall'), (tokenize.OP, '('), (tokenize.STRING, Wrapper (tokvalue)), (tokenize.OP, ')')]) result.append ((toktype, tokvalue)) return tokenize.untokenize (results) Print Chart ( '' Some literal string% 'to continue'% some ')

This printscall (' 'some literal string%' ') prints some%. This vacancy is very strange (it tries very hard to get the blank space - but it is worse for reconstruction of sources from revised AST), but it is fine if you are going to import / run As a result code for (if you want to get a well-readable and editable code, is not OK - but this is a big problem that I suggest a separate Q;).


Comments

Post a Comment