How to extract xml from log file to parse in python -



How to extract xml from log file to parse in python -

i have log file containing xml envelopes (2 types of xml structures: request , response). need parse file, extract xml-s , set them 2 arrays strings (1st array requests , 2nd array responses), can parse them later.

any ideas how can accomplish in python ?

snippet of log file parsed (log contains ):

2014-10-31 12:27:33,600 info recharger_mtelemedia2channel [mbpa.module.mgw.mtelemedia.mtbilling.mtsender][] sending bill request 2014-10-31 12:27:33,601 info recharger_mtelemedia2channel [mbpa.module.mgw.mtelemedia.mtbilling.mtsender][] <?xml version="1.0" encoding="utf-8"?> <request xmlns="xxx" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xmlns:xsd="http://www.w3.org/2001/xmlschema"> <transactionheader> <username>xxx</username> <password>xxx</password> <time>31/10/2014 12:27:33</time> <clientreferencenumber>123</clientreferencenumber> <numberrequests>3</numberrequests> <information>description</information> <postbackurl>http://localhost/status</postbackurl> </transactionheader> <transactiondetails> <items> <item id="1" client="xxx1" keyword="test"/> <item id="2" client="xxx2" keyword="test"/> <item id="3" client="xxx3" keyword="test"/> </items> </transactiondetails> </request> 2014-10-31 12:27:34,487 info recharger_mtelemedia2channel [mbpa.module.mgw.mtelemedia.mtbilling.mtsender][] response code 200 bill request 2014-10-31 12:27:34,489 info recharger_mtelemedia2channel [mbpa.module.mgw.mtelemedia.mtbilling.mtsender][] <?xml version="1.0" encoding="utf-8"?> <response xmlns="xxx" xmlns:xsd="http://www.w3.org/2001/xmlschema" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance"> <serverreferencenumber>xxx123xxx</serverreferencenumber> <clientreferencenumber>123</clientreferencenumber> <information>queued processing</information> <status>ok</status> </response>

many reply!

regards, robert

as both @paco , @lord_gestalter suggested, can utilize xml.etree , replace non-xml elements file, this:

# utilize re substitute non-xml elements import re # utilize xml module parser import xml.etree.elementtree et # read file , store in string 's' open('yourfilehere','r') f: s = f.read() # remove non-xml element re # remove <?xml ...?> part file consists of multiple xml logs s = re.sub(r'<\?xml.*?>', '', ''.join(re.findall(r'<.*>', s))) # wrap s root element s = '<root>'+s+'</root>' # parse s elementtree tree = et.fromstring(s) tree <element 'root' @ 0x7f2ab877e190>

if don't care xml parser , want 'request' & 'response' string, utilize re.search

with open('yourfilehere','r') f: s = f.read() # set string of both request , response 'req' , 'res' # or need build improve re.search if have multiple requests, responses req = [re.search(r'<request.*\/request>', s).group()] res = [re.search(r'<response.*\/response>', s).group()] req ['<request xmlns="xxx" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xmlns:xsd="http://www.w3.org/2001/xmlschema"><transactionheader><username>xxx</username><password>xxx</password><time>31/10/2014 12:27:33</time><clientreferencenumber>123</clientreferencenumber><numberrequests>3</numberrequests><information>description</information><postbackurl>http://localhost/status</postbackurl></transactionheader><transactiondetails><items><item id="1" client="xxx1" keyword="test"/><item id="2" client="xxx2" keyword="test"/><item id="3" client="xxx3" keyword="test"/></items></transactiondetails></request>'] res ['<response xmlns="xxx" xmlns:xsd="http://www.w3.org/2001/xmlschema" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance"><serverreferencenumber>xxx123xxx</serverreferencenumber><clientreferencenumber>123</clientreferencenumber><information>queued processing</information><status>ok</status></response>']

python xml logging xml-parsing

Comments

Post a Comment

Popular posts from this blog

Delphi change the assembly code of a running process -

Delphi change the assembly code of a running process - i'm trying alter address 00741fa5 has push test.009e721c . alter push test.009e71c8 . procedure callback; asm force $9e71c8 end; procedure tform2.btn1click(sender: tobject); var ppid :dword; pprocess : cardinal; begin getwindowthreadprocessid(findwindow(nil,pchar('test')), @ppid); pprocess := openprocess(process_all_access, false, ppid); injectasm(pprocess, $741fa5, 10, integer(@callback), 0); end; here injectasm function (with error checking) function injectasm(process: longword; injectaddress, injectsize, codeaddress, codesize : integer): pointer; var csize, rsize : integer; replaced : array of byte; jmp : array [0..4] of byte; jmpaddress : integer; nopv : byte; : integer; nbr: ulong_ptr; begin //injectsize must equal or greater 5, because need space our //(far) jmp wwxxyyzz instruction //if there's no space, inject in place of few instructions if injectsize ...
Read more

json - Hibernate and Jackson (java.lang.IllegalStateException: Cannot call sendError() after the response has been committed) -

json - Hibernate and Jackson (java.lang.IllegalStateException: Cannot call sendError() after the response has been committed) - hello have problem hibernate , jackson. have 2 pojos @entity @table(name = "users") public class user implements serializable { static final long serialversionuid = 4235637833262722256l; @id @generatedvalue private int id; @column(name = "creation_date") private date creationdate; @column(name = "first_name") private string firstname; @column(name = "last_name") private string lastname; @version @column(name = "version") private int version; @column(name = "e_mail") private string email; @column(name = "enabled") private boolean enabled; @column(name = "login") private string login; @column(name = "password") private string password; @jsonmanagedreference("role") @onetomany(mappedby = "user", cascade = cascadetype.all,fetch = fetchtyp...
Read more

C++ 11 "class" keyword -

C++ 11 "class" keyword - i've started using c++ 11 little bit more , have few questions on special uses of class keyword. know it's used declare class, there's 2 instances have seen don't understand: method<class t>(); and class class_name *var; why precede typename keyword class in first example, , keyword pointer in sec example? that known elaborated type specifier , necessary when class name "shadowed" or "hidden" , need explicit. class t { }; // love of god don't t t; t t2; if compiler smart, give these warnings: main.cpp:15:5: error: must utilize 'class' tag refer type 't' in scope t t2; ^ class main.cpp:14:7: note: class 't' hidden non-type declaration of 't' here t t; ^ if class not defined, deed forwards declaration. for example: template <typename t> void method() { using type = typename t::value_type; } method...
Read more