Decoding log files in binary using an XML file.

[captainfantastic]

Good day.

I am attempting to create a new log decoder based on an existing program, the message data is compared against an XML file and binary strings are translated in to human readable format.

below is an example of one of the messages:

Output:
[01-04-2019 13:16:18.209]  MESSAGE-RX  [DSE->ATO] ID=102 SEQNUM=0000173551 TIME=1554124577 SRC=01 DST=00 LEN=040 VALID=1 CRC=6CC41467 MESSAGE=6600280002A5EF00010008000000005CA20F21000100005CA20F210000050D00000000006714C46C

each line corresponds to one message, below is an example of the XML file that has the protocol that the log explorer uses to decode each message:

Output:
<message id="5" minlen="30" maxlen="30" name="hel locomotive command" msgType="request">
  <header>
    <item name="sequenceNum" len="32" />
    <item name="timestamp" len="32" />
    <item name="version" len="16" />
    <item name="sourceId" len="16" />
    <item name="destinationId" len="16" />
    <item name="length" len="16" />
  </header>
  <payload>
    <field><item name="messageId" len="8" descriptionLink="ATO_MESSAGE_ID" ><values><value>5</value></values></item> </field>
    <field><item name="targetLocmPosition" len="8" descriptionLink="LOCOMOTIVE_POSITION" ><values><range from="0" to="16" /></values></item> </field>
    <field name="locmCommands">
      <field><item name="applyEngineRunIsolate" len="8" descriptionLink="LOCOMOTIVE_RUN_ISOLATED" ><values><range from="0" to="3" /></values></item> </field>
      <field><item name="applyShortHoodRedLights" len="8" descriptionLink="COMMAND_ON_OFF" ><values><range from="0" to="1" /></values></item> </field>
      <field><item name="applyLongHoodRedLights" len="8" descriptionLink="COMMAND_ON_OFF" ><values><range from="0" to="1" /></values></item> </field>
      <field><item name="applySafeToBoardLights" len="8" descriptionLink="COMMAND_ON_OFF" ><values><range from="0" to="1" /></values></item> </field>
      <field><item name="applySafeToAlightLights" len="8" descriptionLink="COMMAND_ON_OFF" ><values><range from="0" to="1" /></values></item> </field>
      <field><item name="applyBell" len="8" descriptionLink="COMMAND_ON_OFF" ><values><range from="0" to="1" /></values></item> </field>
      <field><item name="applyHorn" len="8" descriptionLink="COMMAND_ON_OFF" ><values><range from="0" to="1" /></values></item> </field>
      <field><item name="cmmBypass" len="8" descriptionLink="COMMAND_ON_OFF" ><values><range from="0" to="1" /></values></item> </field>
    </field>
  </payload>
  <integrity poly="0x1EDC6F41" init="0x00000000" refin="false" refout="false" xored="0x00000000">
    <item name="crc" len="32" />
  </integrity>
</message>

Does anyone know of any libraries that I can use to make this process easier? id prefer to not hard code anything in the translation part because I dont want my work to become obsolete when a new protocol is released.

Thank you

[captainfantastic]

I found a library that might fit the bill, PyXML or easeXML seem like they might be able to achieve what I am looking for.

looking at the input and the structure of the message and the structure of the xml that decodes it the lengths quoted dont seem to match up:

Message:

Output:
[01-04-2019 13:16:18.209]  MESSAGE-RX  [DSE->ATO] ID=102 SEQNUM=0000173551 TIME=1554124577 SRC=01 DST=00 LEN=040 VALID=1 CRC=6CC41467 MESSAGE=6600280002A5EF00010008000000005CA20F21000100005CA20F210000050D00000000006714C46C

XML:

Output:
<message id="102" minlen="40" maxlen="40" name="dse internal status" msgType="request">
  <payload>
    <field><item name="messageId" len="8" descriptionLink="DSE_MESSAGE_ID" ><values><value>102</value></values></item> </field>
    <field name="dseHeader">
      <field><item name="length" len="16" /> </field>
      <field><item name="sequenceNumber" len="32" /> </field>
      <field><item name="destinationId" len="8" descriptionLink="DEVICE_ID" ><values><range from="0" to="3" /></values></item> </field>
      <field><item name="sourceId" len="8" descriptionLink="DEVICE_ID" ><values><range from="0" to="3" /></values></item> </field>
      <field><item name="ack" len="8" descriptionLink="BOOLEAN" ><values><range from="0" to="1" /></values></item> </field>
      <field><item name="protocolVersion" len="8" /> </field>
      <field><item name="debug" len="32" /> </field>
      <field><item name="timestamp" len="32" /> </field>
    </field>
    <field><item name="currentDseOperativeMode" len="8" descriptionLink="DSE_MODE" ><values><range from="0" to="2" /></values></item> </field>
    <field><item name="role" len="8" descriptionLink="ROLE" ><values><range from="0" to="1" /></values></item> </field>
    <field><item name="isInsideStopWindow" len="8" descriptionLink="BOOLEAN" ><values><range from="0" to="1" /></values></item> </field>
    <field><item name="liftingStrategyStatus" len="8" descriptionLink="LIFTING_STRATEGY_STATUS" ><values><range from="0" to="2" /></values></item> </field>
    <field><item name="dseInternalUtc" len="32" /> </field>
    <field><item name="dseFault" len="8" descriptionLink="DSE_FAULT" ><values><range from="0" to="8" /></values></item> </field>
    <field name="dseSwVersion">
      <field><item name="unused" len="8" /> </field>
      <field><item name="majorVersion" len="8" /> </field>
      <field><item name="minorVersion" len="8" /> </field>
      <field><item name="buildVersion" len="8" /> </field>
    </field>
    <field name="dseTrackDbVersion">
      <field><item name="unused" len="8" /> </field>
      <field><item name="majorVersion" len="8" /> </field>
      <field><item name="minorVersion" len="8" /> </field>
      <field><item name="buildVersion" len="8" /> </field>
    </field>
  </payload>
  <integrity poly="0x4A503DF1" init="0x00000000" refin="false" refout="false" xored="0x00000000">
    <item name="crc" len="32" />
  </integrity>
</message>

I feel like im missing something obvious here but the message is 80 bytes and the length is listed as 40(DEC) or 64(HEX)