Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Unicode error
#1
I was looking for a python script that will convert hst fils to csv and found it at
https://github.com/nigelsim/mt4reader

Just added file names to the following lines

def main():
parser = argparse.ArgumentParser("Convert HST to CSV")
parser.add_argument('in_file', help="C:\Users\Biju\Desktop\Python\HST\CANBK5.hst")
parser.add_argument('out_file', help="C:\Users\Biju\Desktop\Python\HST\a.csv")


and tried to run it from my ide.
But I am getting an error.
Please help.
I have attached the HST file in here. The HST file is taken from metatrader.

The complete code I am using is the following

# Utility library for loading MT4 HST files
# Based on notes gathered from:
# - http://wejn.org/stuff/mt-hist-info.rb.html
# - http://www.forex-tsd.com/metatrader-4/8116-how-open-hst-file.html#post124232
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
import struct
import datetime
import argparse
import sys
import csv
import numpy


class Bar(object):
    def __init__(self, x):
        if len(x) != 44:
            raise ValueError('Not a full record')
        t, o, l, h, c, v = struct.unpack('<iddddd', x)

        self.time = datetime.datetime.utcfromtimestamp(t)
        self.open = o
        self.low = l
        self.high = h
        self.close = c
        self.volume = v

    def __repr__(self):
        return '%s O:%f L:%f H:%f C:%f' % (self.time, self.open, self.low,
                                           self.high, self.close)

    def __str__(self):
        return '%s O:%f L:%f H:%f C:%f' % (self.time, self.open, self.low,
                                           self.high, self.close)


class History(object):
    def __init__(self, f):
        version, copyright, symbol, period, digits, timesign, lastsync = \
            struct.unpack('<i64s12siiii', f.read(96))
        self.copyright = copyright
        self.symbol = symbol
        self.period = period
        self.digits = digits
        self.timesign = datetime.datetime.utcfromtimestamp(timesign)

        # skip
        f.seek(13*4, 1)
        self.bars = []

        while True:
            x = f.read(44)
            if len(x) != 44:
                break
            self.bars.append(Bar(x))


def convert(in_file, out_file):
    """
    Converts from HST to CSV
    """
    with open(in_file, 'rb') as f:
        history = History(f)

    with open(out_file, 'w') as f:
        writer = csv.writer(f)
        writer.writerow(['time', 'open', 'close', 'high', 'low', 'volume'])
        for b in history.bars:
            writer.writerow([b.time, b.open, b.close, b.high, b.low, b.volume])


def main():
    parser = argparse.ArgumentParser("Convert HST to CSV")
    parser.add_argument('in_file', help="C:\Users\Biju\Desktop\Python\HST\CANBK5.hst")
    parser.add_argument('out_file', help="C:\Users\Biju\Desktop\Python\HST\a.csv")

    args = parser.parse_args()

    convert(args.in_file, args.out_file)

if __name__ == '__main__':
    sys.exit(main())
Reply
#2
Do no use single \ in path,because can be looked at as escape characters.
>>> s = 'C:\Users'
  File "<interactive input>", line 1
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape

>>> # Fix
>>> s = r'C:\Users'
>>> s
'C:\\Users'

# Or
>>> s = 'C:/Users'
>>> s
'C:/Users'
Reply
#3
parser.add_argument('in_file', help='C:\\Users\Biju\Desktop\Python\HST\CANBK5.hst')
parser.add_argument('out_file', help='C:\\Users\Biju\Desktop\Python\HST\a.csv')
Tried the above code ...changed it to single quotes and and double slash. Not working

parser.add_argument('in_file', help='C:/Users/Biju/Desktop/Python/HST/CANBK5.hst')
parser.add_argument('out_file', help='C:/Users/Biju/Desktop/Python/HST/a.csv')
This too not working
Reply
#4
Post Traceback of error,and are using Python 2 or 3?
My first suggestion was just to add r raw string.
>>> s = r'C:\Users\Biju\Desktop\Python\HST\CANBK5.hst'
>>> s
'C:\\Users\\Biju\\Desktop\\Python\\HST\\CANBK5.hst' 
Reply
#5
parser.add_argument('in_file', help='C:\\Users\\Biju\\Desktop\\Python\\HST\\SBIN1.hst')
parser.add_argument('out_file', help='C:\\Users\\Biju\\Desktop\\Python\\HST\\a.csv')
parser.add_argument('in_file', help=r'C:\\Users\\Biju\\Desktop\\Python\\HST\\SBIN1.hst')
parser.add_argument('out_file', help='C:\\Users\\Biju\\Desktop\\Python\\HST\\a.csv')
None working.

The error I am getting is
usage: Convert HST to CSV [-h] in_file out_file
Convert HST to CSV: error: the following arguments are required: in_file, out_file

using python 3.7.3

parser.add_argument('in_file', help='C:\Users\Biju\Desktop\Python\HST\SBIN1.hst')
parser.add_argument('out_file', help='C:\Users\Biju\Desktop\Python\HST\a.csv')
When I use the last code I get unicode error.
Reply
#6
Thank you snippsat for your time.

I was messing with the code from https://mechanicalforex.com/2015/12/conv...cript.html
and i got it to work for me.

Cheers
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  python-resize-image unicode decode error Pedroski55 3 3,471 Apr-21-2020, 10:56 AM
Last Post: Pedroski55
  clean unicode string to contain only characters from some unicode blocks gmarcon 2 3,972 Nov-23-2018, 09:17 PM
Last Post: Gribouillis
  tf.gfile.FastGFile error unicode ( japanese characters ) majinbuu 2 3,081 May-13-2018, 02:11 PM
Last Post: majinbuu

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020