Python -An enhanced script for ShareInvestor price data and Metastock

Hi, I have implemented a few more things on the previous script. The script now do the following:

1. It auto downlaod the historical SGX price data from the ShareInvestor Website
2. It extracts the data to the correct the folder.
3. It fixed the problem with the ticker symbol.
4. It launches the off-line mode of metastock.
5. For interest, it shows the time take to do the ticker replacement and it also state the total time taken to download the file and launching metastock

The script is as follows (until I get my github setup, it will be stored here):

#_*_ coding : UTF-8 _*_# coder    : MK Yuen# code time: 06-Aug-19 11:30 AM# file name: Read all files in shareinvestor.PY# Py ver   : PyCharm
# the problem with shareinvestor data is that the ticker symbol is being read by MS to be the name of the share# as a result, when we run explorer, we get a list of symbols which we need to refer before we know the stock# this is time consuming
# for this program, i use os to join path, I believe I can simply use string to do it, but I did not try# glob is needed to limit the files to csv files though I do not think there are any other files in the folder# csv is used for reading and writing the csv file

import time
import os
import glob
import csv
import subprocess
from zipfile import ZipFile
import webbrowser

first_start = time.time()
# this is to download the historical file and wait until it completewebbrowser.open('http://www.shareinvestor.com/prices/price_download_zip_file.zip?type=history_all&market=sgx')
#r is needed to convert string to raw stringzipfileDir = r"c:\Users\User\Downloads\history_sgx_all.zip"# wait until file is readywhile not os.path.exists(zipfileDir):
    time.sleep(60)
print("historical file downloaded")
# first I define the fields to be used and the field to be replaced# this time exclude download timestart = time.time()
folder_path = 'C:\Metastock Data\ShareInvestor Local Data'key_to_be_replace = 'ticker'key_to_be_duplicated = 'name'
with ZipFile(zipfileDir, 'r') as zipObj:
    # Extract all the contents of zip file into local data directory    zipObj.extractall(folder_path)
# delete the fileos.remove(zipfileDir)
print("file extracted")
# loop through all files with .csvfor filename in glob.glob(os.path.join(folder_path, '*.csv')):
    # open the file first    with open(filename,  'r') as f:
        # actually do i still need variable reader, can i use info_list  list(csv.reader(f) ?        # change the csv file into a list        reader = csv.reader(f)
        info_list = list(reader)
        # the purpose is to skip the field row in the csv, alternatively i can use a boolean variable        counter = 0        for row in info_list:
            if counter != 0:
                info_list[counter][0] = info_list[counter][1]
                counter +=1            else:
                counter +=1        f.close()
    # newline is essential, otherwise writer will generate a blank line    with open(filename, 'w',newline='') as writeFile:
        # make the file to be ready for csv write        writer = csv.writer(writeFile)
        # write each row of list into csv file, note that the method is writerows() not writerow()        writer.writerows(info_list)
    writeFile.close()
    # this is to provide user info on status of write    print(filename)
end = time.time()
print("ticker symbol replaced")
print ("Time for execution is ", end - start, "seconds")
print ("Time include download is ", end - first_start, "seconds")
# this launch metastock off line modeargs = [r'C:\Users\User\AppData\Local\Thomson Reuters\Eikon\Program\Mswin.exe', '-offline']
subprocess.call(args)
end = time.time()
p

It sure saves a lot of time and I have gained much more insight.

I will take a break as I need to do the change management study now.

After which, I will use Python for data analytics.

Until then, see you....