How to Read Text Files in Python and Convert Into Int Arrays

A professional engineer and writer helping people find creative ways to solve everyday bug.

how-to-read-data-from-a-file-in-python — Copyrights © Ali Khan (Author)

This article volition provide you lot a detailed walk-through on how yous can use Python to read large text files. A fully functional, prepare to execute code snippet is included in this walk-through to become you lot up to speed in 10 minutes subsequently reading this article.

Let u.s. first familiarize you with the avant-garde information-structures available in Python that we will use to store and process data from files, only in example you are new to Python programming.

Advanced Data-Structures in Python

Python has two advanced and powerful data structures that make it superior in functionality to C/C++ and make information technology an ideal language for numerical data intensive applications competing with industry dominant Matlab®.

ndarrays and dataframes

Adjusted from Numpy and Pandas opensource documentation

NumPy Arrays

Numpy arrays or ndarrays are arrays that can be scaled up to 'n' dimensions. They are best used as 2-dimensional array structures to represent matrices. The Numpy module itself contains powerful part libraries for a diverseness of numerical and algebraic operations.

Pandas Data-Frames

Data-frames build upon the 2-dimensional ndarrays to add extra functionality. The 2-dimensional ndarray now has a separate column for array index and all column headers are at present individually addressable. More importantly, each column can now agree a different information type (int, float or string).

Text File to exist Read by Python Code

Let'due south move forward to the tutorial and acquaint you with the sit-in text file.

It is a 14 rows x xx column data table saved as a txt file. Information technology contains data in all three data formats: int, float and cord. File name is: BusData.

Fig 1: Data file that is to be parsed with our code.

IEEE 14-Bus Organisation CDF file

Next, view the code snippet given beneath, to read this file and we will explain this code line past line in the following section.

Python Code to Read Data From Text File

                                                            # Copyrights © Ali Khan (Writer)                                                                              # Permitted to utilise with attribution                                                                                                                          import                  numpy                                                              import                  pandas                                                                                                                              def                    Read                    ():                                                                                                                                          global                  BusData, BusDataList, BusDataArray, BusDataReshaped                                                                                                                          X =                  open up('C:/Users/user/OneDrive - Washington State University (email.wsu.edu)/EE - 521/BusData.txt',                  'r')                                                              BusData = X.read()                                                                                                                                          BusDataList = BusData.dissever()                                                              BusDataArray = numpy.array(BusDataList)                                                              BusDataReshaped = BusDataArray.reshape(14,20)                  #Brand a matrix out of 1D Array                                                                                                        Read()                                                                                        BusDataFrame = pandas.DataFrame(BusDataReshaped, columns =['BusNumber',                  'Omnibus',                  'Busx',                  'BusClass',                  'Unused1',                  'Unused2',                  'BusType',                  'Unused3',                  'Unused4',                  'BusLoadMW',                  'BusLoadMVAR',                  'BusGenMW',                  'BusGenMVAR',                  'InitialVoltAngle',                  'InitialVolt',                  'Qlimit+',                  'Qlimit-',                  'Unused5',                  'Shunt',                  'Unused'])                                                                                                          print(BusDataFrame.BusClass[6])

Code Explanation

Initialization: Import Numpy and Pandas

Line iv: Import the numpy package in the project.

Line v: Import the pandas parcel in the project.

Line 7: Start a function definition Read(). It is always a good practise to break your code in functions.

Line ix: Ascertain global variables.

Read More From Owlcation

In Python only global variables will appear in the variable explorer and can be referenced outside functions. For demonstration, hither I have defined all four equally global, otherwise only BusDataReshaped variable should take been declared global.

Fig 2: Spyder variable explorer showing details of every variable.

Copyrights © Ali Khan (Author)

Open and Read the Target File

Line 11: The open() function points to the directory location of file BusData.txt. Definition is assigned to random variable X.

Line 12: read() function reads the entire file as a cord and assigns it to variable BusData. Fig 2 shows that BusData is at present a cord with 1792 characters.

Split the File Graphic symbol-wise

Line 14: split() function in Python, splits the string into a list at the points where their is infinite. The data is now converted into a list of 280 elements and assigned to variable BusDataList. Reference Fig 2.

Catechumen to Numpy Array

Line fifteen: The listing is converted into a numpy array past the numpy.assortment() part. Fig ii shows that BusDataArray is at present an array of datatype string and has 280 elements.

The problem is that it still does not look like our original data table in the file. It needs to be reshaped.

Line sixteen: The numpy.reshape() function from the numpy package reshapes the assortment into our desired dimensions of 14 x twenty. Fig 2 shows that BusDataReshaped variable is now an ndarray and has dimensions 14 x 20.

Equally we can run across, the data type of all the values is still cord, but remember, the original file had integers and floats in the information as well. To make sure that all the information is treated according to its correct data type we demand to convert it into a Pandas Data-frame.

Fig 3: BusDataFrame, showing its first 13 columns in variable explorer — Fig 3: BusDataFrame, showing its commencement 13 columns in variable explorer

Copyrights © Ali Khan (Author)

Converting a Numpy Array to Information-frame

Line 20: line xx finally does the task of converting an array of strings into a pandas dataframe.

Pandas.dataframe() office takes the reshaped numpy array, and the names of all xx column headers as inputs. Fig 3 shows the formed dataframe and Fig 2 verifies this in the variable explorer.

Fig 4: Print result — Fig iv: Impress result

Referencing Values of a Pandas Dataframe

Line 22: Values of this dataframe tin can exist very conveniently accessed by the dataframe.columnheader.[index] syntax.

A check of variable types will show that all the iii information types of string, integer and floats of each column are automatically preserved by the dataframe.

This content is accurate and truthful to the all-time of the author's knowledge and is not meant to substitute for formal and individualized advice from a qualified professional.

clarketapeon50.blogspot.com

Source: https://owlcation.com/stem/How-to-Read-Data-From-a-File-in-Python