Friday, November 16, 2018

Learn Golang Tutorials - File Directory IO operations Guide with examples

Golang File Operations

In Any programming language, File IO operations are basic functionality.  These operations are low-level calls made to underline operating System to operate physical files.
Go Language also provides many operations as part of standard packages. All these operations accept input, process it and output to or from physical files.

There are many standard inbuilt packages that we are going to use in this post
  • bufio package  It contains functions
  • IOUtil package  has functions for Input and Output operations
  • os package provides platform independent functionalities for Operating System.
Here are important notes of a File in any programming language
  • A file is a  collection of data stored under a computer machine.
  • Each file contains the name and path of the file 
  • Filename contains the actual name of the file suffixed with the file extension. The extension is a type the file.
  • The path is a  file location in the underline operation System file system. The path can be the absolute or relative path
  • The absolute path is a path from the Root folder to file location. This is specific to Operating System.
  • A relative path is a path which is relevant from the current directory.
  • In Operation System, Filename is unique in Directory.

Examples

Following are the basic examples that you need to know when you are dealing with go language files.

Empty File Creation

This is a simple and easy example of creating an empty file.
In the below example, Standard os package provides create a function. 
Syntax is Create(path) (*File, error)
It accepts path. the path can be an absolute path or relative path. This creates  a physical named file if already exits the same name, truncate it 
This file returns two parameters, First, one is returned file reference to this file for IO operations. second is an error if any exists on creating a file.
The file is created with default permission mask 0666
package main
import (
 "log"
 "os"
)
func main() {
 emptyFile, error := os.Create("EmptyFile.txt")
 if error != nil {
  log.Fatal(error)
 }
 log.Println(emptyFile)
 emptyFile.Close()
}
a log is a standard inbuilt in basic logging package. And function Fatal is equal to Println+ os.exit.

Copy a files

Copy of files will transfer the file and its content from source to destination
If you want to implement copy functionality, Following things need to be done
  • Open a source file for reading using os.open() function
  • Read the content of a file using in memory or  buffer, In
  • Create a new file in the target directory with the same name using os.OpenFile with O_RDWR|os.O_CREATE for Create, Read and Write permissions
  • Write the source file content to target file using io.copy() method
  • Finally close source and target files using defer reader/writer.close()
Following is an example of copying a file
import (
 "io"
 "log"
 "os"
)

func main() {
 source, sourceError := os.Open("source.txt")
 if sourceError != nil {
  log.Fatal(sourceError)
 }
 defer source.Close()

 target, targetError := os.OpenFile("target.txt", os.O_RDWR|os.O_CREATE, 0666)
 if targetError != nil {
  log.Fatal(targetError)
 }
 defer target.Close()
 _, copyError := io.Copy(target, source)
 if copyError != nil {
  log.Fatal(copyError)
 }
}
and the result is source.txt file is copied to target.txt

File metadata information - Name,size,permissions, Date information 

package Os function Stat() returns FileInfo object.FileInfo contains metadata information of a file. It contains the name of the file, Size of a file in bytes, File Permissions, Last Modified date and time, file type -directory|file, System process information Sys() is an interface which gives underlying System process information
package main
import (
 "log"
 "os"
)

func main() {
 var file os.FileInfo
 file, error := os.Stat("testfile.txt")
 if error != nil {
  log.Fatal(error)
 }
 log.Println("Name of the File:", file.Name())
 log.Println("Size of File:", file.Size())
 log.Println("Permissions File:", file.Mode())
 log.Println("Last Modified of File:", file.ModTime())
 log.Println("Directory check: ", file.IsDir())
 log.Printf("System Process info: %+v\n\n", file.Sys())
}
An output of the above code is
2018/11/16 12:30:33 Name of the File: testfile.txt
2018/11/16 12:30:33 Size of File: 17
2018/11/16 12:30:33 Permissions File: -rw-rw-rw-
2018/11/16 12:30:33 Last Modified of File: 2018-11-16 12:11:10.3375944 +0530 IST
2018/11/16 12:30:33 Directory check:  false
2018/11/16 12:30:33 System Process info: &{FileAttributes:32 CreationTime:{LowDateTime:200182414 HighDateTime:30702958} LastAccessTime:{LowDateTime:200182414 HighDateTime:30702958} LastWriteTime:{LowD
High:0 FileSizeLow:17}

Rename or move a file name to new name

This is useful when you want to rename a file or move a file from one location to other location. without rename functionality, We need to delete the file and create a file with a new name. The syntax is os.Rename(oldpath,newpath) Rename function accepts oldpath and newpath. It returns an error if any error exists.
import (
 "log"
 "os"
)

func main() {
 Filename := "testfile.txt"
 NewFileName := "testing.txt"
 error := os.Rename(Filename, NewFileName)
 if error != nil {
  log.Fatal(error)
 }
}
The output of the above program is nothing, Buth the testfile.txt is renamed to testing.txt 
In the above program Filename and NewFileName are a path including file names. If it is in a different directory, Output of rename command outputs to a new location with renamed file.

Delete a file 

It is very simple to delete a file from a file system. Inbuilt OS package provides Remove function Syntax is fs.Remove(path) path is the name of the file or directory. Directory always starts with /.
import (
 "log"
 "os"
)

func main() {
 err := os.Remove("testing.txt")
 if err != nil {
  log.Fatal(err)
 }
}
The output is testing.txt file will be removed from the current directory. if any error occurs, It returns Error To delete files from a directory, Iterate all the files in the directory and apply os.Remove() function.

truncate - Reduce file size 

function truncate  reduce file size by specified bytes.
The syntax is os.Truncate(path, bytes) 
the path is the name of the file including the absolute or relative path.
a second parameter is a number of reduced bytes of a file size if the name of the file in the path not exists, It creates an empty file with this. In the below code, If filename.txt is 1024 bytes in size, It truncates the file size to 200 if the size of filename.txt content is less than 200, It keeps its original size if the second parameter is zero, It removes content from the file and makes an empty file
package main
import (
 "log"
 "os"
)
 
func main() {
 err := os.Truncate("filename.txt", 200)
 
 if err != nil {
  log.Fatal(err)
 }
}

check if a File exists 

When we are doing any operations on files, the First requirement checks is to check file exists or not. Standard os package provides Stat() function to get File meta information, If file not exists, Returns an error. The error can be of different errors, We need to check whether these errors are related to file or directories exists IsNotExist() function returns true if the error is related to not exists.
import (
 "log"
 "os"
)

func main() {
 _, err := os.Stat("testfile.txt")
 if err == nil {
  log.Fatal(err)
 }
 if os.IsNotExist(err) {
  log.Fatal("Not Exists")
 }
}

Open and close files 

For reading /writing a file, First We need to open the file. Standard package os provides functions to open a file for reading or write a content. There are two ways we can do it Synta ix
os.Open(filename String) (*File, error) 
Parameters accepted is a name of the file with type string. Which opens a file for reading and default file descriptor O_RDONLY Returns multiple parameters - *File and error- return *File and error null,if file is successfully opened, if error, returns *PathError Here is the example code for opening
file, err := os.Open("filename.txt")
 if err != nil {
  log.Fatal(err)
 }
if the file opens throws error at runtime, The error is open First.go: no such file or directory
func OpenFile(filename string, flag int, perm FileMode) (*File, error)
The parameters are
filename is the name of the file
The flag is of type int which contains the following constant values. This will tell the purpose of the file to do operations.

  • os.O_RDONLY - For Reading only
  • os.O_WRONLY - Write only
  • os.O_RDWR - both Read and write
  • os.O_APPEND Concatenate at end of file
  • os.O_CREATE - Create a file if not exist
  • os.O_TRUNC -Truncate file to reduce the size
We can supply single parameter os.O_RDONLY or multiple parameters separated by | ie os.O_CREATE |os.O_RDWR last parameter is permissions which is of type FileMode Here is the example code for an opening file with various options. the program is to open the file filename.txt for appending the content to end of the file with permissions - 0777
file, err := os.OpenFile("filename.txt",os.O_APPEND, 0777)
 if err != nil {
  log.Fatal(err)
 }
File closing 
when you are dealing with a file, You are doing input and output operations hardware disk. It is an expensive operation in terms of performance. It is always advisable to close the file once done. we can use simple close or can be combined with defer keyword to close file before returning data in a function
file.close() // simple closing  
or 
defer file.close() // this will be closed before returning data from a function

Append content to end of a File 

First Open the file with OpenFile with various options, Here used OpenFile function with O_APPEND to open a file for appending String content once the file is opened, Append data to a file using WriteString of a file. This will create a file if not exist and append the string Close the file once done
import (
 "log"
 "os"
)

func main() {
 file, error := os.OpenFile("employee.log", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0777)
 if error != nil {
  log.Fatal(error)
 }
 _, error1 := file.WriteString("1 Kiran 2000\n")
 if error1 != nil {
  log.Fatal(error1)
 }
 if error := file.Close(); error != nil {
  log.Fatal(error)
 }

}
The output of the above program is an employee.log file not exist before, a new file is created with below content 1 Kiran 2000

Write to a file

 It is a basic operation to write a content into a file. ioutil package provides WriteFile function which accepts filename, string content in bytes, permissions.
import (
 "io/ioutil"
 "log"
)

func main() {
 strBytes := []byte("Hi,\nWelcome to Go Tutorials\n")
 err := ioutil.WriteFile("samplefile.txt", strBytes, 0666)
 if err != nil {
  log.Fatal(err)
 }

}
The output is the samplefile.txt file is created and content is as follows
Hi,
Welcome to Go Tutorials

File read line by line

 Normally, reading file transfer content to in-memory. bufio package scanner functions provide for the reading line at a time here is samplefile.txt content
Hi,
Welcome to Go Tutorials
Here is a sequence of steps

  • open a file using os.open() function 
  • Create scanner on file reader using bufio.NewScanner() function 
  • read line by line using a scanner.Scan(). scan function read a line at a time

import (
 "bufio"
 "fmt"
 "log"
 "os"
)

func main() {
 fileReader, error := os.Open("samplefile.txt")
 if error != nil {
  log.Fatal(error)
 }
 defer fileReader.Close()

 scanner := bufio.NewScanner(fileReader)
 for scanner.Scan() {
  fmt.Println(scanner.Text())
 }
 error1 := scanner.Err()
 if error1 != nil {
  log.Fatal(error1)
 }
}
Outputs file content line by line into the console.

Golang Directory Operations 

Directory alias is a folder which contains various types of files and nested directories. This will be helpful to organize the files and directory in the Operating System.
Following are various directory input-output operations.

Create a directory if not exists 

we have to use standard package os to create director or folder First os.Stat() method checks meta information of a file or directory, if already exists it returns an error as nil value, do nothing Once the check is completed, we have to call os.MkdirAll(path, permission) to create a directory with given permissions. MkdirAll is like mkdir command in Unix.
func main() {
 _, error := os.Stat("directory")
 if error == nil {
  log.Fatal(error)
 }
 if os.IsNotExist(error) {
  errorDir := os.MkdirAll("directory", 0755)
  if errorDir != nil {
   log.Fatal(error)
  }

 }
}
Outputs the directory is created in the current folder.
Creation of simple directory works as expected when there are nested directories, like directory\\dir2 code also works in windows, but not on other platforms Operating System. The reason is path contains \\ which will not work in UNIX Operation System. The above code is not feasible for platform dependent and nested directory creation.

Creating Nested directories 

if we want to create a nested directory like dir1/dir2 in a current directory, we have to use os.MkdirAll with path and permissions. Path object needs to be created using path/filepath package. Join function in filepath joins all the pats into a single path. It also picks right separator based on the platform
import (
 "log"
 "os"
 "path/filepath"
)

func main() {
 var nestedDir = "kiran/kiran2"
 path := filepath.Join(".", nestedDir)
 error := os.MkdirAll(path, 0777)
 if error != nil {
  log.Fatal(error)
 }

}
The output is Create a directory dir2 under dir1.

List files from a directory

It is a common requirement to list out files and directories of a current directory. package ioutil ReadDir function returns a list of files in sorted order
import (
 "fmt"
 "io/ioutil"
 "log"
)

func main() {
 files, error := ioutil.ReadDir(".")
 if error != nil {
  log.Fatal(error)
 }
 for _, file := range files {
  fmt.Println(file.Name())
 }

}
Output is
EmptyFile.go
EmptyFile.txt
FileInfo.properties

Current working directory print to console

package os provides GetWd() function returns the working director starting from root directory to the current directory
import (
 "fmt"
 "log"
 "os"
)

func main() {
 currentWorkingDirectory, error := os.Getwd()
 if error != nil {
  log.Fatal(error)
 }
 fmt.Println(currentWorkingDirectory)
}
Output returns current working directory absolute path

walk files and directories 

The directory contains files and subdirectory tree. Sometimes It is required to visit each file and subdirectories of the directory. filepath.Walk function is being used here. It accepts directory and custom visit function parameters.
In the below program, Created an anonymous function that is passed Walk function which iterates each file and directory in the directory tree.
import (
 "fmt"
 "log"
 "os"
 "path/filepath"
)

func main() {
 directory := "."
 customVisit := func(path string, fileInfo os.FileInfo, err1 error) error {
  if err1 != nil {
   return err1
  }
  fmt.Println(path, fileInfo.Size())
  return nil
 }
 error := filepath.Walk(directory, customVisit)
 if error != nil {
  log.Println(error)
 }

}
Output is listing out all files and subdirectories recursively to console

Related article


EmoticonEmoticon