Nimendra's Notes 🪴

Go - File Handling & IO Operations

4 min read

go go/book/learninggo

Golang Bufio (A Complete Guide)

A comprehensive guide to file operations in Go - Honeybadger Developer Blog

File Handling and I/O Operations in Go

File handling and Input/Output (I/O) operations in Go are primarily done through the os and io packages, with additional utilities provided by packages like bufio and io/ioutil.

1. Basic File Operations

  • Open/Read/Write Files: The os package provides core functions for file handling.
  • Buffered I/O: The bufio package wraps basic file handling for efficient reading and writing.
  • Working with Strings/Streams: The io package provides abstractions for streams of data.
  • Convenience Functions: The os and ioutil packages have helpers for quick file reading/writing.

2. Common File Operations

Opening a File

file, err := os.Open("example.txt")
if err != nil {
    log.Fatal(err)
}
defer file.Close()

Creating a File

file, err := os.Create("newfile.txt")
if err != nil {
    log.Fatal(err)
}
defer file.Close()

Reading a File

  • Using os
file, err := os.Open("example.txt")
if err != nil {
    log.Fatal(err)
}
defer file.Close()
 
data := make([]byte, 100) // Buffer to hold data
n, err := file.Read(data)
if err != nil && err != io.EOF {
    log.Fatal(err)
}
fmt.Printf("Read %d bytes: %s\n", n, string(data[:n]))
  • Using ioutil
data, err := ioutil.ReadFile("example.txt")
if err != nil {
    log.Fatal(err)
}
fmt.Println(string(data))

Writing to a File

  • Using os
file, err := os.Create("output.txt")
if err != nil {
    log.Fatal(err)
}
defer file.Close()
 
_, err = file.WriteString("Hello, World!")
if err != nil {
    log.Fatal(err)
}
  • Using ioutil
err := ioutil.WriteFile("output.txt", []byte("Hello, World!"), 0644)
if err != nil {
    log.Fatal(err)
}

Appending to a File

file, err := os.OpenFile("output.txt", os.O_APPEND|os.O_WRONLY, 0644)
if err != nil {
    log.Fatal(err)
}
defer file.Close()
 
_, err = file.WriteString("\nAppended text")
if err != nil {
    log.Fatal(err)
}

3. Buffered I/O

Buffered I/O is a technique that allows a program to read or write data in chunks rather than one byte at a time. This is useful because it allows the program to read or write data more efficiently. It also allows the program to read or write data more predictably.

In Go, this is done by using the bufio package. This package provides buffered readers and writers.

Reading with bufio

file, err := os.Open("example.txt")
if err != nil {
    log.Fatal(err)
}
defer file.Close()
 
scanner := bufio.NewScanner(file)
for scanner.Scan() {
    fmt.Println(scanner.Text())
}
 
if err := scanner.Err(); err != nil {
    log.Fatal(err)
}

Writing with bufio

file, err := os.Create("buffered.txt")
if err != nil {
    log.Fatal(err)
}
defer file.Close()
 
writer := bufio.NewWriter(file)
_, err = writer.WriteString("Buffered write\n")
if err != nil {
    log.Fatal(err)
}
writer.Flush() // Ensure data is written to disk

4. I/O Operations with io

  • The io.Reader and io.Writer interfaces define the standard input and output streams.

Copying Data

srcFile, err := os.Open("source.txt")
if err != nil {
    log.Fatal(err)
}
defer srcFile.Close()
 
dstFile, err := os.Create("destination.txt")
if err != nil {
    log.Fatal(err)
}
defer dstFile.Close()
 
_, err = io.Copy(dstFile, srcFile)
if err != nil {
    log.Fatal(err)
}

Reading Strings

reader := strings.NewReader("Hello, Go!")
data := make([]byte, 5)
n, err := reader.Read(data)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Read %d bytes: %s\n", n, data)

5. Error Handling in File I/O

  • Always check for errors when performing file operations.
  • Use defer to ensure files are closed properly, avoiding resource leaks.
  • Handle EOF (end-of-file) explicitly when reading files.

6. Working with Directories

  • Creating a Directory
err := os.Mkdir("newdir", 0755)
if err != nil {
    log.Fatal(err)
}
  • Removing a File or Directory
err := os.Remove("example.txt")
if err != nil {
    log.Fatal(err)
}
  • Walking Through a Directory
err := filepath.Walk(".", func(path string, info os.FileInfo, err error) error {
    if err != nil {
        return err
    }
    fmt.Println(path)
    return nil
})
if err != nil {
    log.Fatal(err)
}

7. Permissions

File permissions are controlled by os.FileMode values, such as 0644 (readable by all, writable by the owner).

8. Best Practices

  • Use defer to close files.
  • Buffer I/O for large file operations.
  • Use ioutil or os helpers for quick operations.
  • Handle errors gracefully, especially for permissions and missing files.
  • Be cautious with concurrent file operations; use synchronization mechanisms like mutexes when needed.

Bufio vs I/O

The main difference between buffered I/O and normal I/O is that buffered I/O reads or writes data in chunks rather than one byte at a time. While on the other side normal I/O reads or writes data one byte at a time. This might not seem like a big difference but it can make a big difference in performance.

Backlinks

Graph View