Friday, May 30, 2014

Quickly create large files in python (fast disk space allocation)

Quickly create large files in python (fast disk space allocation)

Sometimes you want quickly allocate some disk space to a certain file, without worrying about what the content is. Here are two ways to do it in python (in this case I just want to test whether os.stat handles large files correctly, and it turns out to be fine):

Double click to toggle code
import os

with open("newfile", "wb") as f:
    f.seek(1024 * 1024 * 1024 * 40 - 1)
    f.write(b"\0")
os.stat("newfile").st_size

with open("newfile1", "wb") as out:
    out.truncate(1024 * 1024 * 1024 * 40)
os.stat("newfile1").st_size

In some file systems, the first method (seek and write) may give you file appearing to be 40G, but actually only occupying 1 byte on disk, this is called a sparse file system.

0 comments: