pyarrow.LocalFileSystem

class pyarrow.LocalFileSystem

Bases: pyarrow.filesystem.FileSystem

__init__()

Initialize self. See help(type(self)) for accurate signature.

Methods

cat(path)	Return contents of file as a bytes object
delete(path[, recursive])	Delete the indicated file or directory
disk_usage(path)	Compute bytes used by all contents under indicated path in file tree
exists(path)
get_instance()
isdir(path)	Return True if path is a directory
isfile(path)	Return True if path is a file
ls(path)	Return list of file paths
mkdir(path[, create_parents])
mv(path, new_path)	Alias for FileSystem.rename
open(path[, mode])	Open file for reading or writing
read_parquet(path[, columns, metadata, …])	Read Parquet data from path in file system.
rename(path, new_path)	Rename file, like UNIX mv command
rm(path[, recursive])	Alias for FileSystem.delete
stat(path)	returns:stat (dict)
walk(path)	Directory tree generator, see os.walk

Attributes

pathsep

cat(path)

Return contents of file as a bytes object

Returns:contents (bytes)

delete(path, recursive=False)

Delete the indicated file or directory

Parameters:

• path (string) –
• recursive (boolean, default False) – If True, also delete child paths for directories

disk_usage(path)

Compute bytes used by all contents under indicated path in file tree

Parameters:path (string) – Can be a file path or directory Returns:usage (int)

exists(path)

classmethod get_instance()

isdir(path)

Return True if path is a directory

isfile(path)

Return True if path is a file

ls(path)

Return list of file paths

mkdir(path, create_parents=True)

mv(path, new_path)

Alias for FileSystem.rename

open(path, mode='rb')

Open file for reading or writing

pathsep

read_parquet(path, columns=None, metadata=None, schema=None, use_threads=True, use_pandas_metadata=False)

Read Parquet data from path in file system. Can read from a single file or a directory of files

Parameters:

• path (str) – Single file path or directory
• columns (List[str], optional) – Subset of columns to read
• metadata (pyarrow.parquet.FileMetaData) – Known metadata to validate files against
• schema (pyarrow.parquet.Schema) – Known schema to validate files against. Alternative to metadata argument
• use_threads (boolean, default True) – Perform multi-threaded column reads
• use_pandas_metadata (boolean, default False) – If True and file has custom pandas schema metadata, ensure that index columns are also loaded

Returns:

table (pyarrow.Table)

rename(path, new_path)

Rename file, like UNIX mv command

Parameters:

• path (string) – Path to alter
• new_path (string) – Path to move to

rm(path, recursive=False)

Alias for FileSystem.delete

stat(path)

Returns:stat (dict)

walk(path)

Directory tree generator, see os.walk