File Operations

The file operations module provides utilities for file manipulation, copying, moving, and general file system operations.

Module Overview

Modern file operations for siege_utilities. Provides clean, type-safe file manipulation utilities.

siege_utilities.files.operations.sanitize_filename(name)[source]

Convert an arbitrary string into a safe filename component.

Unsafe characters (path separators, control characters, punctuation other than . and -) are replaced with underscores, runs of underscores are collapsed, and leading/trailing dots and underscores are trimmed. The result is always a non-empty string and the operation is idempotent (sanitize_filename(sanitize_filename(x)) == sanitize_filename(x)).

Parameters:: name (str) – Arbitrary input string.
Returns:: A filesystem-safe filename component.
Return type:: str
Raises:: TypeError – If name is not a string.

siege_utilities.files.operations.atomic_write_path(target)[source]

Context manager that yields a temporary path for writing, then atomically replaces the target on successful exit.

Usage:

with atomic_write_path("/data/output.csv") as tmp:
    df.to_csv(tmp, index=False)
# target is now atomically updated

If writing or syncing the file raises, the temp file is cleaned up and the target is left unchanged. Files use mode 0666 filtered by the process umask, including replacements. This is for data, not credentials: use secure_write_text for secrets. Flush and close writers before exit.

Parameters:: target (str | Path)
Return type:: Generator[Path, None, None]

siege_utilities.files.operations.atomic_write_shapefile(gdf, target, **kwargs)[source]

Write a GeoDataFrame as a Shapefile atomically.

Shapefiles produce sidecar files (.shx, .dbf, .prj, .cpg). This writes to a temp stem then renames all produced files to the final stem.

Parameters:: target (str | Path)
Return type:: None

siege_utilities.files.operations.remove_tree(path)[source]

Remove a file or directory tree safely.

SECURITY: Validates paths to prevent removing sensitive system files.

Parameters:

path (str | Path) – Path to file or directory to remove

Raises:

FileNotFoundError – If path does not exist
PathSecurityError – If path fails security validation
OSError – If removal fails

Return type:

None

Example

>>> remove_tree("temp_directory")
>>> remove_tree(Path("old_files"))

siege_utilities.files.operations.file_exists(path)[source]

Check if a file exists at the specified path.

SECURITY: This function validates paths to prevent path traversal attacks and access to sensitive system files.

Parameters:: path (str | Path) – Path to check
Returns:: True if file exists, False otherwise
Raises:: PathSecurityError – If path fails security validation
Return type:: bool

Example

>>> if file_exists("config.yaml"):
...     print("Config file found")
>>> file_exists("../../../etc/passwd")

Security Changes:

Now validates paths to block path traversal
Blocks access to sensitive system files

siege_utilities.files.operations.touch_file(path, create_parents=True)[source]

Create an empty file, creating parent directories if needed.

SECURITY: Validates paths to prevent path traversal attacks.

Parameters:

path (str | Path) – Path to the file to create
create_parents (bool) – Whether to create parent directories

Raises:

PathSecurityError – If path fails security validation
OSError – If file creation fails

Return type:

None

Example

>>> touch_file("logs/app.log")
>>> touch_file(Path("data/output.txt"))

siege_utilities.files.operations.count_lines(file_path, encoding='utf-8')[source]

Count the total number of lines in a text file.

SECURITY: This function validates paths to prevent path traversal attacks and access to sensitive system files.

Parameters:

file_path (str | Path) – Path to the text file
encoding (str) – File encoding to use

Returns:

Number of lines

Raises:

FileNotFoundError – If file does not exist or is not a file
PathSecurityError – If path fails security validation
OSError – If file cannot be read
UnicodeDecodeError – If file cannot be decoded with given encoding

Return type:

int

Example

>>> line_count = count_lines("data.csv")
>>> print(f"File has {line_count} lines")

siege_utilities.files.operations.copy_file(source, destination, overwrite=False)[source]

Copy a file from source to destination.

SECURITY: Validates both source and destination paths.

Parameters:

source (str | Path) – Source file path
destination (str | Path) – Destination file path
overwrite (bool) – Whether to overwrite existing files

Raises:

FileNotFoundError – If source does not exist or is not a file
FileExistsError – If destination exists and overwrite=False
PathSecurityError – If paths fail security validation
OSError – If copy fails

Return type:

None

Example

>>> copy_file("source.txt", "backup/source.txt")
>>> copy_file(Path("config.yaml"), Path("config.yaml.bak"))

siege_utilities.files.operations.move_file(source, destination, overwrite=False)[source]

Move a file from source to destination.

SECURITY: Validates both source and destination paths.

Parameters:

source (str | Path) – Source file path
destination (str | Path) – Destination file path
overwrite (bool) – Whether to overwrite existing files

Raises:

FileNotFoundError – If source does not exist or is not a file
FileExistsError – If destination exists and overwrite=False
PathSecurityError – If paths fail security validation
OSError – If move fails

Return type:

None

Example

>>> move_file("temp.txt", "archive/temp.txt")
>>> move_file(Path("old.log"), Path("logs/old.log"))

siege_utilities.files.operations.get_file_size(file_path)[source]

Get the size of a file in bytes.

SECURITY: This function validates paths to prevent path traversal attacks and access to sensitive system files.

Parameters:

file_path (str | Path) – Path to the file

Returns:

File size in bytes

Raises:

FileNotFoundError – If file does not exist or is not a file
PathSecurityError – If path fails security validation
OSError – If file size cannot be determined

Return type:

int

Example

>>> size = get_file_size("large_file.zip")
>>> print(f"File size: {size} bytes")

siege_utilities.files.operations.list_directory(path, pattern='*', include_dirs=True, include_files=True)[source]

List contents of a directory with optional filtering.

SECURITY: Validates directory path to prevent traversal attacks.

Parameters:

path (str | Path) – Directory path to list
pattern (str) – Glob pattern for filtering files
include_dirs (bool) – Whether to include directories
include_files (bool) – Whether to include files

Returns:

List of Path objects

Raises:

FileNotFoundError – If directory does not exist or is not a directory
PathSecurityError – If path fails security validation
OSError – If directory cannot be read

Return type:

List[Path]

Example

>>> files = list_directory("data", "*.csv")
>>> dirs = list_directory("logs", include_files=False)

siege_utilities.files.operations.run_command(command, cwd=None, timeout=30, capture_output=True, allow_list=None, unsafe=False)[source]

Run an argv command with optional command and argument validation.

SECURITY: This function now validates commands by default. Only whitelisted commands are allowed unless unsafe=True is explicitly set. Validation rejects selected characters and known sensitive paths; it is not a sandbox and does not guarantee read-only behavior.

Parameters:

command (str | List[str]) – Command to run (string or list)
cwd (str | Path | None) – Working directory for execution and relative operand validation
timeout (int | None) – Timeout in seconds (default: 30)
capture_output (bool) – Whether to capture command output
allow_list (set | None) – Optional custom whitelist of allowed commands
unsafe (bool) – If True, bypass command and argument validation. The command is still executed via argv (shell=False) — string commands are shlex-split and run without shell expansion. Use this when you legitimately need a binary that isn’t in the allow-list, not when you need shell features.

Returns:

CompletedProcess object on success or non-zero exit. Arguments and output are omitted from logs. Raised diagnostics omit command details and captured output, which may contain credentials.

Raises:

SecurityError – If command fails security validation (when unsafe=False).
subprocess.TimeoutExpired – If the command exceeds timeout.
OSError – If the command binary cannot be found or executed.

Return type:

CompletedProcess

Example

>>> result = run_command("ls -la")
>>> if result and result.returncode == 0:
...     print("Command succeeded")
>>> result = run_command("rm -rf /")
>>> result = run_command("git status", allow_list={'git', 'ls'})
>>> result = run_command("custom_cmd", unsafe=True)

Security Changes:

Previously used shell=True with no validation (CRITICAL VULNERABILITY)
Now validates all commands against whitelist by default
Rejects selected metacharacters, ‘..’, and known sensitive paths
Use unsafe=True ONLY with trusted input in trusted environments

siege_utilities.files.operations.ensure_directory_exists(directory)[source]

Ensure a directory exists, creating it if necessary.

Parameters:: directory (str | Path) – Path to directory
Returns:: Path object for the directory
Raises:: PathSecurityError – If path fails security validation.
Return type:: Path

siege_utilities.files.operations.safe_file_write(file_path, content, encoding='utf-8', create_dirs=True)[source]

Atomically write content to a file, optionally creating parent directories.

Parameters:

file_path (str | Path) – Path to file
content (str) – Content to write
encoding (str) – File encoding (default: utf-8)
create_dirs (bool) – Create parent directories if they don’t exist

Returns:

True if successful, False otherwise

Raises:

PathSecurityError – If path fails security validation.

Return type:

bool

siege_utilities.files.operations.safe_file_read(file_path, encoding='utf-8', default=None)[source]

Safely read content from a file.

Parameters:

file_path (str | Path) – Path to file
encoding (str) – File encoding (default: utf-8)
default (str | None) – Default value to return if file doesn’t exist or error occurs

Returns:

File content or default value

Raises:

PathSecurityError – If path fails security validation.

Return type:

str | None

siege_utilities.files.operations.safe_json_write(file_path, data, indent=2, create_dirs=True)[source]

Atomically write data to a JSON file.

Parameters:

file_path (str | Path) – Path to JSON file
data (Any) – Data to write (must be JSON serializable)
indent (int) – JSON indentation (default: 2)
create_dirs (bool) – Create parent directories if they don’t exist

Returns:

True if successful, False otherwise

Raises:

PathSecurityError – If path fails security validation.

Return type:

bool

siege_utilities.files.operations.safe_json_read(file_path, default=None)[source]

Safely read data from a JSON file.

Parameters:

file_path (str | Path) – Path to JSON file
default (Any | None) – Default value to return if file doesn’t exist or error occurs

Returns:

Parsed JSON data or default value

Raises:

PathSecurityError – If path fails security validation.

Return type:

Any | None

siege_utilities.files.operations.get_file_size_mb(file_path)[source]

Get file size in megabytes.

Parameters:

file_path (str | Path) – Path to file

Returns:

File size in MB.

Raises:

FileNotFoundError – If the file does not exist.
PathSecurityError – If path fails security validation.
OSError – If the file size cannot be determined.

Return type:

float

siege_utilities.files.operations.list_files_recursive(directory, pattern='*', exclude_dirs=True)[source]

List all files in a directory recursively.

Parameters:

directory (str | Path) – Directory to search
pattern (str) – Glob pattern (default: “*” for all files)
exclude_dirs (bool) – Exclude directories from results

Returns:

List of Path objects

Raises:

FileNotFoundError – If the directory does not exist.
PathSecurityError – If path fails security validation.
OSError – If the directory cannot be read.

Return type:

list[Path]

siege_utilities.files.operations.rmtree(path)

Remove a file or directory tree safely.

SECURITY: Validates paths to prevent removing sensitive system files.

Parameters:

path (str | Path) – Path to file or directory to remove

Raises:

FileNotFoundError – If path does not exist
PathSecurityError – If path fails security validation
OSError – If removal fails

Return type:

None

Example

>>> remove_tree("temp_directory")
>>> remove_tree(Path("old_files"))

siege_utilities.files.operations.check_if_file_exists_at_path(path)[source]

Check if a file exists at the specified path.

Deprecated since version 3.19.0: Use file_exists() instead. Will be removed in v4.0.0.

Parameters:: path (str | Path)
Return type:: bool

siege_utilities.files.operations.delete_existing_file_and_replace_it_with_an_empty_file(file_path, create_parents=True)[source]

Backward compatibility function that deletes existing file and replaces it with empty file.

Deprecated since version 3.19.0: Use touch_file() for creating empty files. Note that touch_file does not delete the existing file first; if you need delete-then-create semantics, call Path.unlink() before touch_file(). Will be removed in v4.0.0.

SECURITY: Validates paths to prevent path traversal attacks.

Parameters:

file_path (str | Path) – Path to the file
create_parents (bool) – Whether to create parent directories

Raises:

PathSecurityError – If path fails security validation
OSError – If file creation fails

Return type:

None

siege_utilities.files.operations.count_total_rows_in_file_pythonically(file_path, encoding='utf-8')

Count the total number of lines in a text file.

SECURITY: This function validates paths to prevent path traversal attacks and access to sensitive system files.

Parameters:

file_path (str | Path) – Path to the text file
encoding (str) – File encoding to use

Returns:

Number of lines

Raises:

FileNotFoundError – If file does not exist or is not a file
PathSecurityError – If path fails security validation
OSError – If file cannot be read
UnicodeDecodeError – If file cannot be decoded with given encoding

Return type:

int

Example

>>> line_count = count_lines("data.csv")
>>> print(f"File has {line_count} lines")

Functions

siege_utilities.files.operations.copy_file(source, destination, overwrite=False)[source]

Copy a file from source to destination.

SECURITY: Validates both source and destination paths.

Parameters:

source (str | Path) – Source file path
destination (str | Path) – Destination file path
overwrite (bool) – Whether to overwrite existing files

Raises:

FileNotFoundError – If source does not exist or is not a file
FileExistsError – If destination exists and overwrite=False
PathSecurityError – If paths fail security validation
OSError – If copy fails

Return type:

None

Example

>>> copy_file("source.txt", "backup/source.txt")
>>> copy_file(Path("config.yaml"), Path("config.yaml.bak"))

siege_utilities.files.operations.move_file(source, destination, overwrite=False)[source]

Move a file from source to destination.

SECURITY: Validates both source and destination paths.

Parameters:

source (str | Path) – Source file path
destination (str | Path) – Destination file path
overwrite (bool) – Whether to overwrite existing files

Raises:

FileNotFoundError – If source does not exist or is not a file
FileExistsError – If destination exists and overwrite=False
PathSecurityError – If paths fail security validation
OSError – If move fails

Return type:

None

Example

>>> move_file("temp.txt", "archive/temp.txt")
>>> move_file(Path("old.log"), Path("logs/old.log"))

siege_utilities.files.operations.remove_tree(path)[source]

Remove a file or directory tree safely.

SECURITY: Validates paths to prevent removing sensitive system files.

Parameters:

path (str | Path) – Path to file or directory to remove

Raises:

FileNotFoundError – If path does not exist
PathSecurityError – If path fails security validation
OSError – If removal fails

Return type:

None

Example

>>> remove_tree("temp_directory")
>>> remove_tree(Path("old_files"))

siege_utilities.files.operations.ensure_directory_exists(directory)[source]

Ensure a directory exists, creating it if necessary.

Parameters:: directory (str | Path) – Path to directory
Returns:: Path object for the directory
Raises:: PathSecurityError – If path fails security validation.
Return type:: Path

siege_utilities.files.operations.list_directory(path, pattern='*', include_dirs=True, include_files=True)[source]

List contents of a directory with optional filtering.

SECURITY: Validates directory path to prevent traversal attacks.

Parameters:

path (str | Path) – Directory path to list
pattern (str) – Glob pattern for filtering files
include_dirs (bool) – Whether to include directories
include_files (bool) – Whether to include files

Returns:

List of Path objects

Raises:

FileNotFoundError – If directory does not exist or is not a directory
PathSecurityError – If path fails security validation
OSError – If directory cannot be read

Return type:

List[Path]

Example

>>> files = list_directory("data", "*.csv")
>>> dirs = list_directory("logs", include_files=False)

Usage Examples

Basic file operations:

import siege_utilities

# Copy a file
siege_utilities.copy_file('source.txt', 'destination.txt')

# Move a file
siege_utilities.move_file('old_name.txt', 'new_name.txt')

# Delete a file
siege_utilities.remove_tree('unwanted_file.txt')

# Create directory
siege_utilities.ensure_directory_exists('new_folder')

# List files in directory
files = siege_utilities.list_directory('data_folder')
print(f"Found {len(files)} files")

Batch file processing:

import os

# Process multiple files
source_dir = 'input_files'
output_dir = 'processed_files'

# Create output directory
siege_utilities.ensure_directory_exists(output_dir)

# Process all files
for filename in siege_utilities.list_directory(source_dir):
    if filename.endswith('.txt'):
        source_path = os.path.join(source_dir, filename)
        dest_path = os.path.join(output_dir, f"processed_{filename}")
        siege_utilities.copy_file(source_path, dest_path)

Unit Tests

The file operations module has comprehensive test coverage:

✅ test_file_operations.py - All file operation tests pass

Test Coverage:
- File copying with various scenarios
- File moving and renaming
- File deletion with safety checks
- Directory creation and management
- File listing and discovery
- Error handling for missing files
- Permission handling

Test Results: All file operation tests pass successfully with comprehensive coverage.