Python os.posix_spawn Function

Last modified April 11, 2025

This comprehensive guide explores Python’s os.posix_spawn function, a POSIX-compliant process creation mechanism. We’ll cover file actions, spawn attributes, and practical examples for efficient process spawning.

Basic Definitions

The os.posix_spawn function creates a new process executing a specified program. It’s more efficient than fork-exec on some systems.

Key parameters: path (program to execute), argv (argument list), env (environment), file_actions (file operations), and spawn attributes (process characteristics).

Basic Process Creation

The simplest use of os.posix_spawn launches a program with given arguments. This example runs the ’ls’ command to list directory contents.

basic_spawn.py

import os

Basic process creation

pid = os.posix_spawn( “/bin/ls”, [“ls”, “-l”, “/tmp”], os.environ )

Wait for the process to complete

_, status = os.waitpid(pid, 0) print(f"Process {pid} exited with status {status}")

This creates a new process running ’ls -l /tmp’. The parent waits for the child to complete using os.waitpid. The environment is inherited from the parent.

Note that the first argument in argv should match the program name (convention).

Custom Environment Variables

You can specify custom environment variables when spawning a process. This example sets a special environment for the child process.

custom_env.py

import os

Create custom environment

custom_env = os.environ.copy() custom_env[“MODE”] = “DEBUG” custom_env[“LOG_LEVEL”] = “VERBOSE”

Spawn process with custom environment

pid = os.posix_spawn( “/usr/bin/env”, [“env”], custom_env )

Wait for process

_, status = os.waitpid(pid, 0) print(f"Process completed with status {status}")

The child process runs ’env’ which displays its environment. We see our custom variables MODE and LOG_LEVEL in the output.

Environment variables are passed as a dictionary. Always copy os.environ first to avoid modifying the parent’s environment.

File Actions - Redirecting Output

File actions allow modifying file descriptors before program execution. This example redirects stdout to a file.

redirect_output.py

import os

Prepare file actions

file_actions = os.posix_spawn_file_actions() file_actions.addopen(1, “output.txt”, os.O_WRONLY | os.O_CREAT, 0o644) file_actions.addclose(2) # Close stderr

Spawn process with file actions

pid = os.posix_spawn( “/bin/ls”, [“ls”, “-l”, “/”], os.environ, file_actions=file_actions )

os.waitpid(pid, 0) print(“Output written to output.txt”)

This redirects stdout (fd 1) to output.txt and closes stderr (fd 2). The ls output goes to the file instead of the terminal.

File actions must be created before spawning and can include open, close, and dup2 operations.

Spawn Attributes - Process Group

Spawn attributes control process characteristics. This example creates a new process group for the child process.

process_group.py

import os

Prepare spawn attributes

spawn_attrs = os.posix_spawnattr() spawn_attrs.setflags(os.POSIX_SPAWN_SETPGROUP) spawn_attrs.setpgroup(0) # Create new process group

Spawn process with attributes

pid = os.posix_spawn( “/bin/sleep”, [“sleep”, “10”], os.environ, spawn_attrs=spawn_attrs )

print(f"Created process {pid} in new process group") os.waitpid(pid, 0)

The child process (sleep) runs in its own process group. This is useful for process management and signal handling.

Attributes can control signal handling, scheduling, and other process characteristics.

Combining File Actions and Attributes

This example demonstrates using both file actions and spawn attributes to create a process with customized characteristics.

combined_features.py

import os

File actions - redirect stdout/stderr

file_actions = os.posix_spawn_file_actions() file_actions.addopen(1, “out.log”, os.O_WRONLY | os.O_CREAT, 0o644) file_actions.adddup2(1, 2) # Redirect stderr to stdout

Spawn attributes - new process group

spawn_attrs = os.posix_spawnattr() spawn_attrs.setflags(os.POSIX_SPAWN_SETPGROUP) spawn_attrs.setpgroup(0)

Spawn process

pid = os.posix_spawn( “/bin/bash”, [“bash”, “-c”, “echo ‘Output’; echo ‘Error’ >&2”], os.environ, file_actions=file_actions, spawn_attrs=spawn_attrs )

os.waitpid(pid, 0) print(“Both output and error written to out.log”)

The bash command’s output and error streams are redirected to out.log. The process runs in its own process group.

This shows how to combine multiple features of posix_spawn for advanced process control.

Error Handling

Proper error handling is crucial when spawning processes. This example shows how to handle various spawn failures.

error_handling.py

import os import errno

try: # Attempt to spawn non-existent program pid = os.posix_spawn( “/nonexistent/program”, [“program”], os.environ ) except OSError as e: if e.errno == errno.ENOENT: print(“Program not found”) elif e.errno == errno.EACCES: print(“Permission denied”) else: print(f"Spawn failed: {e}")

try: # Invalid file action file_actions = os.posix_spawn_file_actions() file_actions.addopen(1, “/root/protected”, os.O_WRONLY, 0o644) pid = os.posix_spawn( “/bin/true”, [“true”], os.environ, file_actions=file_actions ) except PermissionError: print(“Cannot open protected file”)

The first attempt fails because the program doesn’t exist. The second fails due to permission issues with file actions.

Always handle potential errors when spawning processes, especially with file operations or privileged paths.

Signal Handling

Spawn attributes can control signal handling in the child process. This example resets signal handlers to defaults.

signal_handling.py

import os import signal

Prepare spawn attributes for signal handling

spawn_attrs = os.posix_spawnattr() spawn_attrs.setsigdefault({signal.SIGINT: signal.SIG_DFL}) spawn_attrs.setflags(os.POSIX_SPAWN_SETSIGDEF)

Spawn process that ignores Ctrl-C

pid = os.posix_spawn( “/bin/bash”, [“bash”, “-c”, “echo ‘Running…’; sleep 10; echo ‘Done’”], os.environ, spawn_attrs=spawn_attrs )

print(f"Process {pid} running - Ctrl-C will terminate it") os.waitpid(pid, 0)

The child process resets SIGINT to default behavior, making it interruptible with Ctrl-C. Without this, it might inherit parent’s signal handlers.

Signal handling attributes are particularly important for long-running or interactive child processes.

Security Considerations

• Path security: Validate executable paths to prevent command injection

• Environment sanitization: Carefully control environment variables

• File descriptor handling: Close unnecessary FDs in the child

• Privilege separation: Consider dropping privileges in child

• Error handling: Handle all possible spawn failures

Best Practices

• Use absolute paths: For both executable and file actions

• Clean environments: Pass only necessary environment variables

• Resource limits: Set appropriate limits via spawn attributes

• Signal management: Explicitly set signal handling

• Error checking: Validate all inputs before spawning

Source References

• Python os.posix_spawn Documentation

• POSIX spawn man page

Author

My name is Jan Bodnar, and I am a passionate programmer with extensive programming experience. I have been writing programming articles since 2007. To date, I have authored over 1,400 articles and 8 e-books. I possess more than ten years of experience in teaching programming.

List all Python tutorials.