File Mover Options

FileMoverOptions

File moving options.

Added in 0.8.0

Examples:

from onetl.file import FileMover

options = FileMover.Options(
    if_exists="replace_entire_directory",
    workers=4,
)

Parameters:

• if_exists

  (FileExistBehavior, default: <FileExistBehavior.ERROR: 'error'> ) –

  How to handle existing files in the local directory.

  Possible values:

  • error (default) - mark file as failed
  • ignore - mark file as skipped
  • replace_file - replace existing file with a new one
  • replace_entire_directory - delete directory content before moving files

  Added in 0.8.0

  Changed in 0.9.0

  Renamed mode → if_exists

• workers

  (int, default: 1 ) –

  Number of workers to create for parallel file moving.

  1 (default) means files will me moved sequentially. 2 or more means files will be moved in parallel workers.

  Recommended value is min(32, os.cpu_count() + 4), e.g. 5.

  Added in 0.8.1