Base interface

class onetl.base.base_file_df_connection.BaseFileDFConnection

Implements generic methods for reading and writing dataframe as files

abstract check() → T

Check source availability.

If not, an exception will be raised.

Returns:

Connection itself

Raises:

RuntimeError

If the connection is not available

Examples

connection.check()

abstract check_if_format_supported(format: BaseReadableFileFormat | BaseWritableFileFormat) → None

Validate if specific file format is supported.

Raises:

RuntimeError

If file format is not supported.

abstract read_files_as_df(paths: list[PurePathProtocol], format: BaseReadableFileFormat, root: PurePathProtocol | None = None, df_schema: StructType | None = None, options: FileDFReadOptions | None = None) → DataFrame

Read files in some paths list as dataframe.

abstract write_df_as_files(df: DataFrame, path: PurePathProtocol, format: BaseWritableFileFormat, options: FileDFWriteOptions | None = None) → None

Write dataframe as files in some path.