Executing statements in Hive#

Hive.execute(statement: str) → None#

Execute DDL or DML statement. support_hooks

Parameters:
statementstr

Statement to be executed, like:

DML statements:

  • INSERT INTO target_table SELECT * FROM source_table

  • TRUNCATE TABLE mytable

DDL statements:

  • CREATE TABLE mytable (...)

  • ALTER TABLE mytable ...

  • DROP TABLE mytable

  • MSCK REPAIR TABLE mytable

The exact list of supported statements depends on Hive version, for example some new versions support CREATE FUNCTION syntax.

Examples

Create table:

connection = Hive(cluster="rnd-dwh", spark=spark)

connection.execute(
    "CREATE TABLE mytable (id NUMBER, data VARCHAR) PARTITIONED BY (date DATE)"
)

Drop table partition:

connection = Hive(cluster="rnd-dwh", spark=spark)

connection.execute("ALTER TABLE mytable DROP PARTITION(date='2023-02-01')")