MySQL connection

Bases: JDBCConnection

MySQL JDBC connection.

Based on Maven package com.mysql:mysql-connector-j:9.7.0 (official MySQL JDBC driver).

See also

Before using this connector please take into account MySQL prerequisites

Added in 0.1.0

Parameters:

• host (str) –

  Host of MySQL database. For example: mysql0012.domain.com or 192.168.1.11

• port (int, default: 3306 ) –

  Port of MySQL database

• user (str) –

  User, which have proper access to the database. For example: some_user

• password (str) –

  Password for database connection

• database (str) –

  Database (==schema) in MySQL

• spark (SparkSession) –

  Spark session.

• extra (dict, default: None ) –

  Specifies one or more extra parameters by which clients can connect to the instance.

  For example: {"useSSL": "false", "allowPublicKeyRetrieval": "true"}

  See MySQL JDBC driver properties documentation for more details

Examples:

Create and check MySQL connection:

from onetl.connection import MySQL
from pyspark.sql import SparkSession

# Create Spark session with MySQL driver loaded
maven_packages = MySQL.get_packages()
spark = (
    SparkSession.builder.appName("spark-app-name")
    .config("spark.jars.packages", ",".join(maven_packages))
    .getOrCreate()
)

# Create connection
mysql = MySQL(
    host="database.host.or.ip",
    user="user",
    password="*****",
    extra={"useSSL": "false", "allowPublicKeyRetrieval": "true"},
    spark=spark,
).check()

get_packages(package_version=None) classmethod

Get package names to be downloaded by Spark. Allows specifying a custom JDBC driver version for MySQL.

Added in 0.9.0

Parameters:

• package_version (str, default: None ) –

  Specifies the version of the MySQL JDBC driver to use. Defaults to 9.7.0.

  Added in 0.11.0

Examples:

from onetl.connection import MySQL

MySQL.get_packages()

# specify a custom package version
MySQL.get_packages(package_version="8.2.0")

check()

Check source availability.

If not, an exception will be raised.

Returns:

• Self –

  Connection itself

Raises:

• RuntimeError –

  If the connection is not available

Examples:

connection.check()