MySQL connection#
- class onetl.connection.db_connection.mysql.connection.MySQL(*, spark: SparkSession, user: str, password: SecretStr, host: Host, port: int = 3306, database: str | None = None, extra: MySQLExtra = MySQLExtra(useUnicode='yes', characterEncoding='UTF-8'))#
-
Based on Maven package
com.mysql:mysql-connector-j:8.0.33
(official MySQL JDBC driver).Warning
Before using this connector please take into account Prerequisites
- Parameters:
- hoststr
Host of MySQL database. For example:
mysql0012.domain.com
or192.168.1.11
- portint, default:
3306
Port of MySQL database
- userstr
User, which have proper access to the database. For example:
some_user
- passwordstr
Password for database connection
- databasestr
Database in RDBMS, NOT schema.
See this page for more details
- spark
pyspark.sql.SparkSession
Spark session.
- extradict, default:
None
Specifies one or more extra parameters by which clients can connect to the instance.
For example:
{"useSSL": "false", "allowPublicKeyRetrieval": "true"}
See MySQL JDBC driver properties documentation for more details
Examples
MySQL connection initialization
from onetl.connection import MySQL from pyspark.sql import SparkSession # Create Spark session with MySQL driver loaded maven_packages = MySQL.get_packages() spark = ( SparkSession.builder.appName("spark-app-name") .config("spark.jars.packages", ",".join(maven_packages)) .getOrCreate() ) # Create connection mysql = MySQL( host="database.host.or.ip", user="user", password="*****", extra={"useSSL": "false", "allowPublicKeyRetrieval": "true"}, spark=spark, )
- check()#
-
If not, an exception will be raised.
- Returns:
- Connection itself
- Raises:
- RuntimeError
If the connection is not available
Examples
connection.check()
- classmethod get_packages() list[str] #
Get package names to be downloaded by Spark.
Examples
from onetl.connection import MySQL MySQL.get_packages()