REST Catalog¶
IcebergRESTCatalog
¶
Iceberg REST Catalog.
Added in 0.15.0
Parameters:
-
(url¶HttpUrl) –REST catalog server URL
-
(headers¶dict[str, Any], default:<class 'dict'>) –Additional HTTP headers to include in requests
-
(extra¶dict[str, Any], default:<class 'dict'>) –Additional configuration parameters
-
(auth¶IcebergRESTCatalogAuth | None, default:None) –Authentication configuration
Examples:
from onetl.connection import Iceberg
catalog = Iceberg.RESTCatalog(
url="https://rest.domain.com:8080",
auth=Iceberg.RESTCatalog.BasicAuth(
user="my_user",
password="my_password",
),
)
from onetl.connection import Iceberg
catalog = Iceberg.RESTCatalog(
url="https://rest.domain.com:8080",
auth=Iceberg.RESTCatalog.BearerAuth(
access_token="my_bearer_token",
),
)
from onetl.connection import Iceberg
catalog = Iceberg.RESTCatalog(
url="https://rest.domain.com:8080",
auth=Iceberg.RESTCatalog.OAuth2ClientCredentials(
client_id="my_client_id",
client_secret="my_client_secret",
),
)
from onetl.connection import Iceberg
catalog = Iceberg.RESTCatalog(
url="https://rest.domain.com:8080",
headers={
"X-Custom-Auth": "my_custom_token",
"X-Request-ID": "request-123",
},
extra={
"timeout": "30s",
"retry": "3",
},
)
"""
These options will be passed to Spark config:
spark.sql.my_catalog.uri = "https://rest.domain.com:8080"
spark.sql.my_catalog.header.X-Custom-Auth = "my_custom_token"
spark.sql.my_catalog.header.X-Request-ID = "request-123"
spark.sql.my_catalog.timeout = "30s"
spark.sql.my_catalog.retry = "3"
"""