MySQL Catalog

Introduction

Apache Gravitino provides the ability to manage MySQL metadata.

caution

Gravitino saves some system information in schema and table comment, like (From Gravitino, DO NOT EDIT: gravitino.v1.uid1078334182909406185), do not change or remove this message.

Catalog

Catalog Capabilities

• Gravitino catalog corresponds to the MySQL instance.
• Supports metadata management of MySQL (5.7, 8.0).
• Supports DDL operation for MySQL databases and tables.
• Supports table index.
• Supports column default value and auto-increment.
• Supports managing MySQL table features through table properties, like using engine to set MySQL storage engine.

Catalog Properties

Pass to a MySQL data source any property that isn't defined by Gravitino by adding gravitino.bypass. prefix as a catalog property. For example, catalog property gravitino.bypass.maxWaitMillis will pass maxWaitMillis to the data source property.

Check the relevant data source configuration in data source properties

When using Gravitino with Trino, pass the Trino MySQL connector configuration using the trino.bypass. prefix. For example, using trino.bypass.join-pushdown.strategy to pass the join-pushdown.strategy to the Gravitino MySQL catalog in Trino runtime.

If you use a JDBC catalog, you must provide jdbc-url, jdbc-driver, jdbc-user and jdbc-password to catalog properties. Besides the common catalog properties, the MySQL catalog has the following properties:

Configuration item	Description	Default value	Required
jdbc-url	JDBC URL for connecting to the database. For example, jdbc:mysql://localhost:3306	(none)	Yes
jdbc-driver	The driver of the JDBC connection. For example, com.mysql.jdbc.Driver or com.mysql.cj.jdbc.Driver.	(none)	Yes
jdbc-user	The JDBC user name.	(none)	Yes
jdbc-password	The JDBC password.	(none)	Yes
jdbc.pool.min-size	The minimum number of connections in the pool. 2 by default.	2	No
jdbc.pool.max-size	The maximum number of connections in the pool. 10 by default.	10	No
jdbc.pool.max-wait-ms	The maximum Duration that the pool will wait for a connection to be returned. 30000 by default.	30000	No

caution

Gravitino does not package the MySQL JDBC driver, because MySQL Connector/J is GPLv2 and cannot be redistributed. You must supply it yourself.

Download MySQL Connector/J (com.mysql:mysql-connector-j, 8.0.16 or later; the com.mysql.cj.jdbc.Driver class) from Maven Central and place the JAR in the catalogs/jdbc-mysql/libs directory.

For container or Kubernetes deployments where you cannot copy into that directory directly, supply the driver through your deployment's mechanism for adding catalog libraries. The catalog fails fast at creation time with a clear message if the driver is absent.

Driver Version Compatibility

The MySQL catalog includes driver version compatibility checks for datetime precision calculation:

• MySQL Connector/J versions >= 8.0.16: Full support for datetime precision calculation
• MySQL Connector/J versions < 8.0.16: Limited support - datetime precision calculation returns null with a warning log

This limitation affects the following datetime types:

• TIME(p) - time precision
• TIMESTAMP(p) - timestamp precision
• DATETIME(p) - datetime precision

When using an unsupported driver version, the system will:

1. Continue to work normally with default precision (0)
2. Log a warning message indicating the driver version limitation
3. Return null for precision calculations to avoid incorrect results

Example warning log:

WARN: MySQL driver version mysql-connector-java-8.0.11 is below 8.0.16, 
columnSize may not be accurate for precision calculation. 
Returning null for TIMESTAMP type precision. Driver version: mysql-connector-java-8.0.11

Recommended driver versions:

• mysql-connector-java-8.0.16 or higher

Catalog Operations

Refer to Manage Catalogs and Schemas for more details.

note

Sensitive catalog properties such as jdbc-user and jdbc-password are hidden from the load catalog response. Use the credential vending API to retrieve them at runtime.

Schema

Schema Capabilities

• Gravitino's schema concept corresponds to the MySQL database.
• Supports creating schema, but does not support setting comment.
• Supports dropping schema.
• Supports cascade dropping schema.

Schema Properties

• Doesn't support any schema property settings.

Schema Operations

Refer to Manage Catalogs and Schemas for more details.

Table

Table Capabilities

• Gravitino's table concept corresponds to the MySQL table.
• Supports DDL operation for MySQL tables.
• Supports index.
• Supports column default value and auto-increment..
• Supports managing MySQL table features through table properties, like using engine to set MySQL storage engine.

Table Column Types

Gravitino Type	MySQL Type
Byte	Tinyint
Unsigned Byte	Tinyint Unsigned
Short	Smallint
Unsigned Short	Smallint Unsigned
Integer	Int
Unsigned Integer	Int Unsigned
Long	Bigint
Unsigned Long	Bigint Unsigned
Float	Float
Double	Double
String	Text
Date	Date
Time[(p)]	Time[(p)]
Timestamp_tz[(p)]	Timestamp(p)
Timestamp[(p)]	Datetime[(p)]
Decimal	Decimal
VarChar	VarChar
FixedChar	FixedChar
Binary	Binary
BOOLEAN	BIT

info

MySQL doesn't support Gravitino Fixed Struct List Map IntervalDay IntervalYear Union UUID type. Meanwhile, the data types other than listed above are mapped to Gravitino External Type that represents an unresolvable data type.

Table Column Auto-Increment

note

MySQL setting an auto-increment column requires simultaneously setting a unique index; otherwise, an error will occur.

JSON

{
  "columns": [
    {
      "name": "id",
      "type": "integer",
      "comment": "id column comment",
      "nullable": false,
      "autoIncrement": true
    },
    {
      "name": "name",
      "type": "varchar(500)",
      "comment": "name column comment",
      "nullable": true,
      "autoIncrement": false
    }
  ],
  "indexes": [
    {
      "indexType": "primary_key",
      "name": "PRIMARY",
      "fieldNames": [["id"]]
    }
  ]
}

Java

Column[] cols = new Column[] {
    Column.of("id", Types.IntegerType.get(), "id column comment", false, true, null),
    Column.of("name", Types.VarCharType.of(500), "Name of the user", true, false, null)
};
Index[] indexes = new Index[] {
    Indexes.of(IndexType.PRIMARY_KEY, "PRIMARY", new String[][]{{"id"}}, Map.of())
};

Table Properties

Although MySQL itself does not support table properties, Gravitino offers table property management for MySQL tables through the jdbc-mysql catalog, enabling control over table features. The supported properties are listed as follows:

note

Reserved: Fields that cannot be passed to the Gravitino server.

Immutable: Fields that cannot be modified once set.

caution

• Doesn't support remove table properties. You can only add or modify properties, not delete properties.

Property Name	Description	Default Value	Required	Reserved	Immutable
engine	The engine used by the table. For example MyISAM, MEMORY, CSV, ARCHIVE, BLACKHOLE, FEDERATED, ndbinfo, MRG_MYISAM, PERFORMANCE_SCHEMA.	InnoDB	No	No	Yes
auto-increment-offset	Used to specify the starting value of the auto-increment field.	(none)	No	No	Yes

note

Some MySQL storage engines, such as FEDERATED, are not enabled by default and require additional configuration to use. For example, to enable the FEDERATED engine, set federated=1 in the MySQL configuration file. Similarly, engines like ndbinfo, MRG_MYISAM, and PERFORMANCE_SCHEMA may also require specific prerequisites or configurations. For detailed instructions, refer to the MySQL documentation.

Table Indexes

• Supports PRIMARY_KEY and UNIQUE_KEY.

note

The index name of the PRIMARY_KEY must be PRIMARY Create table index

JSON

{
  "indexes": [
    {
      "indexType": "primary_key",
      "name": "PRIMARY",
      "fieldNames": [["id"]]
    },
    {
      "indexType": "unique_key",
      "name": "id_name_uk",
      "fieldNames": [["id"] ,["name"]]
    }
  ]
}

Java

Index[] indexes = new Index[] {
    Indexes.of(IndexType.PRIMARY_KEY, "PRIMARY", new String[][]{{"id"}}, Map.of()),
    Indexes.of(IndexType.UNIQUE_KEY, "id_name_uk", new String[][]{{"id"} , {"name"}}, Map.of()),
};

Table Operations

Refer to Manage Relational Metadata Using Gravitino for more details.

Alter Table Operations

Gravitino supports these table alteration operations:

• RenameTable
• UpdateComment
• AddColumn
• DeleteColumn
• RenameColumn
• UpdateColumnType
• UpdateColumnPosition
• UpdateColumnNullability
• UpdateColumnComment
• UpdateColumnDefaultValue
• SetProperty

info

• You cannot submit the RenameTable operation at the same time as other operations.
• If you update a nullability column to non-nullability, there may be compatibility issues.