gravitino.client.generic_fileset.GenericFileset¶

class gravitino.client.generic_fileset.GenericFileset(fileset: FilesetDTO, rest_client: HTTPClient, full_namespace: Namespace)¶

Bases: Fileset, SupportsCredentials

__init__(fileset: FilesetDTO, rest_client: HTTPClient, full_namespace: Namespace)¶

Methods

`__init__`(fileset, rest_client, full_namespace)
`audit_info`()
`comment`()	Returns:
`get_credential`(credential_type)	Retrieves Credential object based on the specified credential type.
`get_credentials`()	Retrieves a List of Credential objects.
`name`()	Returns:
`properties`()	Returns:
`storage_location`()	Get the unnamed storage location of the file or directory path managed by this fileset object.
`storage_locations`()	Get the storage location name and corresponding path of the file or directory path that is managed by this fileset object.
`support_credentials`()
`type`()	@Returns:

Attributes

`LOCATION_NAME_UNKNOWN`	The reserved location name to indicate the location name is unknown.
`PROPERTY_CATALOG_PLACEHOLDER`	The reserved property name for the catalog name placeholder, when creating a fileset, all placeholders as {{catalog}} will be replaced by the catalog name
`PROPERTY_DEFAULT_LOCATION_NAME`	The property name for the default location name of the fileset.
`PROPERTY_FILESET_PLACEHOLDER`	The reserved property name for the fileset name placeholder, when creating a fileset, all placeholders as {{fileset}} will be replaced by the fileset name
`PROPERTY_LOCATION_PLACEHOLDER_PREFIX`	The prefix of fileset placeholder property
`PROPERTY_MULTIPLE_LOCATIONS_PREFIX`	The prefix of the location name in the property at the catalog/schema level.
`PROPERTY_SCHEMA_PLACEHOLDER`	The reserved property name for the schema name placeholder, when creating a fileset, all placeholders as {{schema}} will be replaced by the schema name

LOCATION_NAME_UNKNOWN = 'unknown'¶: The reserved location name to indicate the location name is unknown.

PROPERTY_CATALOG_PLACEHOLDER = 'placeholder-catalog'¶: The reserved property name for the catalog name placeholder, when creating a fileset, all placeholders as {{catalog}} will be replaced by the catalog name

PROPERTY_DEFAULT_LOCATION_NAME = 'default-location-name'¶: The property name for the default location name of the fileset.

PROPERTY_FILESET_PLACEHOLDER = 'placeholder-fileset'¶: The reserved property name for the fileset name placeholder, when creating a fileset, all placeholders as {{fileset}} will be replaced by the fileset name

PROPERTY_LOCATION_PLACEHOLDER_PREFIX = 'placeholder-'¶: The prefix of fileset placeholder property

PROPERTY_MULTIPLE_LOCATIONS_PREFIX = 'location-'¶: The prefix of the location name in the property at the catalog/schema level.

PROPERTY_SCHEMA_PLACEHOLDER = 'placeholder-schema'¶: The reserved property name for the schema name placeholder, when creating a fileset, all placeholders as {{schema}} will be replaced by the schema name

class Type(value)¶

Bases: Enum

An enum representing the type of the fileset object.

EXTERNAL = 'external'¶: Fileset is not managed by Gravitino. When specified, the data will not be deleted when the fileset object is deleted

MANAGED = 'managed'¶: Fileset is managed by Gravitino. When specified, the data will be deleted when the fileset object is deleted

comment() → str | None¶

Returns:: The comment of the fileset object. Null is returned if no comment is set.

get_credential(credential_type: str) → Credential¶

Retrieves Credential object based on the specified credential type.

Args:: credential_type: The type of the credential like s3-token, s3-secret-key which are defined in the specific credentials.
Returns:: An Credential object with the specified credential type.
Raises:: NoSuchCredentialException If the specific credential cannot be found. IllegalStateException if multiple credential can be found.

get_credentials() → List[Credential]¶

Retrieves a List of Credential objects.

Returns:: A List of Credential objects. In most cases the array only contains

one credential. If the object like Fileset contains multiple locations for different storages like HDFS, S3, the array will contain multiple credentials. The array could be empty if you request a credential for a catalog but the credential provider couldn’t generate the credential for the catalog, like S3 token credential provider only generate credential for the specific object like Fileset,Table. There will be at most one credential for one credential type.

name() → str¶

Returns:: Name of the fileset object.

properties() → Dict[str, str]¶

Returns:: The properties of the fileset object. Empty map is returned if no properties are set.

storage_location() → str¶

Get the unnamed storage location of the file or directory path managed by this fileset object.

The returned storageLocation can be either the one specified when creating the fileset object (using storageLocation field or storageLocations field), or the one specified at the catalog/schema level (using property “location” or properties with prefix “location-”) if the fileset object is created under this catalog/schema.

The storageLocation at each level can contain placeholders, formatted as {{name}}, which will be replaced by the corresponding fileset property value when the fileset object is created. The placeholder property in the fileset object is formed as “placeholder-{{name}}”. For example, if the storageLocation is “file:///path/{{schema}}-{{fileset}}-{{version}}”, and the fileset object “catalog1.schema1.fileset1” has the property “placeholder-version” set to “v1”, then the storageLocation will be “file:///path/schema1-fileset1-v1”.

For managed fileset, the storageLocation can be:

The one specified when creating the fileset object, and the placeholders in the storageLocation will be replaced by the placeholder value specified in the fileset properties.
When catalog property “location” is specified but schema property “location” is not specified, then the storageLocation will be:
1. “{catalog location}/schemaName/filesetName” - if {catalog location} has no placeholders
2. “{catalog location}” - placeholders in {catalog location} will be replaced by values specified in fileset properties
When catalog property “location” is not specified but schema property “location” is specified, then the storageLocation will be:
1. “{schema location}/filesetName” - if {schema location} has no placeholders
2. “{schema location}” - placeholders in {schema location} will be replaced by values specified in fileset properties
When both catalog property “location” and schema property “location” are specified, then the storageLocation will be:
1. “{schema location}/filesetName” - if {schema location} has no placeholders
2. “{schema location}” - placeholders in {schema location} will be replaced by values specified in fileset properties

5) null value - when catalog property “location”, schema property “location”, storageLocation field of fileset, and “unknown” location in storageLocations are not specified.

For external fileset, the storageLocation can be: 1) The one specified when creating the fileset object, and the placeholders in the

storageLocation will be replaced by the placeholder value specified in the fileset properties.

Returns:: str: The storage location of the fileset object.

storage_locations() → Dict[str, str]¶

Get the storage location name and corresponding path of the file or directory path that is managed by this fileset object. The key is the name of the storage location and the value is the storage location path.

Each storageLocation in the values can either be the one specified when creating the fileset object, or the one specified in the catalog / schema level if the fileset object is created under this catalog / schema.

The “unknown” location name is reserved to indicate the default storage location of the fileset. It can be specified in catalog / schema level by the property “location” or in the fileset level by the field “storageLocation”. Other location names can be specified in the fileset level by the key-value pairs in the field “storageLocations”, and by “location-{name}” properties in the catalog / schema level.

For managed fileset, the storageLocation can be:

The one specified when creating the fileset object, and the placeholders in the storageLocation will be replaced by the placeholder value specified in the fileset properties.
When catalog property “location” is specified but schema property “location” is not specified, then the storageLocation will be:
1. “{catalog location}/schemaName/filesetName” - if {catalog location} has no placeholders
2. “{catalog location}” - placeholders in {catalog location} will be replaced by values specified in fileset properties
When catalog property “location” is not specified but schema property “location” is specified, then the storageLocation will be:
1. “{schema location}/filesetName” - if {schema location} has no placeholders
2. “{schema location}” - placeholders in {schema location} will be replaced by values specified in fileset properties
When both catalog property “location” and schema property “location” are specified, then the storageLocation will be:
1. “{schema location}/filesetName” - if {schema location} has no placeholders
2. “{schema location}” - placeholders in {schema location} will be replaced by values specified in fileset properties
When there is no location specified in catalog level, schema level, storageLocation of fileset, and storageLocations of fileset at the same time, this situation is illegal.

For external fileset, the storageLocation can be: 1) The one specified when creating the fileset object, and the placeholders in the

storageLocation will be replaced by the placeholder value specified in the fileset properties.

Returns:: The storage locations of the fileset object, the key is the name of the storage

location and the value is the storage location path.

type() → Type¶

@Returns:: The type of the fileset object.